Join to apply for the Senior Code Reviewer for LLM Data Training (Python) role at SME Work 3 days ago Be among the first 25 applicants Join to apply for the Senior Code Reviewer for LLM Data Training (Python) role at SME Work Get AI-powered advice on this job and more exclusive features. Direct message the job poster from SME Work Talent Acquisition | Ex-Invisible | Hired 6,000 Annotators Globally About Company SME is a platform that bridges subject-matter experts with AI projects, enabling them to contribute their knowledge to improve AI models. It offers flexible opportunities to work on tasks like data labeling, quality assurance, and domain-specific problem-solving while earning competitive pay. About the Role We’re hiring a Code Reviewer with deep Python expertise to review evaluations completed by data annotators assessing AI-generated Python code responses. Your role is to ensure that annotators follow strict quality guidelines related to instruction-following, factual correctness, and code functionality. Responsibilities - Review and audit annotator evaluations of AI-generated Python code. - Assess if the Python code follows the prompt instructions, is functionally correct, and secure. - Validate code snippets using proof-of-work methodology. - Identify inaccuracies in annotator ratings or explanations. - Provide constructive feedback to maintain high annotation standards. - Work within Project Atlas guidelines for evaluation integrity and consistency. Required Qualifications - 5–7+ years of experience in Python development, QA, or code review. - Strong knowledge of Python syntax, debugging, edge cases, and testing. - Comfortable using code execution environments and testing tools. - Excellent written communication and documentation skills. - Experience working with structured QA or annotation workflows. - English proficiency at B2, C1, C2, or Native level. Preferred Qualifications - Experience in AI training, LLM evaluation, or model alignment. - Familiarity with annotation platforms. - Exposure to RLHF (Reinforcement Learning from Human Feedback) pipelines. Compensation : - $35 Hourly Why Join Us? Join a high-impact team working at the intersection of AI and software development. Your Python expertise will directly influence the accuracy, safety, and clarity of AI-generated code. This role offers remote flexibility, milestone-based delivery, and competitive compensation. Seniority level - Seniority level Mid-Senior level Employment type - Employment type Contract Job function - Job function Engineering, Information Technology, and Quality Assurance - Industries Technology, Information and Internet Referrals increase your chances of interviewing at SME Work by 2x Get notified about new Software Engineer jobs in Medellín, Antioquia, Colombia . Senior Front End Web Developer (Global-Remote-Non.US) Fullstack Engineer (work from home in Medellin, Colombia) Software Engineer - Front-end/ Full stack Senior Software Engineer (Python) | Remote | Startup Transforming legal tech with AI Developer Relations Engineer - Crypto & AI (Global - Remote - Non.US) Mid Full Stack Developer (Angular, .NET & Python) We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr Required Skill Profession Desarrollo De Software