AI Annotation Quality Reviewer

About the Role

We are seeking a detail-oriented QA Review Team Lead to oversee quality assurance activities for AI training data and ensure annotated outputs are accurate, consistent, and delivered on time. In this role, you will lead QA reviewers, manage audit and review queues, assess annotation quality, identify recurring errors, and provide clear, actionable feedback to improve team performance. You will also train new reviewers, clarify guidelines for annotators, and escalate complex or recurring issues to the Senior QA Lead.

The ideal candidate combines strong quality assurance skills with experience mentoring team members and applying detailed guidelines in a fast-paced environment.

CONTRACT: Contractor assignment, 16 weeks

COMMITMENT: Hourly, up to 40h/week

LOCATIONS: Remote - Brazil, Colombia, India

Key Responsibilities

• Lead QA reviewers and manage audit and review queues to ensure timely and accurate completion of assigned work.
• Spot-check annotated data and reviewer decisions to verify compliance with project guidelines and quality standards.
• Provide clear, actionable, and consistent feedback to reviewers and annotators.
• Conduct office hours and clarify annotation guidelines, review criteria, and quality expectations.
• Monitor quality scores, error trends, audit results, and reviewer performance.
• Train and onboard new reviewers on project guidelines, tools, and review procedures.
• Identify recurring errors, inconsistencies, and process gaps, and recommend corrective actions.
• Escalate persistent quality issues, unclear guidelines, and complex cases to the Senior QA Lead.
• Support continuous improvement of review processes and maintain consistent quality across the team.

Requirements

• 3–5 years of overall professional experience, including at least 1 year of experience leading, mentoring, or supervising reviewers or team members.
• Strong attention to detail and the ability to apply complex guidelines consistently.
• Excellent written and verbal communication skills.
• Ability to provide constructive, actionable feedback and explain quality expectations clearly.
• Strong organizational skills and the ability to manage multiple review queues and competing deadlines.
• Ability to identify patterns in errors and support corrective actions.
• Availability to work the specified schedule for the full 16-week project.