AI Quality Evaluator (Remote)

• Role: AI Quality Evaluator (Remote)
• Location: Remote (Australia, Canada, Ireland, New Zealand, United Kingdom, United States)
• Pay: US$18 per hour

Applicants must currently reside in Australia, Canada, Ireland, New Zealand, the United Kingdom or the United States.

Overview

We are recruiting quality-focused contributors to assess AI responses across accuracy, usefulness, clarity, and safety criteria. You will make balanced ratings, recognize high-impact defects, and keep decisions aligned with calibration guidance.

Key Responsibilities

• Evaluate each response on the full set of quality dimensions defined for the task.

• Distinguish minor presentation issues from errors that make an answer misleading or unusable.

• Apply severity levels to factual, instructional, reasoning, and communication failures.

• Use calibration examples to maintain consistent standards across different topics and response lengths.

• Document the most consequential strength and weakness that support the final quality outcome.

Requirements

• Must be at least 18 years old and able to communicate in English.

• Ability to weigh several quality dimensions while recognizing decisive errors.

• Ability to apply severity guidance consistently across varied AI responses.

• Reliable high-speed internet and a suitable home-working setup.

• Previous experience with AI quality evaluation, response review, or rubric scoring is helpful but not required.

Equal Opportunity

We welcome applications from qualified individuals of all backgrounds. Applicants are considered based on relevant skills, qualifications and project eligibility requirements, without discrimination on grounds protected by applicable law.

Apply Now!