AI Response Evaluator (Remote)
Crewlia
What the role involves
- Evaluating AI outputs
- Comparing similar examples
- Recording evaluation findings
What they ask for
At least 18 years old; able to communicate in English; able to evaluate AI outputs consistently against supplied criteria; able to keep evaluation status aligned with completed judgments; reliable high-speed internet and suitable home-working setup
About the job
- Role: AI Response Evaluator (Remote)
- Location: Remote (Australia, Canada, Ireland, New Zealand, United Kingdom, United States)
- Pay: US$18 per hour
Applicants must currently reside in Australia, Canada, Ireland, New Zealand, the United Kingdom or the United States.
Overview
We are recruiting contributors to evaluate AI outputs for AI projects. You will use the supplied evaluation criteria to guide your work and record findings about accuracy, reasoning, relevance, and instruction-following issues for review.
Key Responsibilities
- Evaluate AI outputs against the supplied evaluation criteria.
- Compare similar examples before making close quality decisions.
- Identify the specific evidence that supports each judgment.
- Check that review status matches the latest completed judgment.
- Record findings about accuracy, reasoning, relevance, and instruction-following issues and flag unresolved questions for review.
Requirements
- Must be at least 18 years old and able to communicate in English.
- Ability to evaluate AI outputs consistently against the supplied evaluation criteria.
- Ability to keep evaluation status aligned with completed judgments.
- Reliable high-speed internet and a suitable home-working setup.
- Previous experience in AI output evaluation, content review or quality checking is helpful but not required.
Equal Opportunity
We welcome applications from qualified individuals of all backgrounds. Applicants are considered based on relevant skills, qualifications and project eligibility requirements, without discrimination on grounds protected by applicable law.
Apply Now!