RLHF Evaluator (Canada, bilingual an asset)
Corpshore AI's RLHF Evaluator role is a remote contract position open across Canada, comparing and ranking AI model outputs against a rubric to produce LLM alignment data, with French fluency valued as a bilingual asset.
From anywhere in Canada, you'll join Corpshore AI's RLHF evaluation team, comparing and ranking AI model outputs against a structured rubric to produce the preference data foundation-model teams use for LLM alignment. French fluency is an asset for projects needing bilingual English/French coverage.
Key responsibilities
- Compare pairs of model outputs against a written rubric and record structured preference judgments in Jwuma
- Write concise rationale notes justifying ranking decisions
- Flag unsafe, biased, or low-quality outputs per project guidelines
- Support bilingual English/French evaluation batches where projects require it
- Maintain consistency against gold-standard calibration sets
What we look for
- Excellent English reading comprehension and written communication; French fluency an asset
- Strong, consistent judgment on nuanced or subjective content comparisons
- Reliable internet and a quiet remote work environment
- Legally eligible to work as a contractor in Canada
- Comfortable with flexible, deadline-driven remote work
Nice to have
- Background in writing, editing, translation, or research
- Prior content-review, QA, or BPO experience
- Interest in AI/LLM technology as a user
Compensation and benefits
- Hourly contract pay (CAD), paid via Jwuma timesheets
- Fully remote, flexible scheduling across Canadian time zones
- Paid rubric and platform onboarding
- Ongoing project flow with bilingual-project priority for French speakers
Common questions
Do I need to be bilingual to apply for this role?
No, but French fluency is an asset that opens up additional bilingual English/French project assignments.
Is this role open across all of Canada?
Yes, it's a remote, contract role open to applicants anywhere in Canada with reliable internet.
Similar roles
Apply once, get matched to what fits
One application covers every role you qualify for. A person reads it.
Apply now