Senior Machine Learning Data Engineer, Annotation Tooling

Corpshore AI's engineering team builds and operates Jwuma, the proprietary annotation and workforce platform that runs 150M+ delivered annotations across 15,000+ specialists in 18+ countries. As Senior ML Data Engineer, Annotation Tooling, you'll design the pipelines, QA-automation systems, and dataset-delivery infrastructure that keep accuracy above 97% at global scale.

Key responsibilities

• Design and maintain data pipelines that move annotation batches through Jwuma's three-tier QA cascade at scale
• Build tooling for gold-set benchmarking, consensus scoring, and inter-annotator agreement analysis
• Partner with QA, operations, and client delivery teams to translate labeling requirements into platform features
• Own reliability, performance, and data-integrity for annotation and dataset-delivery pipelines
• Mentor junior engineers and contribute to platform architecture decisions

What we look for

• 5+ years of experience in data engineering, ML infrastructure, or backend engineering, ideally involving labeled/annotated datasets
• Strong proficiency in Python and SQL; experience with cloud data infrastructure (AWS/GCP/Azure)
• Experience designing systems for data quality, consistency scoring, or large-scale human-in-the-loop workflows
• Excellent written communication for cross-functional collaboration with global, distributed teams
• Eligible to work remotely in the United States

Nice to have

• Direct experience building or operating annotation/labeling platforms
• Familiarity with RLHF, preference-data, or LLM evaluation pipelines
• Experience with multilingual or low-resource-language data systems

Compensation and benefits

• Competitive base salary plus performance bonus and equity-style incentives where applicable
• Fully remote within the US with flexible hours
• Health benefits per US employment standards
• Direct collaboration with Corpshore AI's global platform, QA, and operations leadership

Common questions