Human Data Evals Lead
About the job
- At least 5 years in technical delivery, quality, or program management, with recent experience in AI or machine-learning data, model evaluation, or benchmarking.
- Hands-on experience delivering data or evaluation work to AI labs or enterprise machine-learning teams, from scoping through delivery.
- Working fluency with frontier-model evaluation, including benchmarks, rubrics, pass rates, headroom, and what makes a task discriminate a model.
- Experience recruiting and calibrating a team or expert pool and holding it to a quality standard.
- Fluency in English.
- Own the company’s data initiatives and proposals to AI labs, from preparing proposals or responding to requests through pilot delivery.
- Design how proposals are built and develop sample packages and benchmarks across reasoning, coding, agents, tool use, and multimodal tasks, collaborating with subject-matter experts and ensuring expert-verified ground truth, multi-model headroom results, and buyer-ready quality control.
- Design samples that demonstrate quality and convert pilots into production engagements.
- Study public benchmarks and evaluation targets, and turn them into proposals and sample packages that demonstrate capability and win work.
- Respond to lab data requests and pilots.
- Design and build sample packages with subject-matter experts, ensuring expert-verified, exact-match-checkable ground truth and gold reasoning trajectories; multi-model evaluation demonstrating real headroom and task discrimination; and rigorous quality-control structures such as calibration layers, severity-weighted rubrics, deterministic verifiers, and evidence maps.
- Recruit, brief, calibrate, and review subject-matter experts across coding, agentic/tool-use, and STEM/reasoning; raise and maintain their output to the required standard and determine what constitutes correctness and frontier difficulty.
- Serve as a direct point of contact for lab partners on Slack and calls; keep senior lab contacts informed, identify their needs, and involve the CEO and subject-matter experts as appropriate.
- Own pilots end to end, including scoping, statements of work, staffing, production, quality control, and delivery; ensure work is lab-ready before shipment and understand in advance why any work might be rejected as not frontier-level.
- Experience originating data or benchmark proposals for AI labs, translating evaluation targets into sample tasks that demonstrate capability, and owning the engagement through delivery.
- Deep evaluation and quality expertise in large language model benchmarking, with strength in code-model evaluation.
- Experience building quality-control processes and artifact standards meeting enterprise or lab requirements, and setting a quality bar for a team of experts.
- Ability to thrive in ambiguous, fast-moving environments and deliver under pressure.
- Spanish language ability is a plus.
About the company
AnyoneAI helps people with no AI background start a career in AI/ML by offering an intensive, job-focused training program in the edtech space. The program includes 650+ hours of classroom-style and practical training plus 400+ hours of real-world ML project work, with ongoing mentorship to build skills and prepare for interviews. It operates on a pay-after-you-get-a-job model: students pay only after they secure employment in the AI field, making upfront tuition free. Revenue comes from the post-employment fees once a job is landed. The company also connects students with its partners to explore job opportunities. Compared to competitors, AnyoneAI aligns its revenue with student success and emphasizes practical project experience, mentorship, and career placement support for beginners transitioning into AI development roles. The goal is to help individuals transition into AI roles and place them in jobs, creating a clear path from training to employment.
Company Size
1,001-5,000
Company Stage
Seed
Total Funding
$1.5M
Headquarters
Wilmington, Delaware
Founded
2021
Get referred to Anyone AI
See people who can refer or advise you
Simplify's Take
What believers are saying
- The June 2026 job posting suggests active partner demand for AI lab services.
- Anyone AI’s 2026 website still advertises remote cohorts and continuous mentor access.
- AI hiring remains crowded in 2026, expanding job-seeker demand for reskilling programs.
What critics are saying
- Anyone AI’s last disclosed capital is the June 2022 $1.5 million pre-seed.
- The 2026 strategic project lead posting signals a tiny team and fragile execution.
- If employers hire fewer junior ML developers by 2027, revenue collapses with placements.
What makes Anyone AI unique
- Pay-after-job pricing lowers student risk and aligns outcomes with placements.
- Anyone AI targets Latin America with remote AI/ML training and English support.
- 650-hour curriculum plus 400 project hours creates unusually intensive workforce training.
Help us improve and share your feedback! Did you find this helpful?
Benefits
- Remote Work Options
- Flexible Work Hours