A

Anthropic

Builds Claude AI models with a focus on safety

Research Engineer - Post-Training Model Evaluations

AI ResearchUpdated on Nov 7
Full-time$500k - $850k /yr

About the Job

Requirements

  • Have designed, run, and analyzed evaluations for machine-learning models at scale.
  • Have strong Python skills and be comfortable working with production systems.
  • Maintain clarity and rigor when debugging complex, time-sensitive issues.
  • Be able to turn ambiguous results into clear recommendations and influence direction across teams.
  • Have a field of study relevant to the role, demonstrated through coursework, training, or professional experience; a bachelor’s degree or an equivalent combination of education, training, and experience is the minimum education requirement.

Responsibilities

  • Steer the evaluation strategy for production Claude models by studying and optimizing the evaluation mix to provide an accurate and complete assessment of model quality.
  • Run and monitor the evaluation fleet live to understand how each Claude model develops during training.
  • Advise teams across post-training on evaluation methodology and help evaluation authors bring new evaluations up to production standards.
  • Investigate regressions in production runs and inform training interventions when appropriate.
  • Build dashboards, alerts, and reports that researchers and leadership use to track model quality.

Desired Qualifications

  • Have hands-on experience post-training large language models.
  • Have research experience in machine-learning evaluation or benchmarking.
  • Have a background in statistics and experimental design.
  • Have experience developing robust evaluation metrics for machine-learning systems.

About the Company

Anthropic

Builds Claude AI models with a focus on safety

Anthropic builds the Claude family of AI models and sells them to businesses, software developers, and consumers, with research on making AI safe and understandable at the center of its work. In 2024 it introduced the Model Context Protocol, an open standard for connecting AI assistants to outside apps and data that OpenAI and Google adopted the next year. People use the Claude app to write and analyze files, programmers hand coding tasks to Claude Code, and companies build Claude into their own products through its API. Anthropic earns money from subscriptions, business plans, and usage-based API fees. Siblings Dario and Daniela Amodei founded Anthropic in 2021 with other former OpenAI employees, and it is based in San Francisco.

Company size
5,001-10,000
Company stage
N/A
Total funding
$243.8B
Headquarters
San Francisco, CA
Founded
2021

Recently Posted Jobs

Explore more opportunities at Anthropic

See more jobs