A

Anthropic

Builds Claude AI models with a focus on safety

Staff Software Engineer - Safeguards Evals

AI Application EngineeringConfirmed live in the last 24 hours
Full-time$320k - $485k /yr

About the Job

Requirements

  • Experience building and maintaining data pipelines.
  • Experience working with large language models and a working understanding of their capabilities and failure modes, especially agentic systems with tool use and multi-step reasoning.
  • Strong data analysis skills, including the ability to draw reliable insights from large datasets.
  • Ability to move fluidly between research prototyping and production-quality code.
  • Ability to translate ambiguous problems into concrete, testable experiments.
  • Bachelor’s degree or an equivalent combination of education, training, and/or experience.
  • A field of study relevant to the role, as demonstrated through coursework, training, or professional experience.

Responsibilities

  • Design and run experiments to improve evaluation quality, including developing methods to generate representative test data, simulate realistic user behavior, and validate grading accuracy.
  • Research how multi-turn conversations, tools, long context, and user diversity affect model safety behavior.
  • Analyze evaluation coverage to identify measurement gaps, and evolve evaluations so they remain unsaturated and high-signal as model and agent capabilities advance.
  • Build and own the evaluation harness for an agentic investigation system by defining metrics, test cases, and grading approaches for a complex, long-horizon agent.
  • Measure agent performance end-to-end, including detection precision and recall, investigation quality, and robustness, and drive hill-climbing on the hardest harm areas.
  • Construct reinforcement learning environments to improve Claude’s safety investigation capabilities.
  • Construct high-quality evaluation datasets representing real-world misuse across harm areas such as cyber attacks, biological weapons, and influence operations, drawing from real traffic patterns and synthetic generation.
  • Collaborate with Policy and Enforcement to translate observed harm patterns into measurable evaluations.
  • Ship successful research into evaluation, regression, and release pipelines that run during model training, on every agent change, prompt update, underlying model upgrade, and beyond launch.
  • Build tooling that enables policy experts to author, run, and iterate on evaluations without engineering support.
  • Surface findings to research and training teams to drive upstream model improvements.

Desired Qualifications

  • At least 8 years of industry software engineering or machine learning engineering experience.
  • Expertise in building or contributing to large language model or agent evaluation frameworks, benchmarks, or automated grading systems.
  • Extensive experience in trust and safety, content moderation, or abuse detection systems.
  • Experience in red teaming, adversarial testing, or jailbreak research on artificial intelligence systems.
  • Experience with synthetic data generation or data augmentation.
  • Experience with distributed systems or large-scale data processing.
  • Experience with prompt engineering or building large-language-model-powered applications.

About the Company

Anthropic

Builds Claude AI models with a focus on safety

Anthropic builds the Claude family of AI models and sells them to businesses, software developers, and consumers, with research on making AI safe and understandable at the center of its work. In 2024 it introduced the Model Context Protocol, an open standard for connecting AI assistants to outside apps and data that OpenAI and Google adopted the next year. People use the Claude app to write and analyze files, programmers hand coding tasks to Claude Code, and companies build Claude into their own products through its API. Anthropic earns money from subscriptions, business plans, and usage-based API fees. Siblings Dario and Daniela Amodei founded Anthropic in 2021 with other former OpenAI employees, and it is based in San Francisco.

Company size
5,001-10,000
Company stage
IPO
Headquarters
San Francisco, CA
Founded
2021

Recently Posted Jobs

Explore more opportunities at Anthropic

See more jobs