V

Vals AI

Independent AI model evaluation platform

Member of Technical Staff - Research, AI Safety

AI ResearchUpdated on Nov 11
Full-timeSan Francisco, CA, USA

About the Job

Requirements

  • Substantial hands-on experience in safety evaluations, monitorability, scheming, deception, or multi-agent safety; this may come from an AI safety or alignment team, safety research publications, or high-quality write-ups.
  • Understanding of neural networks and the fundamentals of model training.
  • Strong coding skills and the ability to independently design and build complex evaluation environments, primarily in Python.
  • Ability to communicate complex research ideas clearly to technical and non-technical audiences.
  • Experience with mechanistic interpretability is a nice-to-have.
  • Contributions to open-source safety or evaluation tools and datasets are a nice-to-have.
  • Ability to learn new skills and technologies quickly.
  • Initiative and autonomy in a small team, including building what is needed.
  • A solution-oriented approach to crafting solutions to difficult problems.

Responsibilities

  • Design and develop novel, high-impact benchmarks and environments to assess real-world behavior and misbehavior—including reward hacking, scheming, deception, and other misaligned behavior—in frontier models and agents, including multi-agent systems.
  • Conduct research to ensure benchmarks are valid, reliable, and meaningful.
  • Collaborate with foundation model labs and enterprises to understand evolving safety evaluation needs.
  • Analyze model behavior across benchmarks and articulate results.
  • Publish research findings and contribute to the broader safety and evaluation research communities.
  • Stay current with developments in large language model capabilities and alignment evaluation methodologies.

About the Company

Vals AI

Independent AI model evaluation platform

Vals AI provides independent evaluation and benchmarking for large language models and AI applications with domain-specific leaderboards. It tests models on real-world tasks using private datasets to prevent test leakage and reports metrics such as tool use, multi-modality, long-context reasoning, accuracy, latency, and cost, while offering an evaluation framework for private labs and enterprise teams and public benchmarks like Finance Agent and Valkyrie. The company differentiates itself by delivering unbiased, practitioner-focused measurements built with domain experts and by focusing on economically valuable, complex professional tasks rather than academic problems. Its goal is to become the trusted measurement layer for the AI economy, helping research labs and enterprises make informed decisions about AI model adoption and deployment.

Company size
11-50
Company stage
Series A
Total funding
$40M
Headquarters
San Francisco, CA
Founded
2023

Recently Posted Jobs

Explore more opportunities at Vals AI

See more jobs