S

Snorkel AI

Builds training data and test environments for AI

Research Lead - Open Benchmarks Grants

AI ResearchConfirmed live in the last 24 hours
Full-time$231k - $390kRemote

About the Job

Requirements

  • A track record of leading or substantially contributing to benchmarks, evaluation methods, or environments used by researchers or model-development teams.
  • Ability to assess whether evaluations measure their stated goals, identify alternative explanations for results, and design experiments to resolve uncertainty.
  • Ability to identify limitations that should change research direction.
  • Technical depth to build experiments, inspect tasks and harnesses, debug verifiers, and analyze model behavior.
  • Experience helping other researchers make difficult technical decisions and bring work to completion.
  • Ability to communicate complex ideas and precise claims through papers, research artifacts, specifications, or reviews.

Responsibilities

  • Identify gaps in frontier AI evaluation; review proposals with senior researchers; work with prospective collaborators to strengthen proposals; and help decide Open Benchmark Grants allocations.
  • Work with grantees, Research Fellows, and Snorkel research leads to define benchmark measurements, design tasks and experiments, and produce artifacts for other researchers.
  • Work with frontier-lab researchers to understand measurement needs, support integration and interpretation of benchmarks, and use findings to inform subsequent research.

About the Company

Snorkel AI

Builds training data and test environments for AI

Snorkel AI builds training datasets and simulated work environments for AI labs and companies developing models and software agents. Its data products combine expert demonstrations, scoring rules, and quality checks, while runnable environments let agents practice tasks such as coding or using business software. The company pairs domain specialists with its own data development technology to generate examples, evaluate model behavior, and refine weak areas through repeated feedback. Customers buy ready-made datasets or commission tailored data and evaluation work, and enterprises also use its specialized agent solutions. Snorkel AI was founded in 2019 and is headquartered in San Francisco, California.

Company size
201-500
Company stage
Late Stage VC
Total funding
$586.2M
Headquarters
San Francisco, CA
Founded
2019

Recently Posted Jobs

Explore more opportunities at Snorkel AI

See more jobs