S

Scale AI

Supplies AI training data and builds AI apps

Research Scientist - AI Safety and Alignment

AI ResearchUpdated on Nov 10
Full-timeSan Francisco, CA, USA

About the Job

Requirements

  • Practical experience conducting technical research collaboratively, including building and instrumenting machine-learning pipelines, building evaluation harnesses, running experiments on frontier models and agents, and translating research-literature ideas into working prototypes.
  • Familiarity with current research in at least one of the listed AI safety and alignment areas.
  • A track record of published machine-learning research, particularly in AI safety and alignment.
  • At least three years of experience working on sophisticated machine-learning problems in research or product development.
  • Strong written and verbal communication skills, including explaining technical findings to policymakers and working across functions.

Responsibilities

  • Design and stress-test control protocols and monitoring systems that detect and limit unsafe behavior by AI models and agents in deployment.
  • Build alignment evaluations that probe for deception, sabotage, scheming, reward hacking, and other misaligned behaviors.
  • Study monitorability, including whether a model’s reasoning stays faithful and legible enough to supervise, and how training methods and deployment practices can preserve or erode it.
  • Investigate risks that emerge when multiple AI agents interact, such as collusion, coordination failures, and steerability.
  • Develop threat models and evaluations for AI systems that automate AI research and development, to track progress toward recursive self-improvement and inform preparedness planning.
  • Work with government agencies and other labs to scope and design evaluations that measure and mitigate risks from advanced AI.
  • Publish research and evaluation methodologies, and write technical reports for policymakers.

Desired Qualifications

  • Experience designing evaluations and benchmarks, or building tools and pipelines to analyze model and agent trajectories at scale.
  • Experience with red-teaming or adversarial testing of AI systems, including stress-testing safeguards such as monitors and control protocols.
  • Familiarity with AI safety policy and governance frameworks, including the NIST AI Risk Management Framework, EU AI Act, Korea AI Basic Act, and frontier AI safety frameworks.

About the Company

Scale AI

Supplies AI training data and builds AI apps

Scale AI supplies the data that AI labs use to train and test their models, and it builds custom AI applications for companies and government agencies such as the Pentagon. It first made its name labeling camera and sensor data for self-driving car companies like Waymo, then worked with OpenAI on InstructGPT, the 2022 forerunner of ChatGPT. Freelancers, some of them doctors and lawyers, write, label, and grade examples through its Outlier platform, and labs pay Scale for this data and for tests of model quality. Clients pay Scale to build AI tools on their own data, and some of those tools run on classified military networks. Alexandr Wang and Lucy Guo founded Scale AI in 2016, and it is based in San Francisco.

Company size
1,001-5,000
Company stage
Acquired
Total funding
$1.6B
Headquarters
San Francisco, CA
Founded
2016

Recently Posted Jobs

Explore more opportunities at Scale AI

See more jobs