S

Scale AI

Supplies AI training data and builds AI apps

Subject Matter Expert

QA & TestingConfirmed live in the last 24 hours
Part-timeNo salary listedIn-person

About the Job

Requirements

  • Deep domain expertise in a specialized field, such as medicine, law, or engineering.
  • Strong analytical skills to evaluate complex outputs against defined standards.
  • Ability to produce clear, structured, and actionable feedback.
  • High attention to detail and commitment to accuracy and consistency.

Responsibilities

  • Conduct structured audits of task outputs to evaluate accuracy, consistency, and adherence to project guidelines.
  • Apply domain expertise to assess nuanced or complex work requiring specialized knowledge.
  • Identify patterns of quality issues and flag systemic gaps impacting output standards.
  • Define and refine quality benchmarks, scoring rubrics, and evaluation criteria based on project needs.
  • Contribute to the development of gold standards and reference materials for high-quality outputs.
  • Ensure quality frameworks reflect domain-specific rigor where applicable.
  • Analyze audit results to generate actionable insights on quality trends and performance gaps.
  • Provide structured reports with clear recommendations to improve overall output quality.
  • Partner with internal stakeholders to calibrate on quality expectations and scoring consistency.
  • Recommend improvements to workflows, guidelines, and evaluation processes based on audit findings.
  • Support calibration efforts to ensure alignment across reviewers and quality stakeholders.
  • Contribute to continuous improvement initiatives strictly related to quality outcomes.

Desired Qualifications

  • Experience in quality assurance, auditing, or evaluation frameworks.

About the Company

Scale AI

Supplies AI training data and builds AI apps

Scale AI supplies the data that AI labs use to train and test their models, and it builds custom AI applications for companies and government agencies such as the Pentagon. It first made its name labeling camera and sensor data for self-driving car companies like Waymo, then worked with OpenAI on InstructGPT, the 2022 forerunner of ChatGPT. Freelancers, some of them doctors and lawyers, write, label, and grade examples through its Outlier platform, and labs pay Scale for this data and for tests of model quality. Clients pay Scale to build AI tools on their own data, and some of those tools run on classified military networks. Alexandr Wang and Lucy Guo founded Scale AI in 2016, and it is based in San Francisco.

Company size
1,001-5,000
Company stage
Acquired
Total funding
$1.6B
Headquarters
San Francisco, CA
Founded
2016

Recently Posted Jobs

Explore more opportunities at Scale AI

See more jobs