A

AfterQuery

LLM app development platform with observability

Quality Control Lead

AI & Machine LearningConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USAIn-person

About the Job

Requirements

  • At least 3 years of experience in a technical role such as software engineering, data science, or machine learning engineering, with hands-on proficiency in at least one modern programming language.
  • Ability to review, debug, and evaluate code, technical data, and knowledge-work data for correctness, structure, and quality, with demonstrated rigor in reviewing others' work.
  • Experience building or leading a quality assurance or quality control function, ideally for a technical or data-centric product.
  • High attention to detail and comfort making judgment calls on ambiguous or contested technical content.
  • Ability to translate technical quality issues into clear, actionable feedback across a distributed team.
  • Comfort operating with minimal direction in ambiguous, fast-moving environments.
  • Genuine passion for artificial intelligence and an entrepreneurial inclination.
  • Demonstrated competitive success.

Responsibilities

  • Own the final quality checkpoint before datasets are delivered or published, auditing outputs including code, technical data, and reasoning data for correctness, clarity, structure, and adherence to specification.
  • Work with infrastructure engineers to automate and continuously improve quality control workflows.
  • Partner with data delivery teams to identify systemic quality issues, feed corrections back into contributor training, and prevent recurrence.
  • Own quality metrics and key performance indicators, reporting on error rates, root causes, and improvement trends to company leadership.
  • Contribute to company-wide initiatives across building, analysis, coordination, and execution as the company scales.

Desired Qualifications

  • Experience in artificial intelligence training data, reinforcement learning from human feedback, or data-annotation quality assurance.
  • Background reviewing datasets across multiple domains, including code, reasoning, and knowledge work.
  • Experience designing rubrics, grading criteria, or automated quality-check tooling or scripts.
  • Familiarity with Claude Code or similar artificial intelligence-assisted workflow tools.
  • Computer science degree or equivalent hands-on engineering background.

About the Company

AfterQuery

LLM app development platform with observability

AfterQuery provides a data platform for developers to build, evaluate, and monitor LLM-powered applications. It connects to enterprise data sources like Notion, Slack, and databases and prepares data for use in Retrieval-Augmented Generation (RAG). It offers evaluation tools to test models before deployment and observability to monitor performance, costs, and issues such as hallucinations and prompt drift in production. Its end-to-end, data-centric workflow targets production-ready AI solutions for enterprises, backed by YC.

Company size
201-500
Company stage
Series A
Total funding
$30.1M
Headquarters
San Francisco, CA
Founded
2024

Recently Posted Jobs

Explore more opportunities at AfterQuery

See more jobs