L

Lilt

AI-powered multilingual translation platform for enterprises

Senior Quality Manager - Applied AI

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-timeNo salary listedHybrid

About the Job

Requirements

  • A B.S. or M.S. in a quantitative field such as Computer Science, Statistics, Data Science, Engineering, Linguistics, or Human-Computer Interaction, or equivalent practical experience.
  • At least 6 years of experience building or operating quality programs for machine learning or large language model systems, including data labeling, benchmark creation, model evaluation, or human-in-the-loop pipelines.
  • Ability to design and run end-to-end quality frameworks, including rubrics, sampling plans, quality-control gates, acceptance criteria, and escalation paths, across multiple concurrent programs.
  • Strong grasp of reliability and agreement methods such as Krippendorff’s alpha and Cohen’s kappa, power and sample-size intuition, error analysis, and reporting for executive stakeholders.
  • Experience with calibration, adjudication, targeted retraining, drift monitoring, and integrity or fraud detection in high-throughput human evaluation programs.
  • Hands-on experience with labeling or evaluation tooling such as Label Studio or similar, designing deterministic validations, and partnering with engineering or research to implement quality instrumentation.
  • Strong understanding of dataset and version management, traceability, privacy and compliance constraints, and audit-ready documentation for deliverables.
  • Excellent written and verbal communication, ability to translate ambiguous research goals into measurable quality targets, and ability to negotiate tradeoffs with Applied AI, product, and external partners.
  • Authorization to work in the United States is required.

Responsibilities

  • Ensure every Applied AI deliverable ships with measured quality, including rater agreement meeting agreed thresholds per dimension, reproducible rubric scores across raters, and reported quality figures that withstand customer audit.
  • Own technical tooling and quality infrastructure so that quality metrics for every active program—including defect rate, acceptance rate, rework, reviewer reliability, throughput, and service-level agreement adherence—are available without manual compilation, and systemic quality issues are detected from workflow data before customers report them.
  • Own operational quality at scale so concurrent programs meet quality targets within agreed cost and throughput, acceptance rates meet or exceed targets, rework declines over time, and Production staff can execute and interpret standard quality control on standardized workflows without Quality Manager involvement by the end of the second quarter in role.
  • Make a documented go/no-go decision for every deliverable before shipping, based on programmatic quality assurance and a readiness memo with flags; keep post-sign-off quality escapes below an agreed threshold and document process changes for each escape.
  • Own customer quality remediation by closing disputes with quantitative analysis, normally at first response; completing re-adjudication within service-level agreements; reducing recurring defect classes release over release; and detecting annotator integrity issues before they reach a deliverable.

Desired Qualifications

  • Fluency in multiple human languages.
  • Proficiency in a programming language and experience working with modern artificial intelligence frameworks.

About the Company

Lilt

AI-powered multilingual translation platform for enterprises

Lilt provides an AI-powered translation platform for enterprises to manage multilingual content. It uses a Contextual AI Engine with a human-in-the-loop: AI translations are reviewed by professional linguists, and feedback helps the model improve. It supports 70+ languages and formats and integrates with 100+ enterprise systems, sold as a subscription SaaS (plus direct translation services). Its goal is to speed up high-volume translation while ensuring accuracy and consistency across global content.

Company size
501-1,000
Company stage
Series C
Total funding
$95.6M
Headquarters
San Francisco, CA
Founded
2015

Recently Posted Jobs

Explore more opportunities at Lilt

See more jobs