Data Quality and Labeling Specialist
Collate
Data Quality and Labeling Specialist
Collate
AI-driven documentation automation for life sciences
We're looking for a Data Quality and Labeling Specialist to bring life-sciences expertise to the data behind our AI. Our models draft, extract from, and review regulated documents across drug, diagnostic, and medical device development, and this role makes sure the data they learn from and are measured against is accurate, consistent, and grounded in how regulated work is actually done.
This is a hands-on role at the intersection of domain expertise and data quality. You'll work side by side with our AI researchers and engineers, and your judgment about what a correct document looks like will shape the product directly.
We are 5 days a week onsite in San Francisco.
Review and label regulatory, quality, and clinical documents and model-generated outputs for accuracy, completeness, and compliance.
Build and maintain gold-standard datasets used to train and evaluate our models.
Write and maintain labeling guidelines and evaluation rubrics as new document types and use cases are added.
Run calibration with other reviewers to keep labels consistent and resolve disagreements.
Audit labeled data before it reaches training and evaluation, and track label quality over time.
Turn recurring model errors into clear, actionable feedback for the research and engineering teams.
Experience in a regulated life-sciences function such as regulatory affairs, quality, clinical, or CMC, at a pharma, biotech, diagnostics, or medical device company.
Familiarity with the documents and standards that govern that work (e.g., FDA, EMA, ICH, or ISO requirements).
Strong attention to detail: you notice an unsupported claim or inconsistent value and can explain why it matters.
Ability to apply structured rubrics consistently and to flag when a rubric itself needs changing.
Clear written communication.
Prior experience with data annotation, model evaluation, or writing guidelines for other reviewers is a plus.
Excitement to work in an early-stage environment on-site in San Francisco.
Impact: The documents you help get right are the ones that put drugs, diagnostics, and devices in front of patients sooner.
Ownership: You define what "correct" means for a new category of AI product and own the data that enforces it, from day one.
Learning: Work daily with AI researchers, engineers, and life-sciences leaders; see how your labels change model behavior within the week.
Upside: Join early enough for meaningful equity and a role that grows with the data function—into leading a team of reviewers and contract SMEs as we scale.
Collate provides an AI-powered platform that automates and streamlines the creation of documentation for drug development, medical devices, and diagnostics, reducing time and resources spent on regulatory paperwork. The platform auto-generates, organizes, and governs documents while offering data discovery and observability to ensure high-quality, compliant data assets. It differentiates itself by delivering end-to-end documentation and data governance for life sciences in one platform that covers drugs, devices, and diagnostics, minimizing manual work. Its goal is to accelerate bringing life-saving innovations to patients by shortening development and regulatory timelines so researchers can focus on discovery.
Explore more opportunities at Collate