A

Ambience Healthcare

Generative AI platform for clinician workflow

Staff Machine Learning Engineer - Frontier AI

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USA$244k - $305k /yr

About the Job

Requirements

  • Have 7+ years of experience in production machine learning, research engineering, or applied artificial intelligence.
  • Have deep expertise with modern large language models, transformers, and complex production artificial intelligence systems.
  • Have led or materially shaped consequential artificial intelligence systems with real production impact.
  • Be able to reason across models, data, orchestration, evaluation, serving, and product behavior.
  • Have significant experience designing evaluations for large language models, agents, or other complex artificial intelligence systems.
  • Be able to turn ambiguous product-quality problems into robust metrics, datasets, and experiments.
  • Have experience with grader bias, leakage, contamination, misleading aggregate metrics, regression detection, and offline-to-online mismatch.
  • Know how to establish whether a model or system improvement translates into better user outcomes.
  • Have built or architected production systems involving multiple models, tools, retrieval, context and state management, routing, orchestration, tracing, and failure recovery.
  • Understand how to evaluate and debug complex agent behavior across model, tool, and system boundaries.
  • Be comfortable making architectural decisions that span multiple products or teams.
  • Have materially improved model behavior through post-training or model adaptation techniques such as supervised fine-tuning, preference optimization, reinforcement learning, distillation, synthetic data, or related approaches.
  • Understand how data quality, objective design, evaluation, and post-training choices affect downstream model behavior.
  • Know when post-training is the right intervention versus prompting, retrieval, context optimization, routing, or changes to the surrounding system.
  • Have taken model improvements from experimentation through rigorous evaluation and production deployment.
  • Be proficient in Python and modern machine learning frameworks.
  • Be comfortable with modern MLOps, continuous integration and continuous deployment, observability, and containerized deployments.
  • Remain hands-on by writing code, inspecting traces, analyzing failures, and prototyping solutions.
  • Be skilled at building large, high-quality datasets and feedback loops.
  • Have experience using production failures, user feedback, and active learning to improve model and system quality.
  • Be able to design data and evaluation infrastructure that creates leverage across multiple teams.
  • Be able to work closely with clinicians, product managers, researchers, and engineers.
  • Have passion for healthcare or other mission-driven industries.
  • Thrive in a fast-paced, early-stage environment.
  • Take broad ownership of technical outcomes and raise the effectiveness of the people around them.

Responsibilities

  • Establish how production artificial intelligence quality is measured across large language model and agentic systems, including automated graders, regression testing, human evaluation, failure taxonomies, and offline-to-online validation.
  • Identify high-impact failure modes across products and lead improvements across prompting, retrieval, context, routing, data, and post-training; design experiments to determine the right intervention and verify improvements in production.
  • Shape how the company uses supervised fine-tuning, preference optimization, reinforcement learning, distillation, and synthetic data to improve model behavior.
  • Set technical direction for production systems involving tool use, retrieval, context and state management, routing, orchestration, tracing, and failure recovery.
  • Design systems that turn production failures and user feedback into better datasets, evaluations, and model behavior, creating scalable improvement flywheels across products.
  • Co-develop the 12-month technical roadmap, balancing near-term product wins with longer-term research and platform investments.
  • Lead technical reviews, mentor engineers, establish best practices, and create reusable systems and frameworks that improve the effectiveness of the broader artificial intelligence team.
  • Distill insights from recent research in large language models, agents, post-training, natural language processing, speech, and multimodal artificial intelligence, and drive experiments to advance clinical artificial intelligence.
  • Own model-quality problems across the clinical artificial intelligence suite, including coding models that navigate a proprietary million-term ontology with multi-objective precision, a scribe that learns from edit signals without introducing regressions, long-context chart understanding that stays faithful under clinical complexity, and population-level reasoning that surfaces auditable and actionable patterns across patients.

Desired Qualifications

  • Proficiency with PyTorch.
  • Experience with real-time voice, conversational artificial intelligence, or multimodal systems.
  • Prior work in healthcare, clinical artificial intelligence, or other regulated, high-stakes industries.
  • Experience interviewing or hiring machine learning engineers.
  • Open-source contributions to machine learning, agent, evaluation, or post-training tooling.

About the Company

Ambience Healthcare

Generative AI platform for clinician workflow

Ambience Healthcare provides AI-powered tools to support clinicians and streamline healthcare workflows. It develops an AI operating system and product suite that integrates generative AI into clinical documentation and administrative tasks, working with hospitals and clinics across North America through a subscription model. The software plugs into electronic health records (EHRs) to structure data for better reporting, research, and daily workflows. Unlike competitors that may offer standalone apps, Ambience Healthcare partners with large health systems to embed AI directly into clinical processes, focusing on reducing clinician burnout and boosting system efficiency while maintaining high-quality care. The company’s goal is to make healthcare workflows smoother and more efficient by embedding AI into everyday clinical operations.”} വ```

Company size
201-500
Company stage
Series C
Total funding
$343M
Headquarters
San Francisco, CA
Founded
2020

Recently Posted Jobs

Explore more opportunities at Ambience Healthcare

See more jobs