A

Apple

Designs phones, computers and their operating systems

Senior Machine Learning Engineer - Conversational AI, Services Special Projects

Natural Language Processing & LLMsPosted on Nov 18
Full-timeCupertino, CA, USANo salary listedIn-person

About the Job

Requirements

  • A Master's degree in Computer Science, Machine Learning, NLP, or a related field.
  • At least 7 years of experience as a software or machine learning engineer, with a significant focus on conversational AI, search, or LLM-powered systems.
  • Strong programming skills in Go and Python, plus at least one of Java or Scala.
  • Hands-on experience building and shipping conversational AI or dialogue systems, including intent and slot understanding, multi-turn context management, clarification, and disambiguation.
  • Experience building production LLM-powered applications, including prompt design, tool or function calling, structured output generation, and agentic orchestration patterns.
  • Demonstrated LLM inference-optimization experience, including distillation, quantization, batching, KV-cache tuning, caching, and profiling to identify real bottlenecks.
  • Hands-on experience deploying and serving LLMs or ML models directly in the production request path using serving frameworks such as vLLM, SGLang, Triton, or TorchServe.
  • Experience with knowledge graphs and low-latency lookup and retrieval services.
  • Fluency with PyTorch and the Hugging Face ecosystem, including transformers and PEFT.
  • Familiarity with cloud environments, including AWS, and containerization with Docker and Kubernetes.

Responsibilities

  • Own the agent's planning and tool-calling layer, including tool schemas, dynamic tool selection, and structured output generation; ground every response in retrieval, knowledge graph, and real-time data.
  • Design and ship conversational flows, including intent understanding, context tracking, and disambiguation, using LLM- and small-model-based query parsers that feed downstream retrieval and ranking.
  • Profile and reduce serving latency and cost through distillation, quantization, compact output formats, dynamic batching, KV-cache tuning, and semantic caching; use inference runtimes such as vLLM, SGLang, TensorRT-LLM, and ONNX Runtime, and determine whether each request needs a large model, a small model, or no model.
  • Design and own automated evaluation for conversational and generative quality, including multi-turn rubrics, LLM-as-a-judge pipelines, judge calibration against human ratings, and error-analysis tooling; integrate evaluation runs into CI.
  • Define offline metrics that predict user-facing outcomes and partner on A/B tests to confirm them.
  • Take models and services from experiment to the live request path with versioning, monitoring, and rollback, and own their quality, latency, and reliability after launch.
  • Work closely with research scientists, engineers, and product teams to turn research in conversational AI, retrieval, and inference into production systems.

Desired Qualifications

  • A Ph.D. in Computer Science, Machine Learning, NLP, or a related field.
  • Experience fine-tuning and distilling LLMs for task-specific uses such as query parsing, classification, and response generation, using techniques such as LoRA/adapters and teacher/student setups.
  • Experience designing knowledge graph schemas, building graph compilation and ingestion pipelines, and working with graph databases such as TigerGraph or Neo4j.
  • Deep experience with key-value stores and in-memory serving for low-latency lookups, such as Cassandra, Redis, and SSTables.
  • Experience calibrating LLM judges against human judgments for multi-turn conversations, including inter-annotator agreement and judge reliability tracking.
  • Experience with personalization and memory in conversational systems, and applying privacy principles such as data minimization and on-device processing.
  • Experience implementing safety guardrails for LLM-powered agents, including prompt-injection defenses and adversarial evaluation.
  • Familiarity with experiment tracking and data/model versioning tools such as MLflow, Weights & Biases, and DVC.
  • Deep understanding of distributed systems and service design, and experience architecting and leading projects with cross-functional stakeholders.
  • Published work or patents in conversational AI, information retrieval, knowledge graphs, or efficient inference.

About the Company

Apple

Designs phones, computers and their operating systems

Apple designs and sells consumer electronics, from iPhone smartphones and Mac computers to tablets and wearable devices. Its operating systems manage apps and built-in functions for communication and work, while iCloud stores files and synchronizes data across devices. Customers buy additional digital services through subscriptions such as Apple Music and Apple TV, and developers distribute software through the App Store. Apple earns money from hardware sales, subscriptions and digital transactions, selling directly to customers and through retailers and mobile carriers around the world. It was founded in 1976 and has its headquarters in Cupertino, California.

Company size
10,001+
Company stage
IPO
Headquarters
Cupertino, CA
Founded
1976

Recently Posted Jobs

Explore more opportunities at Apple

See more jobs