A

Apple

Designs phones, computers and their operating systems

Machine Learning Engineer - Evaluation, Responsible AI and Safety

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-timeCambridge, MA, USANo salary listedIn-person

About the Job

Requirements

  • A Master of Science or PhD in Computer Science, Linguistics, Cognitive Science, Human Computer Interaction, Psychology, Mathematics, Physics, or a similar science or technology field with a strong basis in scientific data collection and analysis, plus at least 4 years of relevant work experience; alternatively, a Bachelor of Arts or Bachelor of Science with 8 or more years of relevant work experience.
  • Experience with LLM judges and human-in-the-loop techniques, including designing human annotation projects, writing guidelines, and handling highly multi-labeled, nuanced, and often conflicting data.
  • Proficiency in data science, machine learning, analytics, Python, Pandas, AI-assisted coding tools, and at least one plotting or visualization library.
  • Ability to collaborate cross-functionally, prioritize competing project needs, set and maintain schedules for project milestones, and communicate with team members at all levels and internal and external stakeholders.
  • A rigorous approach to metric development, interpreting experiments and evaluations, and presenting results to executives.
  • Comfort working with sensitive or potentially offensive content.

Responsibilities

  • Operationalize policy and risk assessments into measurable criteria, design datasets, develop LLM- and human-in-the-loop methods for data analysis, and implement robust data pipelines to produce reliable, scalable insights for product ship/no-ship decisions.
  • Analyze evaluation results before and after launch, conduct failure investigations, and use human and automated feedback to continuously improve feature safety and build user trust.
  • Provide technical execution for Responsible AI evaluation efforts for global generative AI products.
  • Champion technical excellence, provide data-driven safety recommendations to leadership, and systematically solve complex engineering roadblocks.
  • Collaborate with cross-functional partners to minimize unintended consequences across people, systems, and society while improving feature capabilities and user experience.

Desired Qualifications

  • Experience working in the Responsible AI space.
  • Curiosity about fairness and bias in generative AI systems and interest in making the technology more equitable.
  • A social science or humanities background, or demonstrable interest in socio-technical issues.
  • Prior scientific research and publication experience.
  • Experience contributing to diverse teams in a collaborative environment.
  • Experience working with multimodal models, agentic systems, and/or diverse languages and locales.
  • Experience with compliance and regulatory needs in AI, especially child safety, clinical or health applications, or companionship chatbots.
  • Experience conducting large-scale benchmarking across models and capabilities.

About the Company

Apple

Designs phones, computers and their operating systems

Apple designs and sells consumer electronics, from iPhone smartphones and Mac computers to tablets and wearable devices. Its operating systems manage apps and built-in functions for communication and work, while iCloud stores files and synchronizes data across devices. Customers buy additional digital services through subscriptions such as Apple Music and Apple TV, and developers distribute software through the App Store. Apple earns money from hardware sales, subscriptions and digital transactions, selling directly to customers and through retailers and mobile carriers around the world. It was founded in 1976 and has its headquarters in Cupertino, California.

Company size
10,001+
Company stage
IPO
Headquarters
Cupertino, CA
Founded
1976

Recently Posted Jobs

Explore more opportunities at Apple

See more jobs