Data Scientist - Evaluation

Summary

Do you get excited by building software systems to enhance the automatic evaluation of various Apple AI products? Our Evaluation organization is responsible for providing principled assessments across a diverse range of Apple features, from Search, Siri to the latest Apple Intelligence capabilities. As part of our online metrics and insights team, we are looking for a Sr Data Scientist to contribute to product understanding and development of Apple features. This role involves using analytical and engineering methods augmented by LLMs and python to develop conceptual and analytical models and perform insights analyses to uncover product quality issues that can inform improving the overall user experience.

Minimum Qualifications

3+ years of data science experience applying quantitative methods to structured and unstructured data for exploratory data analysis, pattern recognition, metrics development, and analytical insights. Strong programming skills in Python and SQL, with experience in data manipulation, processing, and analytics pipelines. Experience with LLM-driven analytics workflows, using Python coding frameworks and evaluation pipelines. Proven ability to analyze user behavior, design and interpret A/B experiments, and translate findings into product improvements. Master degree in a technical or quantitative field such as Statistics, Mathematics, Computer Science, Engineering, Linguistics, or Physics (PhD preferred).

Preferred Qualifications

Experience with Natural Language Processing (NLP) and LLMs. Experience with development of dataset and analysis conceptual methodologies. Self-motivated and curious with demonstrated creative and critical thinking capabilities and a highly developed drive to determine and improve how things work. Excellent communication skills—able to synthesize analyses into clear insights and influence multi-functional partners. High tolerance for ambiguity. You find a way through. You anticipate. You connect and synthesize. Demonstrated experience in encouraging partner relationships and influencing decisions with data.

Description

As a Sr Data Scientist, you will use statistical and LLM analysis of structured and linguistic data to deepen our understanding of Apple’s AI product and support the development of future versions. Investigate world-wide product usage to identify weaknesses and chances to create the product better. Define metrics and instrumentation strategies to assess the success of new features. Build relationships with Engineering leads and product partners to drive a privacy-first strategy for analytical insight. Employ a diverse toolkit of detailed analytical approaches, methodologies, frameworks, and technical strategies with a particular focus on language data analysis. Build credibility with partners through clarity and structured, critical thinking.