Staff ML/Research Engineer - Reinforcement Learning

Summary

Apple Cloud AI is at the forefront of ML and generative AI, where data and models come together to enable the next generation of intelligent experiences on Apple products and services. We build scalable reinforcement learning (RL) and post-training services and frameworks that accelerate the development of real-world AI applications, designed with both usability and efficiency in mind. Our platform covers the full model development lifecycle, from the first experiment to large-scale reinforcement learning, preference optimization, and agentic post-training on thousands of GPUs and TPUs. Our team applies the latest RL research to production systems, with user experience and accelerator efficiency in mind. You'll play a key role in shaping AI research at Apple and how its models are aligned, optimized, and scaled. We're looking for engineers and researchers who are passionate about research and building a high-performance RL stack that supports diverse real-world use cases. You'll have the autonomy to experiment, the scale to make an impact, and the support to take ideas from prototype to production. You'll work alongside a world-class team to build RL systems that make ML development faster, more reliable, and more efficient.

Minimum Qualifications

8+ years of experience in large-scale machine learning, including reinforcement learning, from research to production. 5+ years of experience building and operating large-scale RL or post-training systems in production. Experience with RL and alignment techniques, such as RLHF, RLVR, PPO, GRPO, and DPO, as well as supervised fine-tuning as a starting point for RL. Hands-on experience with at least one higher-level RL or post-training framework such as VeRL, OpenRLHF, TRL, NeMo-RL, AXLearn, or Megatron-LM. Deep expertise in PyTorch or JAX, including distributed computing features. Strong programming skills in Python or C++. Strong collaboration and written and verbal communication skills, with a track record of leading technical work across teams. BS, MS, or PhD degree in Computer Science, Computer Engineering, or equivalent practical experience.

Preferred Qualifications

Experience in the below is preferred: Experience designing reward functions, reward models, and verifiers for reasoning, coding, and tool-use tasks. Experience with agentic RL, including multi-turn environments, tool use, and sandboxed code execution. Experience with high-throughput inference engines for rollouts, such as vLLM or SGLang, and with weight synchronization between learner and sampler. Experience scaling RL with data, tensor, pipeline, or expert parallelism across hundreds to thousands of GPUs or TPUs. Strong understanding of GPU and TPU architecture, including memory hierarchy, matrix units, and high-bandwidth memory, and how they shape RL performance. Experience with collective communication libraries such as NCCL and with high-speed interconnects such as NVLink, InfiniBand, and TPU ICI. Experience profiling and optimizing RL workloads with tools such as Nsight Systems, PyTorch Profiler, or XProf. Open source contributions to RL or post-training frameworks, or publications at top ML venues such as NeurIPS, ICML, or ICLR.

Description

The Apple Cloud AI Platform team works with Apple's researchers and ML engineers to improve and ship models at Apple's bar for quality and privacy. In this role, you will bring deep hands-on experience in large-scale reinforcement learning to Apple's most important post-training efforts. You will develop RL recipes for reasoning, tool use, and agentic behavior. You will apply techniques such as RLHF, RLVR (reinforcement learning with verifiable rewards), preference optimization (DPO), and policy-gradient methods (PPO, GRPO). You will diagnose reward hacking, policy collapse, and unstable optimization, and choose the algorithm, reward design, and parallelism settings that let policies improve reliably at scale on GPUs and TPUs. As a Staff ML/Research Engineer, you will set the technical direction for how Apple applies RL to its models and lead efforts that span multiple teams.

Responsibilities

Work with researchers and ML engineers to develop cutting-edge RL methods, and turn their needs into platform features and performance improvements that every team can use. Develop RL algorithms and recipes, such as PPO, GRPO, and DPO variants, that improve reasoning, instruction following, and agentic capabilities. Design reward modeling and evaluation approaches, including learned reward models, verifiable rewards, and LLM-as-judge pipelines, and defend them against reward hacking. Dig into RL behavior at scale, such as reward collapse, KL divergence drift, entropy loss, off-policy mismatch, and drops in throughput, and track each problem down to its root cause in the data, the reward, the algorithm, the code, or the hardware. Build and own scalable, efficient RL frameworks and services on GPUs and TPUs, built on PyTorch or JAX and on higher-level frameworks. Build reinforcement learning infrastructure for post-training, including high-throughput rollout and inference services, asynchronous actor-learner pipelines, reward pipelines, and secure sandboxes where models can run code and use tools. Apply RL to long-horizon, multi-turn, multimodal, and mixture-of-experts settings, and solve the stability and efficiency problems each one brings. Integrate the RL platform with Apple's data and governance stack so that every run meets privacy and security standards. Set technical direction across teams, mentor engineers, and raise the engineering bar.