Tesla

Makes electric cars, robotaxis and grid batteries

SI Engineer - Action Conditioned World Models

AI ResearchConfirmed live in the last 24 hours

Full-timePalo Alto, CA, USANo salary listedIn-person

About the Job

Requirements

Expertise in generative video and/or world-model architectures, including diffusion, flow-matching, autoregressive, or hybrid spatiotemporal models.
Strong background in spatiotemporal modeling, 3D/4D scene understanding, or neural simulation of physical environments.
Proven experience with large-scale distributed training and the major forms of parallelism used for frontier multimodal models.
Familiarity with reinforcement learning, imitation learning, or closed-loop evaluation in simulation or embodied settings.
Proficiency with PyTorch or JAX and a production engineering mindset for reliable large-scale experiments.

Responsibilities

Design and train action-conditioned video and world models that predict future frames, sensor states, and dynamics given ego/bot actions and multimodal context.
Develop causal, physics-aware architectures that capture motion, contact, multi-agent interaction, and environmental dynamics over long horizons.
Integrate 3D/4D generative techniques, such as Gaussian Splatting, neural fields, volumetric or feed-forward scene representations, to improve geometric fidelity and controllable novel-view/rollout realism.
Build closed-loop training and evaluation systems where world models support policy rollouts, counterfactual “what-if” generation, and iterative refinement via simulation feedback and real-world residuals.
Scale distributed multimodal training and low-latency inference using data/model/pipeline parallelism, efficient attention and tokenization, and distillation where needed for real-time use.
Partner across Autonomy and Robotics to define metrics for FSD and Optimus, including physical plausibility, controllability, and long-horizon consistency, and ship models into evaluation and reinforcement learning loops.

Category	Tesla SI
Location	Palo Alto, California
Requisition ID	288838
Type	Full-time

Job Description

<div class="MSWordContent"><p>Video generation and world models have advanced rapidly, yet still struggle with physics, causality, long-horizon consistency, and fine controllability under real action inputs. At Tesla SI, we train action-conditioned world models on millions of hours of real-world video and multimodal telemetry from the car and bot fleets, on one of the largest compute clusters in the industry. The goal is not open-ended generative content but it is to faithfully reproduce real-world successes and failures on Full Self-Driving and Optimus so we can evaluate policies offline, generate synthetic experience, and run closed-loop reinforcement learning at a scale the physical fleet alone cannot support.</div>

Job Responsibilities

Design and train action-conditioned video and world models that predict future frames, sensor states, and dynamics given ego/bot actions and multimodal context
Develop causal, physics-aware architectures that capture motion, contact, multi-agent interaction, and environmental dynamics over long horizons
Integrate 3D/4D generative techniques (e.g., Gaussian Splatting, neural fields, volumetric or feed-forward scene representations) to improve geometric fidelity and controllable novel-view / rollout realism
Build closed-loop training and evaluation systems where world models support policy rollouts, counterfactual “what-if” generation, and iterative refinement via simulation feedback and real-world residuals
Scale distributed multimodal training and low-latency inference (data/model/pipeline parallelism, efficient attention and tokenization, distillation where needed for real-time use)
Partner across Autonomy and Robotics to define metrics that matter for FSD and Optimus—physical plausibility, controllability, long-horizon consistency—and ship models into evaluation and RL loops

Job Requirements

Expertise in generative video and/or world-model architectures (diffusion, flow-matching, autoregressive, or hybrid spatiotemporal models)
Strong background in spatiotemporal modeling, 3D/4D scene understanding, or neural simulation of physical environments
Proven experience with large-scale distributed training and the major forms of parallelism used for frontier multimodal models
Familiarity with reinforcement learning, imitation learning, or closed-loop evaluation in simulation or embodied settings
Proficiency with PyTorch or JAX and a production engineering mindset for reliable large-scale experiments
Curiosity about building models that understand, predict, and generate the physical world for robots and cars, not just photorealistic video

Tesla

SI Engineer - Action Conditioned World Models

About the Job

Requirements

Responsibilities

Job Description

Job Responsibilities

Job Requirements

About the Company

Tesla

Recently Posted Jobs