T

Tesla

Makes electric cars, robotaxis and grid batteries

SI Engineer - Model Pre-Training

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-timePalo Alto, CA, USANo salary listedIn-person

About the Job

Requirements

  • Deep, proven expertise in deep learning fundamentals, particularly in pre-training large-scale language, vision, or multimodal models.
  • Proven experience with scaling law analyses or compute-optimal training studies for large AI models, with a strong understanding of how model, data, and compute trade off.
  • Strong grasp of large-scale data pipelines, data mixture design, and the tradeoffs between model capacity, data quality and quantity, and compute.
  • In-depth knowledge of optimization at scale, training stability, loss and objective design, and modern neural architectures, including mixture of experts and hybrid attention.
  • Hands-on familiarity with distributed training techniques for frontier-scale models, including data, tensor, and pipeline parallelism, sharded optimizers, and mixed precision.
  • Strong expertise in distributed computing and large-scale training infrastructure.
  • Proficiency in Python and a deep understanding of software engineering best practices.
  • Experience with deep learning frameworks such as PyTorch, TensorFlow, or JAX.
  • Ability to work collaboratively in a cross-functional team environment.
  • Problem-solving skills and the ability to troubleshoot complex system-level issues across data, training, and evaluation.

Responsibilities

  • Design and iterate on pretraining recipes powering Full Self-Driving, Optimus, and Digital Optimus policy foundation models.
  • Perform scaling law analyses on model size, data size, data mixture, architecture, training compute, and other critical parameters to predict and optimize pre-training outcomes.
  • Develop and iterate on pre-training methodologies, including data curation and mixture design, tokenization, objective design, and training recipes informed by measured scaling behavior.
  • Run controlled experiments that isolate scaling variables, fit predictive laws, and translate findings into compute-optimal training plans and architecture choices, such as mixture of experts and hybrid attention.
  • Drive pretraining data strategy through curation, filtering, deduplication, domain mix, synthetic data, and quality signals that improve evaluations and capabilities.
  • Build and maintain infrastructure for efficient large-scale distributed pre-training and high-throughput experimental sweeps, resolving compute and memory bottlenecks end-to-end.
  • Evaluate pre-trained models with rigorous intermediate and downstream metrics; identify and address scaling limitations related to data quality, stability, and saturation regimes before models move into post-training.
  • Work with cross-functional teams across data, distillation, and deployment to ship stronger, compute-efficient base models into production that meet performance, safety, and reliability standards.
  • Contribute tools and frameworks that make pre-training and scaling analyses reproducible, measurable, and reusable across Tesla SI model families.

About the Company

Tesla

Makes electric cars, robotaxis and grid batteries

Tesla makes electric cars and batteries that store power for homes and utilities, and it develops self-driving software and humanoid robots. In 2023, its Model Y became the first electric vehicle to rank as the world's best-selling car. Buyers order the Model 3 sedan, Model Y SUV, or Cybertruck pickup directly from Tesla instead of through dealers, and they charge on its Supercharger fast-charging network. It also sells Megapack and Powerwall batteries, runs a robotaxi service, and charges drivers a monthly fee for self-driving software that still needs their supervision. Tesla was founded in 2003, with Elon Musk as its lead early investor and a recognized co-founder, and it is based in Austin, Texas.

Company size
10,001+
Company stage
IPO
Headquarters
Austin, TX
Founded
2003

Recently Posted Jobs

Explore more opportunities at Tesla

See more jobs