R

Rivian

Makes electric pickups, SUVs and delivery vans

Staff Software Engineer - Compute & Storage, Autonomy

Platform EngineeringUpdated on Nov 12
Full-timePalo Alto, CA, USA$206.5k - $258.1kIn-person

About the Job

Requirements

  • A bachelor's degree in Computer Science, Electrical Engineering, or a related field, or equivalent experience.
  • At least 8 years of software engineering experience building and operating distributed systems in production.
  • At least 5 years owning a distributed compute, batch execution, or scheduling platform used by other engineering teams, including building the scheduler, orchestration layer, or execution engine rather than only jobs run on one.
  • At least 5 years with Kubernetes as an execution substrate, including scheduling, resource management, custom controllers or operators, autoscaling, and failure modes at scale.
  • At least 5 years with large-scale object storage, including data layout, partitioning, lifecycle and tiering, caching, and performance and cost tradeoffs.
  • At least 3 years with a distributed processing or training framework such as Ray, Spark, Flink, or Dask.
  • At least 3 years of hands-on experience with Go, C++, or Rust, plus strong Python skills.
  • At least 3 years with infrastructure as code and configuration management, such as Terraform, AWS CDK, or CloudFormation, and container build and supply chain.
  • At least 3 years with queueing and event-driven systems such as SQS, Kafka, or Kinesis, including autoscaling workloads on queue depth.
  • At least 3 years with production monitoring and alerting tools such as Prometheus, Grafana, Datadog, or CloudWatch, including defining metrics and building dashboards rather than only consuming them.
  • At least 3 years debugging production distributed systems, including resource contention, stragglers, data skew, network saturation, and cascading failure, and performing root-cause analysis on incidents.
  • At least 2 years carrying an infrastructure cost target and meeting it.
  • Ability to turn ambiguous, high-level requirements into a detailed system design and drive it to completion unprompted.
  • Technical influence beyond personal contributions, demonstrated through designs others build on, standards others adopt, and engineers who improve from working with the candidate.

Responsibilities

  • Own the architecture and roadmap for Autonomy's distributed compute platform, including job scheduling, quota and fair-share across teams, autoscaling, and spot and preemption strategy across multiple clouds.
  • Build scalable tools and APIs that translate high-level job requests into executed work, making large-scale computation accessible to engineers who are not infrastructure specialists.
  • Read jobs run by other teams, profile them, identify inefficiencies and bottlenecks, and either fix them directly or provide tools that help the teams see them.
  • Treat cluster efficiency as a primary metric by eliminating GPU fragmentation, right-sizing quota, and reclaiming capacity stranded on partially filled nodes.
  • Own the storage architecture for autonomy data at petabyte scale, including layout, partitioning, hot, warm, and cold tiering, lifecycle policy, and caching and prefetch layers that keep training and simulation jobs from starving on I/O.
  • Own the multi-cloud compute and data path as training expands beyond one provider, including replication strategy, consistency, and cross-provider egress economics.
  • Drive throughput and turnaround time for the heaviest workloads, including training data loading, large-scale log replay, and batch resimulation running tens of thousands of concurrent jobs.
  • Own the queue-based and event-driven infrastructure behind job submission and autoscaling, and keep it stable as queue depth changes by orders of magnitude.
  • Own platform observability by defining metrics and building dashboards and alerting for job throughput, queue health, cluster utilization, and per-team cost.
  • Define service-level agreements for job admission, completion, and data availability; set the on-call strategy and participate in the rotation.
  • Own compute and storage cost as a first-class engineering metric, instrumented per team and per workload across multiple AWS accounts and services.
  • Work with the security and privacy team on data governance, access control, retention, and audit for vehicle-collected data.
  • Set technical standards for building and running distributed workloads, and raise standards through design review, code review, and mentorship of senior engineers.

Desired Qualifications

  • Experience with GPU cluster management and utilization, including topology-aware placement, gang scheduling, multi-tenant sharing, and fragmentation.
  • Experience in autonomous vehicles, robotics, or another domain involving continuous high-volume sensor data.
  • Experience with multi-cloud or hybrid cloud and on-premise operation, including data movement economics between providers.
  • Knowledge of Linux internals, including the CPU scheduler, memory management, file systems, and networking.

About the Company

Rivian

Makes electric pickups, SUVs and delivery vans

Rivian designs and builds electric pickup trucks, SUVs, and delivery vans, pairing off-road ability with electronics and software it develops itself. In 2021, its R1T became the first fully electric pickup truck on the US consumer market, ahead of larger rivals such as Ford and Tesla. It makes the R1T pickup, the R1S and R2 SUVs, and commercial vans in Normal, Illinois, and sells them directly online and in its own stores rather than through dealers. Its vehicles run on Rivian's own software, which gains features through updates sent over the internet, and most revenue comes from vehicle sales, with service and optional software adding more. Rivian was founded in 2009 and is based in Irvine, California.

Company size
10,001+
Company stage
IPO
Headquarters
Irvine, CA
Founded
2009

Recently Posted Jobs

Explore more opportunities at Rivian

See more jobs