R

Rivian

Makes electric pickups, SUVs and delivery vans

Senior Data Platform Engineer

Platform EngineeringPosted on Nov 16
Full-timeAtlanta, GA, USA

About the Job

Requirements

  • At least 8 years of experience in software engineering, data infrastructure, platform engineering, or site reliability engineering.
  • At least 3 years of experience managing production Databricks environments through Terraform and APIs or SDKs rather than the user interface, with deep knowledge of Unity Catalog, System Tables, compute policies, and Asset Bundles.
  • Knowledge of Apache Spark internals and tuning, and the tradeoffs among batch, microbatch, and streaming at multi-petabyte scale.
  • Strong AWS experience, including IAM, S3, KMS, VPC peering, subnets, security groups, and cost management.
  • Experience with Databricks Asset Bundles, CI/CD, GitOps, containers such as Docker and Kubernetes, and zero-downtime infrastructure updates across multi-tier environments (development, staging, and production), tested and reviewed as code.
  • Experience with observability and governance, including metrics, logs, alerting, SLOs, RBAC and ABAC, lineage, auditing, encryption, and secret management.
  • Knowledge of open table formats and distributed data systems such as Delta Lake, Apache Iceberg, Kafka, and columnar stores.
  • Experience building internal developer tools, APIs, or command-line interfaces, and applying agile methods to create transparency and prioritization.
  • A bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, Artificial Intelligence, or a related field, or equivalent experience.

Responsibilities

  • Build partnerships with data engineers, software engineers, security, cloud infrastructure, and business partners who depend on the platform, and deliver projects end to end within the roadmap and architecture set by Staff engineers.
  • Manage Databricks environments as code at petabyte scale across multiple AWS accounts, using Terraform modules for workspaces, Unity Catalog metastores, cluster policies, and AWS networking and identity, including VPC, Transit Gateway, PrivateLink, and IAM roles and KMS.
  • Build platform and data observability through monitoring, alerting, lineage, data quality, and dashboards for reliability, performance, and cost.
  • Build self-service capabilities for engineers and analysts to request access, create environments, and deploy pipelines safely, with automation for workspace lifecycle, onboarding, and access audits.
  • Automate cloud and Databricks configuration with infrastructure as code, Databricks Asset Bundles, and CI/CD such as GitHub Actions or GitLab CI, using GitOps workflows and multi-tier deployments across development, staging, and production, with zero-downtime updates.
  • Build Go and Python libraries, command-line tools, and templates that make secure, tested, cost-aware patterns the easy path, treating the platform as a software product.
  • Own the reliability and security of components, meet SLOs, respond to incidents, identify root causes, and improve systems to prevent recurrence.
  • Own platform cost by monitoring Databricks System Tables for DBU consumption, enforcing cluster policies, right-sizing compute, tuning Spark and Delta workloads, working with data teams to eliminate inefficient workloads, and providing cost data for chargeback models.
  • Implement governance and security with Unity Catalog, including row- and column-level security, ABAC, automated lineage, identity-provider synchronization, and secret rotation.
  • Use open formats such as Delta Lake, with UniForm and Apache Iceberg interoperability, to preserve long-term flexibility and recommend where open architecture is more sustainable.
  • Use AI models and agents to increase personal productivity and help the business use data.

Desired Qualifications

  • Terraform experience is preferred.
  • A higher degree is a plus.
  • Experience with data transformation.

About the Company

Rivian

Makes electric pickups, SUVs and delivery vans

Rivian designs and builds electric pickup trucks, SUVs, and delivery vans, pairing off-road ability with electronics and software it develops itself. In 2021, its R1T became the first fully electric pickup truck on the US consumer market, ahead of larger rivals such as Ford and Tesla. It makes the R1T pickup, the R1S and R2 SUVs, and commercial vans in Normal, Illinois, and sells them directly online and in its own stores rather than through dealers. Its vehicles run on Rivian's own software, which gains features through updates sent over the internet, and most revenue comes from vehicle sales, with service and optional software adding more. Rivian was founded in 2009 and is based in Irvine, California.

Company size
10,001+
Company stage
IPO
Headquarters
Irvine, CA
Founded
2009

Recently Posted Jobs

Explore more opportunities at Rivian

See more jobs