W

Wayve

End-to-end autonomous-driving software for vehicles

Software Engineer - Data Ingestion

Data EngineeringConfirmed live in the last 24 hours
Full-timeSunnyvale, CA, USA

About the Job

Requirements

  • Strong production experience with Apache Spark and Python.
  • Experience building, operating, or debugging large-scale ingestion, extract, transform, and load (ETL), or distributed data-processing pipelines.
  • Ability to diagnose production failures involving corrupt data, stalled queues, retries, and complex dependencies.
  • Understanding of how to optimise high-throughput systems for reliability, performance, and cost.
  • Ability to learn an existing system quickly, work independently, and deliver practical improvements in a fast-moving environment.

Responsibilities

  • Debug and unblock failing ingestion pipelines, stalled jobs, and processing queues.
  • Investigate corrupt, malformed, or unexpected data from a variety of sources.
  • Optimise Spark jobs and batch-processing pipelines for throughput, reliability, and compute efficiency.
  • Design and implement more resilient pipelines so individual bad data segments do not block wider workflows.
  • Improve monitoring, retries, and failure recovery across multi-step ingestion workflows.
  • Help prioritise and unblock important datasets for downstream annotation, data science, and model-training teams, and contribute to the technical direction of the ingestion platform.
  • Work on high-volume pipelines that process real-world driving data for annotation, data science, model training, and evaluation.
  • Build resilient systems that isolate problematic data rather than allowing a single bad segment to block a wider workflow.
  • Orchestrate complex dependencies, queues, retries, and priority data flows.
  • Improve the performance, cost, and operability of the Spark-based data platform.
  • Ingest varied data formats from partners, suppliers, and other third-party sources.

Desired Qualifications

  • Experience with Airflow, Flyte, Databricks, Delta Lake, Scala, or Java.

About the Company

Wayve

End-to-end autonomous-driving software for vehicles

Wayve provides end-to-end autonomous-driving software for automakers and fleets that runs entirely on onboard compute and sensors, without relying on HD maps or city-by-city engineering. The system uses foundation models capable of zero-shot driving and has demonstrated ability to drive across 500+ cities in Europe, North America, and Japan. Unlike traditional AV stacks that depend on mapped data and per-city customization, Wayve aims to generalize driving with its onboard AI across diverse environments. Its goal is to scale commercial robotaxi trials from 2026 and bring supervised autonomy to consumer vehicles from 2027, backed by substantial funding partnerships to support rapid deployment.

Company size
1,001-5,000
Company stage
Series D
Total funding
$2.6B
Headquarters
London, United Kingdom
Founded
2017

Recently Posted Jobs

Explore more opportunities at Wayve

See more jobs