Principal Data Engineer
Confirmed live in the last 24 hours
Locations
London, UK
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Apache Spark
AWS
Data Analysis
C/C++/C#
Python
Requirements
  • Demonstrated experience creating automated, reproducible data pipelines and working with complex, large data sets
  • 8+ years working in Data Engineering
  • 1 or more years experience working with and integrating AWS tools such as Lambda, Athena, API Gateway, Batch, Glue, and RDS
  • Experience with Spark/PySpark and other big data tools
  • Experience creating cloud-based ETL workflows
  • Experience with infrastructure-as-code platforms
  • Proficient with at least one general-purpose programming language, such as Python or Scala
  • Comfortable working with APIs to access data from a variety of sources
  • Experience with the software development life cycle, modern software/hardware/system testing methods, and continuous integration
  • Self-motivated and able to identify and fix issues before they become blockers
  • Excellent written and verbal communication skills
  • Strong analytical, curious mindset
  • Experience architecting server-less systems
  • Experience with Docker
  • Experience with Apache Arrow
  • Experience with robotics, autonomous trucks or other automated distribution yard products
  • Experience in ROS and ROS bag format
  • Experience with C++ and Python
Responsibilities
  • Design, implement, and improve all aspects of the data pipeline, from ingestion through analysis to visualization, to bring the right data to the right internal customers
  • Work cross-functionally with other teams at Outrider to ensure the data pipeline (including on the cloud) serves the various internal and external customers of Outrider data
  • Drive and advise the Engineering teams on how to approach data engineering and analytics, and be an advocate for data-driven decision making at all levels of Outrider by providing data tools that are clean and reliable
  • Work with the team to understand and prioritize development and sustainment projects and drive toward a product that meets all customer, business, and technical expectations
  • Collaborate and mentor other data engineering team members and participate in peer design and code reviews
Outrider

51-200 employees

Autonomous yard operations