Viaduct

AI-powered predictive maintenance from vehicle data

Data Engineer

Data EngineeringConfirmed live in the last 24 hours

Full-timeSloveniaNo salary listedIn-person

About the Job

Requirements

At least 3 years of experience as a data engineer.
Experience as a tech lead or mentor.
Proficiency in Python, Scala, Java, C++, and SQL.
At least 3 years of experience with Spark or equivalent technologies.
At least 2 years of experience with a workflow scheduler such as Airflow, Prefect, or Argo.
At least 2 years of experience with distributed file systems such as HDFS or S3.
Familiarity with tools in the open-source data ecosystem, including Apache and CNCF.
Experience with incremental or real-time processing technologies such as Delta Lake, Apache Hudi, Kafka Stream, or Spark Streaming.
Follow security and privacy policy and procedure documents and the guidelines in the Employee Handbook.
Participate in new-hire and annual security and privacy training.
Treat data security and privacy as a primary job responsibility.
Report security incidents discovered as bugs.
Obtain Security Team approval before adding third-party software to the codebase.
Explicitly consider security implications when conducting pull-request reviews.

Responsibilities

Create and support batch, incremental, and real-time ETL pipelines.
Standardize ingestion, validation, and cleaning processes across clients.
Automate data validation to increase data quality.
Manage and evolve schemas across all parts of the pipeline.
Monitor, tune, and optimize Spark jobs.
Become a domain expert in connected vehicle data.

Desired Qualifications

Experience with Kubernetes.
Experience working with machine learning teams.
Contribution to open-source projects.
Experience in the automotive industry or a strong interest in cars.
Prior work in small, agile teams.

Who We Are

We are a diverse and focused team motivated to solve the hardest problems in the automotive industry through machine learning. We have developed an end-to-end machine learning platform to empower automakers to build safer, smarter, and personalized vehicles. Our platform increases connected vehicle data accessibility and actionability for automakers, their partners and their end-customers.

Who You Are

You are a thoughtful engineer. You understand the complexities of distributed systems and how to triage and solve issues that arise with them. Scalability is top of mind when designing any system or writing code. You believe building a better ETL system requires close collaboration with the machine learning and data science teams.

About the Role

As an early data engineer at Viaduct, an analytics and ML platform company, your work is critical to our success. You are responsible for ensuring data quality and reliability in every part of our ETL pipeline, from ingestion to client integrations.

Responsibilities

Creating and supporting batch, incremental, and real-time ETL pipelines
Standardizing ingestion, validation, and cleaning processes across clients
Automating data validation to increase data quality
Managing and evolving schemas in all parts of our pipeline
Monitoring, tuning, and optimizing Spark jobs
Becoming a domain-expert in connected vehicle data

About You

3+ years as a data engineer
Experience as a tech lead or mentor
Proficiency in Python/Scala/Java/C++ and SQL
3+ years of experience with Spark or equivalent technologies
2+ years of experience with a workflow scheduler (Airflow, Prefect, Argo, etc)
2+ years of experience with distributed file-systems (HDFS, S3, etc)
Familiar with the tools in open-source data ecosystem (Apache, CNCF, etc)
Experience with incremental or real-time processing (Delta Lake, Apache Hudi, Kafka Stream, Spark Streaming, etc)

Security and Privacy Responsibilities

Follow our policy and procedure documents related to security and privacy
Follow the security and privacy guidelines in the Employee Handbook
Participate in new hire and annual training for security and privacy
Treat data security and privacy as one of your primary job responsibilities
Report Security Incidents you discover as bugs
Get approval from the Security Team before adding new 3rd party software to our codebase
Explicitly consider security implications when doing PR reviews

Bonus

Experience with Kubernetes
Experience working with ML teams
Contributor to open source projects
Experience in the Automotive industry or a love of cars
Prior work in small, agile teams

About the Company

Viaduct

AI-powered predictive maintenance from vehicle data

Viaduct analyzes connected vehicle data through a cloud platform powered by its patented TSI Engine. It uses AI to automatically process large data volumes to identify patterns and predict issues with vehicle components. Its applications include predictive maintenance, driver identification, and driver safety scoring for automotive manufacturers and other automotive ecosystem players. Practically, the TSI Engine ingests telematics and vehicle data, runs machine learning models in the cloud, and outputs insights and risk assessments that help schedule maintenance and improve safety. Viaduct differentiates itself with a patented engine and a cloud-based data analytics platform focused on connected vehicles, combining predictive maintenance with driver-related analytics. The company’s goal is to improve vehicle reliability and safety by turning data into actionable insights through machine learning and data analytics.

Company size: 51-200
Company stage: Acquired
Total funding: $21M
Headquarters: Menlo Park, CA
Founded: 2019

Recently Posted Jobs

Explore more opportunities at Viaduct

See more jobs