L

Lely

Dairy automation and robotic milking systems

Data Engineer

Data EngineeringUpdated on Nov 11
Full-timeMaassluis, Netherlands€5k - €7.2k /mo

About the Job

Requirements

  • At least 5 years of relevant work experience as a Data Engineer or in a comparable role in data and cloud environments.
  • Strong Python engineering skills, including object-oriented programming, modular architectures, and testable code.
  • Strong SQL skills for data extraction, transformation, and validation in data-ingestion processes.
  • Experience accessing data from diverse sources, including APIs, SQL databases, and file-based sources such as FTP/SFTP.
  • Hands-on experience with Databricks, including Workflows, Spark, Delta Lake, and Unity Catalog.
  • Knowledge of Delta Lake and data warehouse patterns such as CDC, SCD Type 2, incremental loading, and deduplication.
  • Experience with unit and integration testing.
  • Experience with CI/CD processes and tools such as GitLab.
  • Knowledge of cloud technologies, preferably in the Azure ecosystem.
  • Experience with monitoring, incident debugging, and maintaining reliable production environments.
  • A pragmatic approach to AI-assisted software development and engineering.
  • Must reside in the Netherlands.
  • Must successfully complete a pre-employment screening that includes a Certificate of Conduct (VOG) and a background and integrity check before starting employment.

Responsibilities

  • Build, maintain, and optimize data-ingestion processes for diverse data sources using object-oriented Python in Databricks.
  • Design and implement reliable data warehouse and Delta Lake patterns, including CDC, SCD Type 2, incremental loading, and deduplication.
  • Manage and configure Databricks components, including Workflows, Unity Catalog, and Compute Clusters, and maintain infrastructure as code using Databricks Asset Bundles and OpenTofu/Terraform.
  • Investigate new source and integration types, conduct proof-of-concepts, and contribute to continued development of the data platform.
  • Own the reliability of production environments through monitoring, incident analysis, root-cause analysis, and cost monitoring.
  • Ensure quality through design reviews, code reviews, unit and integration tests, addressing technical debt, and continuously improving engineering processes in an agile environment.
  • Contribute to agile practices through backlog refinement, estimation, and risk management.
  • Work with colleagues in the Data Warehouse Management team and with the BI and Data Governance teams to deliver a reliable, scalable data platform.

Desired Qualifications

  • Knowledge of cloud technologies, preferably in the Azure ecosystem.

About the Company

Lely

Dairy automation and robotic milking systems

Lely develops dairy automation and robotics to ease farming labor. Its products include robotic milking systems (Astronaut), automated feeding systems, and barn cleaning technology. The Astronaut robotic milking system allows cows to choose when to be milked, using sensors and automation to milk cows without human intervention, while automated feeding optimizes rations and feeding schedules. Lely distinguishes itself by being privately held and family-owned, with a strong long-term R&D focus, evident in over 1,600 active patents and a history of investing in technology rather than short-term profits. After focusing on dairy automation (and divesting its forage division in 2017), the company operates in more than 50 countries and aims to create a sustainable and profitable future for farmers through dependable, technology-driven solutions.

Company size
1,001-5,000
Company stage
Not listed
Total funding
Not listed
Headquarters
Tavistock, United Kingdom
Founded
1948

Recently Posted Jobs

Explore more opportunities at Lely

See more jobs