O

Our Future Health

UK health data collection for research

Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeLondon, UK£62k /yr

About the Job

Requirements

  • Proven experience building production-grade data pipelines and ETL processes.
  • Excellent Python skills with experience developing scalable data solutions.
  • Experience working with PySpark or distributed data processing frameworks.
  • Experience using GitHub and modern software engineering practices.
  • Understanding of cloud-based data platforms, ideally Azure.
  • Experience working with sensitive or regulated data and knowledge of secure data handling principles.
  • Ability to review technical designs and architecture documents and translate them into practical implementations.
  • Experience conducting code reviews and maintaining engineering standards.
  • Experience working with NHS or healthcare datasets.
  • Knowledge of healthcare coding systems and standards such as SNOMED CT, ICD, or READ codes.
  • Experience with Databricks.
  • Experience with Kubernetes.
  • Experience using Dagster or other workflow orchestration platforms.
  • SQL and data modelling experience.
  • Understanding of metadata management and data governance frameworks.

Responsibilities

  • Build and maintain data pipelines across the Azure-based platform.
  • Translate technical specifications, architecture designs, and Jira tickets into secure, scalable implementations.
  • Develop data transformation and ETL processes using Python and PySpark.
  • Work with healthcare and NHS datasets to curate trusted, research-ready data assets.
  • Review and approve code through pull requests, ensuring consistency, quality, security, and adherence to engineering standards.
  • Collaborate with researchers, engineers, and stakeholders to understand requirements and deliver solutions.
  • Identify risks, challenge assumptions, and ask questions to improve designs before implementation.
  • Produce documentation, source-to-target mappings, and support technical standards across the platform.
  • Contribute to best practices in data engineering, security, governance, and cloud architecture.

Desired Qualifications

  • Experience with NHS or healthcare datasets.
  • Knowledge of healthcare coding systems and standards such as SNOMED CT, ICD, or READ codes.
  • Experience with Databricks.
  • Experience with Kubernetes.
  • Experience using Dagster or other workflow orchestration platforms.

About the Company

Our Future Health

UK health data collection for research

Our Future Health is a UK charity running the country's largest health research programme. It invites up to five million adults to participate by completing a questionnaire and giving a blood sample, with their data linked to health records. This creates a detailed picture of the nation’s health to help researchers find new ways to prevent, detect, and treat diseases such as cancer, diabetes, and dementia. The project combines ethical, practical, and technological work to handle vast amounts of data and develop useful insights. Unlike smaller studies, it aims for large-scale, long-term data collection across the entire population. The ultimate goal is to help future generations stay healthier for longer by guiding better disease prevention and care.

Company size
51-200
Company stage
Not listed
Total funding
Not listed
Headquarters
London, United Kingdom
Founded
2020

Recently Posted Jobs

Explore more opportunities at Our Future Health

See more jobs