V

Valtech

Global digital transformation and design consultancy

Senior Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeRemote in PortugalNo salary listed

About the Job

Requirements

  • Strong hands-on experience with Apache Spark and Delta Lake.
  • Strong Python and SQL skills.
  • Proven experience building batch and streaming pipelines and production-grade data platforms.
  • Solid understanding of data modeling, data quality, and governance.
  • Strong, demonstrable hands-on experience with the Databricks Lakehouse Platform, ideally on AWS, including relevant services such as S3, Glue, EMR, Redshift, Kinesis, and Lambda.
  • Deep expertise in at least one major cloud platform—AWS, GCP, or Azure/Fabric—and demonstrable openness and ability to work across clouds.
  • Experience with lakehouse architectures and distributed systems.
  • Strong understanding of scalability, reliability, and performance.
  • Ability to challenge business requirements rather than accept them at face value.
  • Adaptability and willingness to work across platforms.

Responsibilities

  • Design and implement scalable data platforms and pipelines, primarily on AWS and Databricks, and on GCP or Azure/Fabric where projects require.
  • Develop reliable batch, streaming, and near-real-time pipelines using Spark and Delta Lake.
  • Build ingestion, transformation, and curation workflows for structured and unstructured data.
  • Implement lakehouse patterns and medallion layering (bronze, silver, gold), ensuring reusable, scalable systems aligned with enterprise needs.
  • Contribute to cloud-to-cloud migrations, including re-platforming pipelines and data models.
  • Deliver model-ready, production-grade datasets for analytics, machine learning, causal modeling, and optimization.
  • Build data pipelines for generative AI use cases, including LLMs, RAG, vector-based data flows, and agent-based architectures.
  • Leverage Databricks MLflow and Unity Catalog, or equivalent tooling on other platforms.
  • Design logical and physical data models for analytical and operational use cases.
  • Orchestrate workflows with Databricks Workflows, Airflow, dbt, or equivalents such as Azure Data Factory and Cloud Composer, focusing on automation, reliability, and maintainability.
  • Apply event-driven and streaming architecture patterns.
  • Ensure data governance, lineage, quality, and access control (RBAC/ABAC) using Unity Catalog, AWS Lake Formation, or equivalents such as Microsoft Purview and GCP Dataplex.
  • Establish observability for data freshness, pipeline reliability, and SLA adherence.
  • Enable serving layers, including APIs, feature inputs, and analytical endpoints, for downstream ML and AI platforms.
  • Continuously optimize pipelines and infrastructure for performance, scalability, and cost.
  • Ask questions to understand stakeholders’ underlying needs, not just their stated requests.
  • Identify gaps and better alternatives instead of simply executing a brief as written.
  • Work closely with data scientists, ML engineers, analysts, and business stakeholders.
  • Support adoption of data products and contribute to best practices.

Desired Qualifications

  • GCP experience, including BigQuery, Dataflow, Pub/Sub, Cloud Storage, or Dataproc, especially in migration contexts.
  • Hands-on experience across more than one cloud or cloud migration experience, such as Azure to GCP.
  • Snowflake experience.
  • Generative AI and AI data systems experience, including RAG pipelines, vector databases, and LLM data preparation.
  • CI/CD for data pipelines and infrastructure-as-code tools such as Terraform or CloudFormation.
  • Streaming technologies such as Kafka, Spark optimization, or ML/experimentation platforms.
  • Experience building data products or large-scale analytics platforms.
  • Agile or consulting experience.

About the Company

Valtech

Global digital transformation and design consultancy

Valtech is a global business transformation agency that offers digital strategy, design, and engineering services. It works by pairing strategic advisory with hands-on technology implementation to build and optimize digital experiences, platforms, and data-driven capabilities across areas like experience design, commerce, and data and AI. Valtech differentiates itself with a network of 80+ offices in over 25 countries, a client roster of blue-chip brands, and growth through both organic expansion and acquisitions backed by BC Partners. Its goal is to help clients transform their business models and customer experiences at scale in the digital era.

Company size
5,001-10,000
Company stage
IPO
Headquarters
London, United Kingdom
Founded
1993

Recently Posted Jobs

Explore more opportunities at Valtech

See more jobs