Inactive
C

Capgemini

Global consulting, technology services, outsourcing

Data Engineer

Data EngineeringPosted on Sep 30
Full-timeNew York, NY, USAIn-person

About the Job

Requirements

  • Five to seven years of data engineering experience, including at least two years of production Databricks work with Delta Lake, Unity Catalog, Delta Live Tables or Lakeflow, structured streaming, and SQL warehouses.
  • Strong Python skills for transformation logic, data-quality automation, and pipeline orchestration.
  • Expert-level SQL and the ability to design schemas for analytical and serving workloads.
  • Fluency with GCP services including BigQuery, Cloud Storage, Cloud Run, and IAM.
  • Hands-on experience with at least one clean room or privacy-preserving data exchange technology.
  • Knowledge of PII governance in a regulated-industry context, including data residency, consent frameworks, GLBA, Fair Lending, and UDAAP.

Responsibilities

  • Design, build, and operate the full medallion stack on Databricks, including raw landing in Bronze, identity-resolved and feature-engineered Silver, and governed Gold serving queried by the Helm agent layer on every conversational turn.
  • Implement Delta Live Tables and Lakeflow pipelines for batch and structured-streaming ingestion of transaction signals, behavioral events, and third-party identity data.
  • Own schema enforcement and evolution at the Bronze boundary so upstream changes surface as controlled events rather than downstream failures.
  • Build the unstructured processing tier from Bronze to Silver using Databricks AI Functions or ai_query to convert PDFs, brand guidelines, compliance rule sets, and creative assets into governed Silver tables.
  • Own Unity Catalog as the governance layer, including access control, lineage tracking, and per-client tenant isolation across Helm's financial institution client base.
  • Implement pipeline-level PII governance controls, including redacted ID egress, consent signal propagation, and alignment with GLBA, Fair Lending, and UDAAP guardrails.
  • Build and maintain data-quality expectation frameworks for reconciliation, deduplication, and lineage checks that reduce manual investigation cycles.
  • Build and operate the Gold serving layer behind Helm's twelve analytical models: MTA, MMM, incrementality, LTV:CAC, reach/frequency, cohorts, cobrand overlap, channel viability, card activation, product holdings, scenario reallocation, and channel universe.
  • Meet the serving contract required by the agentic layer, including fixed signal shapes, live queries on every turn, predictable latency, and per-client isolation.
  • Own CI/CD, infrastructure-as-code, cost management, and pipeline observability for the Databricks environment.

Desired Qualifications

  • Experience using Databricks AI Functions (ai_query) or Vertex AI inside pipelines for document extraction and normalization.
  • Experience with dbt or a comparable transformation and lineage framework.
  • Knowledge of identity resolution at the pipeline layer, including deterministic matching, RampID or UID2 linkage, and match-rate monitoring.
  • Experience with LiveRamp, AMC, GMP, or XMi clean room connectors.
  • Familiarity with LangSmith or LangGraph as a data consumer and understanding of the agentic layer's serving-layer needs.

About the Company

Capgemini

Global consulting, technology services, outsourcing

Capgemini provides consulting, technology services, and outsourcing to help organizations transform digitally. Its offerings include strategy and transformation consulting, system integration, application development, IT infrastructure management, and managed IT operations. It differentiates itself with a global presence in over 50 countries and a focus on applying AI, cloud, and cybersecurity to real business problems. Its goal is to help clients navigate digital transformation and improve efficiency and innovation through tailored, contract-based or managed services.

Company size
10,001+
Company stage
IPO
Headquarters
Paris, France
Founded
1967

Recently Posted Jobs

Explore more opportunities at Capgemini

See more jobs