Inactive
C

Capgemini

Global consulting, technology services, outsourcing

Databricks Lead

Engineering LeadershipPosted on Oct 21
Full-timeWarsaw, Poland

About the Job

Requirements

  • At least 8 years of experience in data engineering or data architecture, including at least 4 years on Databricks and several years leading engineers or delivery teams.
  • Delivery experience with at least two production lakehouse platforms end to end, covering platform setup, ingestion and pipelines, data model, analytics or machine-learning consumption, and operations.
  • Deep hands-on knowledge of Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL, and MLflow; infrastructure as code with Terraform and Databricks Asset Bundles; and CI/CD.
  • Strong expertise in one cloud, preferably Azure, and ability to work with AWS or GCP, including networking, identity, storage, security, and cost.
  • Experience with performance tuning and cost optimization of Databricks workloads.
  • Databricks Certified Data Engineer Professional certification.
  • Presales experience in solution design, estimation, and client presentations, including defending architecture and its cost to executives.
  • Experience leading and developing engineers.
  • Fluency in English.

Responsibilities

  • Own lakehouse architecture on client engagements, including account and workspace design, networking and identity, Unity Catalog, data model, compute and cost strategy, and CI/CD.
  • Lead Databricks programs end to end, including migrations from legacy warehouses and Hadoop, greenfield lakehouse builds, platform hardening, and cost reviews; set engineering standards for code review, testing, Databricks Asset Bundles, and Terraform.
  • Deliver data engineering, analytics, and machine learning on the platform, including ingestion with Lakeflow Connect, pipelines and data products, Databricks SQL and AI/BI, machine learning and generative AI with MLflow and Mosaic AI, performance tuning, and cost optimization.
  • Lead and grow the Databricks engineering team through hiring, technical coaching, certification planning, and staffing across engagements.
  • Run the technical side of bids, including solution design, estimates, proofs of concept, sizing, and cost models; present architecture choices and their costs to client architects and executives.
  • Implement Data & AI Governance designs on Databricks, including access models in Unity Catalog, lineage and audit evidence, data quality in pipelines, and model governance with MLflow.
  • Build reusable assets, including reference architectures, migration playbooks, ingestion and cost accelerators, platform standards, and a governance-by-design reference architecture developed with the governance team.
  • Track the Databricks roadmap, including Lakeflow, Mosaic AI, AI/BI, Lakebase, and serverless, and decide what to adopt, when, and for which clients.

Desired Qualifications

  • Experience with analytics and machine-learning workloads such as Databricks SQL, AI/BI, and Mosaic AI.
  • Further Databricks certifications or partner accreditations, such as Solutions Architect or Platform Administrator.
  • Experience in financial services, energy, pharma, or the public sector.

About the Company

Capgemini

Global consulting, technology services, outsourcing

Capgemini provides consulting, technology services, and outsourcing to help organizations transform digitally. Its offerings include strategy and transformation consulting, system integration, application development, IT infrastructure management, and managed IT operations. It differentiates itself with a global presence in over 50 countries and a focus on applying AI, cloud, and cybersecurity to real business problems. Its goal is to help clients navigate digital transformation and improve efficiency and innovation through tailored, contract-based or managed services.

Company size
10,001+
Company stage
IPO
Headquarters
Paris, France
Founded
1967

Recently Posted Jobs

Explore more opportunities at Capgemini

See more jobs