Senior Data Engineer

Overview

About Business Unit:

A consolidated team of database technology professionals, we deliver expert insights across all major database engines and Big Data solutions to support Epsilon products and platforms. The team has a one-of-a-kind alignment that combines a horizontal organizational structure with vertically aligned sub-groups dedicated to our clients. By adhering to consistent standards and methodologies, the team ensures operational excellence and has close ties with product teams at Oracle, IBM, Cloudera, Amazon and Microsoft.

We are seeking an experienced Senior Databricks Engineer to design, implement, automate, and support enterprise-scale Databricks platforms across AWS, Azure, and GCP. The role will focus on Databricks workspace management, Lakehouse architecture, cloud migrations, platform automation, governance, security, cost optimization, and operational excellence. The candidate will work closely with Data Engineering, Decision Science, Product Engineering, Cloud Infrastructure, and Security teams to deliver scalable and secure data platforms.

Click here to view how Epsilon transforms marketing with 1 View, 1 Vision and 1 Voice.

Responsibilities

• Databricks Platform Engineering – Own the design, deployment, administration, and lifecycle management of enterprise Databricks workspaces across AWS, Azure, and GCP.
• Cloud & Infrastructure Integration – Enable secure connectivity, authentication, networking, and integration between Databricks and enterprise cloud platforms and databases.
• Migration & Modernization – Lead migration of legacy data platforms, analytics workloads, and databases to the Databricks Lakehouse architecture.
• Automation & DevOps – Drive Infrastructure-as-Code, CI/CD, and operational automation to improve platform scalability, consistency, and efficiency.
• Security & Governance – Implement enterprise-grade security, access controls, compliance, auditing, and data governance using Unity Catalog and identity integrations.
• Performance & Cost Optimization – Continuously optimize Databricks workloads, storage layouts, and compute resources to improve performance and reduce cloud spend.
• Monitoring & Operational Support – Establish proactive monitoring, incident management, troubleshooting, and platform reliability practices for business-critical workloads.
• AI & Innovation – Evaluate and implement Databricks AI, GenAI, and automation capabilities to enhance platform operations and business value.
• Stakeholder – Partner with architects, engineering teams, security teams, and business partners while mentoring engineers and driving platform standards.

Qualifications

• 6+ years overall IT experience with 3+ years in Databricks.
• Solid experience with Databricks administration and platform engineering.
• Expertise in Spark, PySpark, SQL, Delta Lake, Unity Catalog, and Workflows.
• Experience with AWS, Azure, and/or GCP.
• Strong Python scripting and automation experience.
• Experience with Terraform and Infrastructure as Code.
• Hands-on experience with CI/CD pipelines and Git-based deployments.
• Good understanding of networking, private link connectivity, SSO, SAML/OIDC, and security concepts.
• Experience with PostgreSQL, SQL Server, Oracle, or cloud-native databases.
• Knowledge of data governance, metadata management, and compliance controls.

Preferred Skills:

• Databricks Certified Professional Data Engineer and Cloud certifications (AWS/Azure/GCP)
• Experience with Lakehouse migrations.
• Knowledge of Kafka, ELK, Prometheus, Grafana, and cloud observability tools.
• Experience with MLflow, Databricks Genie, and GenAI solutions.
• Experience supporting large-scale enterprise analytics platforms.
• Exposure to multi-cloud environments and global data platforms.

Education/Certification:

B.S./B.E in related field or equivalent work experience required. Databrick/AWS certification is a plus.

Additional Information

Epsilon is a global data, technology and services company that powers the marketing and advertising ecosystem. For decades, we’ve provided marketers from the world’s leading brands the data, technology and services they need to engage consumers with 1 View, 1 Vision and 1 Voice. 1 View of their universe of potential buyers. 1 Vision for engaging each individual. And 1 Voice to harmonize engagement across paid, owned and earned channels.

Epsilon’s comprehensive portfolio of capabilities across our suite of digital media, messaging and loyalty solutions bridge the divide between marketing and advertising technology. We process 400+ billion consumer actions each day using advanced AI and hold many patents of proprietary technology, including real-time modeling languages and consumer privacy advancements. Thanks to the work of every employee, Epsilon has been consistently recognized as industry-leading by Forrester, Adweek and the MRC. Epsilon is a global company with more than 9,000 employees around the world.

Our pillars aren't just words. They're how we show up every day.

• People centricity: We focus on employee well-being in an environment where colleagues truly care about each other.
• Collaboration: We work together, support one another, and collectively achieve goals.
• Growth: There are endless opportunities for growth through learning, development and career advancement.
• Innovation: We drive progress through cutting-edge solutions and forward-thinking approaches.
• Flexibility: We’ve created a balance between work and personal life, and we encourage adaptability to solve problems creatively.

Our values guide us to create value for our clients, our people and consumers.

• Act with integrity
• Work together to win together
• Innovate with purpose
• Respect all voices
• Empower with accountability

These pillars and values are our foundation—shaping our culture, guiding our decisions, and uniting us in common purpose.

Epsilon is an Equal Opportunity Employer. 
Epsilon is committed to promoting diversity, inclusion, and equal employment opportunities by using reasonable efforts to attract, recruit, engage and retain qualified individuals of all ethnicities and backgrounds, including, but not limited to, women, people of color, LGBTQ individuals, people with disabilities and any other underrepresented groups, traits or characteristics.