M

Mastercard

Processes payments across a global network

Lead Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeO'Fallon, MO, USA

About the Job

Requirements

  • Deep expertise in designing and implementing large-scale data engineering solutions and distributed data processing systems.
  • Advanced proficiency with Databricks, Apache Spark and PySpark, Delta Lake, SQL, and Python.
  • Experience building and operating cloud-based data platforms using Azure, AWS, or Google Cloud Platform.
  • Strong experience developing enterprise-grade ETL/ELT pipelines, data integration frameworks, and streaming architectures.
  • Strong understanding of data modeling and data architecture for analytical and operational workloads.
  • Experience implementing data quality, lineage, metadata management, observability, and data governance frameworks in enterprise environments.
  • Experience working with modern data formats and storage technologies, including Parquet, Avro, and ORC.
  • Working knowledge of continuous integration and continuous delivery, infrastructure as code, automated testing, Git/GitHub, and DevOps practices for data engineering environments.
  • Experience with workflow orchestration tools, such as Apache Airflow.
  • Strong understanding of security, privacy, regulatory, and compliance requirements associated with sensitive financial and customer data.
  • Proven ability to lead complex technical initiatives across multiple teams and influence engineering direction without direct authority.
  • Demonstrated ability to mentor engineers and elevate technical capability, including providing design guidance, knowledge sharing, and constructive feedback.
  • Experience with Java-based application development is a strong plus, particularly in environments where data platforms integrate with enterprise applications and services.
  • Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related STEM discipline is preferred; equivalent practical experience will also be considered.

Responsibilities

  • Lead the design, development, and evolution of enterprise-grade data platforms and pipelines supporting credit risk products, decisioning capabilities, analytics, and regulatory reporting.
  • Architect and implement scalable ETL/ELT frameworks using Databricks, Apache Spark, Delta Lake, and cloud-native technologies to support high-volume data workloads.
  • Drive data platform modernization and cloud transformation, including migration of legacy data assets to modern cloud-based and Data Lakehouse architectures.
  • Establish and advance data quality, lineage, metadata, governance, observability, and monitoring capabilities to deliver trusted, compliant, and reliable data products.
  • Partner across Risk, Product, Data Science, Architecture, Engineering, and Business teams to translate complex requirements into scalable, sustainable technical solutions.
  • Define and promote engineering standards and best practices across coding, testing, data quality, deployment automation, documentation, and operational excellence.
  • Lead technical design reviews and influence architectural decisions for data-intensive applications, platforms, and services.
  • Optimize large-scale data workloads and platforms for performance, reliability, scalability, resiliency, and cost efficiency.
  • Build reusable, governed data products and capabilities that enable AI, machine learning, advanced analytics, and emerging risk use cases.
  • Drive technical initiatives from strategy through execution, identifying dependencies, managing technical risks, and ensuring solutions align with business priorities and long-term platform strategy.
  • Mentor and coach engineers across the organization, providing technical guidance, design expertise, knowledge sharing, and feedback to strengthen engineering capabilities.
  • Shape technology roadmaps and delivery priorities across the ECR portfolio, balancing business value, engineering feasibility, risk, compliance, and long-term platform sustainability.
  • Ensure engineering solutions meet security, privacy, regulatory, compliance, and audit requirements through appropriate controls, documentation, and engineering practices.
  • Communicate technical trade-offs to senior stakeholders and foster a culture of ownership, accountability, and continuous improvement.

Desired Qualifications

  • Preferred experience with Databricks, Apache Spark and PySpark, Hadoop, Delta Lake, SQL, Python, Apache Airflow, Azure Data Services, Kafka and event streaming, GitHub and continuous integration and delivery tooling, and cloud-based Data Lakehouse architectures.

About the Company

Mastercard

Processes payments across a global network

Mastercard operates a global payment network connecting financial institutions, merchants, businesses and consumers. It authorizes, clears and settles transactions between participating institutions, while banks issue Mastercard-branded cards and extend credit to their customers. Its services also include fraud detection, identity verification, payment tokenization, data analysis and consulting for organizations managing commerce and financial operations. The company earns network fees linked to payment volumes and processed transactions, along with fees for its services and solutions. Founded by a group of banks in 1966, Mastercard is headquartered in Purchase, within Harrison, New York.

Company size
10,001+
Company stage
IPO
Headquarters
Town of Harrison, New York
Founded
1966

Recently Posted Jobs

Explore more opportunities at Mastercard

See more jobs