Data Engineering Associate Principal Engineer

Overview

As a data engineering lead, you will be the key technical expert overseeing PepsiCo's data product build & operations and drive a strong vision for how data engineering can proactively create a positive impact on the business. You'll be empowered to create & lead a strong team of data engineers who build data pipelines into various source systems, rest data on the PepsiCo Data Lake, and enable exploration and access for analytics, visualization, machine learning, and product development efforts across the company. As a member of the data engineering team, you will help lead the development of very large and complex data applications into public cloud environments directly impacting the design, architecture, and implementation of PepsiCo's flagship data products around topics like revenue management, supply chain, manufacturing, and logistics. You will work closely with process owners, product owners and business users. You'll be working in a hybrid environment with in-house, on-premises data sources as well as cloud and remote systems.

Responsibilities

• Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
• Create reusable accelerators and solutions to migrate data from legacy data warehouse platforms such as Teradata to Azure Databricks and Azure SQL.
• Enable and accelerate standards-based development prioritizing reuse of code, adopt test-driven development, unit testing and test automation with end-to-end observability of data
• Build and own the automation and monitoring frameworks that captures metrics and operational KPIs for data pipeline quality, performance and cost.
• Collaborate with internal clients (product teams, sector leads, data science teams) and external partners (SI partners/data providers) to drive solutioning and clarify solution requirements.
• Define and manage SLA’s for data products and processes running in production.

Qualifications

• 9+ years of overall technology experience that includes at least 5+ years of hands-on software development, data engineering, and systems architecture.
• 4+ years of experience with Data Lake Infrastructure, Data Warehousing, and Data Analytics tools.
• 4+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala etc.).
• 3+ years in cloud data engineering experience in Azure.
• Fluent with Azure cloud services. Azure Certification is a plus.
• Experience with data modelling, data warehousing, and building high-volume ETL/ELT pipelines.
• Experience with running and scaling applications on the cloud infrastructure and containerized services like Kubernetes.