AWS Data bricks : Senior Manager

Line of Service

Advisory

Industry/Sector

Not Applicable

Specialism

Analytics

Management Level

Senior Manager

Job Description & Summary

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth.

In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.

Growing as a strategic advisor, you leverage your influence, expertise, and network to deliver quality results. You motivate and coach others, coming together to solve complex problems. As you increase in autonomy, you apply sound judgment, recognising when to take action and when to escalate. You are expected to solve through complexity, ask thoughtful questions, and clearly communicate how things fit together. Your ability to develop and sustain high performing, diverse, and inclusive teams, and your commitment to excellence, contributes to the success of our Firm.

Examples of the skills, knowledge, and experiences you need to lead and deliver value at this level include but are not limited to:

• Craft and convey clear, impactful and engaging messages that tell a holistic story.
• Apply systems thinking to identify underlying problems and/or opportunities.
• Validate outcomes with clients, share alternative perspectives, and act on client feedback.
• Direct the team through complexity, demonstrating composure through ambiguous, challenging and uncertain situations.
• Deepen and evolve your expertise with a focus on staying relevant.
• Initiate open and honest coaching conversations at all levels.
• Make difficult decisions and take action to resolve issues hindering team effectiveness.
• Model and reinforce professional and technical standards (e.g. refer to specific PwC tax and audit guidance), the Firm's code of conduct, and independence requirements.

At PwC - AC, as an AWS Senior Manger, the candidate will interact with Offshore Manager/ Onsite Business Analyst to understand the requirements and the candidate is responsible for end-to-end implementation of Cloud data engineering solutions like Data Lakehouse. The candidate should have strong experience in AWS cloud technology and Databricks. Strong in planning and organization skills. Ability to work as cloud developer/lead on an agile team and provide automated cloud solutions.

Years of Experience: Candidates with 12+ years of experience in architecting and delivering scalable big data pipelines using Apache Spark and Databricks on AWS.

Position Requirements:

Must Have:

• Design, build, and maintain scalable data pipelines using Databricks and Apache Spark.
• Good knowledge on Medallion Architecture in Databricks Lakehouse
• Develop and optimize ETL/ELT processes for structured and unstructured data.
• Implement Lakehouse architecture for efficient data storage, processing, and analytics.
• Orchestrating ETL/ELT Pipelines: Design and manage data workflows using Databricks Workflows, Jobs API.
• Work with AWS Data Services (S3, Lambda, CloudWatch) for seamless integration.
• Performance Optimization: Optimize queries using pushdown capabilities and indexing strategies.
• Implement data governance with Unity Catalog, security policies, and access controls.
• Collaborate with data scientists, analysts, and engineers to enable advanced analytics.
• Monitor, troubleshoot, and improve Databricks jobs and clusters.
• Strong expertise in end-to-end implementation of migration projects to AWS Cloud
• Should be aware of Data Management concepts and Data Modelling
• AWS & Python Expertise with hands-on cloud development.
• Spark Performance Tuning: Core, SQL, and Streaming.
• Orchestration: Airflow
• Code Repositories: Git, GitHub.
• Strong in writing SQL
• Experience in Teradata, DataStage , SSIS.
• Knowledge on Lakehouse Federation
• Knowledge of Delta Lake.
• Knowledge of Databricks Delta Live Table.
• Streaming: Kafka, Spark Streaming.
• Knowledge of CICD
• Knowlege on integrating 3party APIs to Databricks.
• Cloud Data Migration: Deep understanding of processes.
• Strong Analytical, Problem-Solving & Communication Skills.

Good to have Knowledge / Skills:

• Experience in Mainframe (Cobol, JCL, Zeke Scheduler)
• IaC & Automation: Terraform for Databricks deployment.
• Knowledge of Transport & Mobility domain.

Professional and Educational Background:

• BE / B.Tech / MCA / M.Sc / M.E / M.Tech / MBA

Certification:

Amazon Web Services (AWS) certifications (AWS certified Data Engineer is recommended). 

Amazon Web Services (AWS) certifications (AWS Solutions Architect Associate/Professional).

Databricks Certified Associate Developer for Apache Spark

Additional Information:

• Travel Requirements: Travel to client locations may be required as per project requirements.
• Line of Service: Advisory
• Horizontal: Technology Consulting
• Designation: Manager
• Location: Bangalore, India

Education (if blank, degree and/or field of study not specified)

Degrees/Field of Study required:Degrees/Field of Study preferred:

Certifications (if blank, certifications not specified)

Required Skills

Optional Skills

Accepting Feedback, Accepting Feedback, Active Listening, Agile Scalability, Amazon Web Services (AWS), Analytical Thinking, Apache Hadoop, Azure Data Factory, Coaching and Feedback, Communication, Creativity, Data Anonymization, Database Administration, Database Management System (DBMS), Database Optimization, Database Security Best Practices, Data Engineering, Data Engineering Platforms, Data Infrastructure, Data Integration, Data Lake, Data Modeling, Data Pipeline, Data Quality, Data Transformation {+ 26 more}

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Not Specified

Available for Work Visa Sponsorship?

No

Government Clearance Required?

No

Job Posting End Date