Data Engineering Manager

Data Engineering Manager

This role has been designed as ‘Hybrid’ with an expectation that you will work on average 2 days per week from an HPE office.

Job Description:

HPE Operations is our innovative IT services organization. It provides the expertise to advise, integrate, and accelerate our customers’ outcomes from their digital transformation. Our teams collaborate to transform insight into innovation. In today’s fast paced, hybrid IT world, being at business speed means overcoming IT complexity to match the speed of actions to the speed of opportunities. Deploy the right technology to respond quickly to market possibilities. Join us and redefine what’s next for you.

What you’ll do:

We are seeking a Data Engineering Manager to lead, manage, and deliver complex data engineering projects. This role requires a strong blend of technical expertise, project management skills, stakeholder collaboration, and leadership. The ideal candidate will ensure the successful delivery of data solutions aligned with business objectives, driving innovation and operational excellence across the organization. Also collaborate with cross-functional teams to understand data requirements and optimize systems for data analytics and machine learning applications.

Key Responsibilities:

Delivery Management: Plan, execute, and manage end-to-end delivery of data engineering projects, ensuring they meet quality, timeline, and budgetary requirements. Implement best practices in Agile, Scrum, or other relevant methodologies for iterative and efficient project delivery. Establish robust mechanisms to monitor project progress, identify risks, and ensure proactive resolution.

Technical Leadership: Provide guidance on the design, development, and deployment of scalable data pipelines, ETL/ELT processes, and data storage solutions. Oversee the integration of cloud-based or on-premises data platforms.

Team Leadership: Build, mentor, and lead a high-performing team of data engineers, fostering a culture of collaboration, innovation, and accountability. Conduct performance reviews, provide feedback, and develop professional growth plans for team members.

Stakeholder Engagement: Act as a primary point of contact for stakeholders, ensuring alignment of technical solutions with business needs. Translate business requirements into technical deliverables and prioritize tasks effectively. Collaborate with cross-functional teams, including product managers, data scientists, and analysts, to deliver cohesive solutions.

Technical Expertise:

• Architect and develop end-to-end data pipelines that efficiently ingest, process, and store structured and unstructured data from diverse sources.
• Leverage batch and real-time processing methods to handle various data flows, ensuring that both historical and streaming data are seamlessly integrated.
• Implement automated data validation and quality checks at each stage of the pipeline to ensure that data is accurate, complete, and consistent.
• Define and implement data transformation rules that standardize, clean, and enrich data to make it usable for analysis and reporting.
• Design efficient ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) processes that balance processing time with resource utilization.
• Set up monitoring and alerting systems to track the health and performance of data pipelines, ensuring that any failures or performance issues are quickly identified and resolved
• Collaborate with data architects and infrastructure teams to integrate data pipelines with other systems, ensuring seamless data movement across the organization.
• Ensure proper lineage and governance of the data as it flows through the pipeline, making it easier to track, audit, and comply with regulatory requirements.
• Develop comprehensive proofs of concept (POCs) for our core platforms, CDP and EDF, focusing on their ecosystem components. This will provide a solid foundation for most of our projects and accelerate initial setup and configuration.
• Design and implement reusable templates for common data pipeline patterns, covering ingestion, transformation, and loading processes. These templates will serve as starting points for new projects, reducing development time.

What you need to bring:

• The overall experience is between 10-15 years
• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Proficiency in programming languages such as Python or Java.
• Extensive experience working with big data technologies such as Airflow, Hadoop, Spark, Kafka,
• Strong proficiency with SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra, Cloudera).
• Experience in on prem data engineering open source tools.
• Strong experience in data modeling, data warehousing, and ETL frameworks.
• Familiarity with data governance practices, data privacy, and security standards.
• Experience with containerization and orchestration tools like Docker and Kubernetes.
• Good to have databricks and Snowflake experience
• Knowledge of CI/CD pipelines, version control systems, and agile methodologies.
• Good to have: Experience in building AI systems using data

Accountability, Accountability, Action Planning, Active Learning (Inactive), Active Listening, Bias, Business Growth, Business Planning, Cloud Computing, Cloud Migrations, Coaching, Commercial Acumen, Creativity, Critical Thinking, Cross-Functional Teamwork, Customer Experience Strategy, Data Analysis Management, Data Collection Management (Inactive), Data Controls, Design Thinking, Empathy, Follow-Through, Growth Mindset, Hybrid Clouds, Infrastructure as a Service (IaaS) {+ 10 more}

Health & Wellbeing

We strive to provide our team members and their loved ones with a comprehensive suite of benefits that supports their physical, financial and emotional wellbeing.

Personal & Professional Development

We also invest in your career because the better you are, the better we all are. We have specific programs catered to helping you reach any career goals you have — whether you want to become a knowledge expert in your field or apply your skills to another division.

Diversity, Inclusion & Belonging

We are unconditionally inclusive in the way we work and celebrate individual uniqueness. We know diverse backgrounds are valued and succeed here. We have the flexibility to manage our work and personal needs. We make bold moves, together, and are a force for good.

#india#operations

Job:

Services

Job Level:

Manager_1

Hewlett Packard Enterprise is EEO F/M/Protected Veteran/ Individual with Disabilities.

HPE will comply with all applicable laws related to employer use of arrest and conviction records, including laws requiring employers to consider for employment qualified applicants with criminal histories.