Head of Data Engineering

Purpose of Position:

The Head of Data Engineering will lead the development and evolution of the Pantheon Data Platform, establishing it as the core enabler of analytics and insight across the firm. This role focuses on the design, implementation, and optimization of robust, scalable data infrastructure—including data pipelines, data lakes, and integration frameworks—that power the firm’s decision-making capabilities. The successful candidate will ensure the platform delivers high-quality, timely, and accessible data to internal stakeholders, enabling self-service analytics and driving operational and investment excellence.

Key Responsibilities: 

• Partner with leaders from Technology, Transformation Office, Investment Team & Portfolio Management and Investor Relations groups to refine and execute on the firm’s data engineering strategy.
• Stay updated on private markets industry trends, regulatory needs, software vendor offerings, and investment team feedback to guide product decisions and maintain Pantheon’s competitive edge.
• Align our data strategy with firm priorities, establish the data sources, identify the tools needed, and oversee data governance and quality standards.
• Enhance the existing Pantheon Data Platform, including data lake, data pipelines and reporting infrastructure.
• Partner with the Change Office to deliver & enhance Pantheon’s data & analytics product (“Prism”) as required by strategic transformation & change initiatives or by projects initiated within the Pantheon Innovation Lab.
• Champion a user-centric approach to data pipelines and scalable data platforms, focusing on usability, intuitiveness, and overall user satisfaction.
• Establish relevant KPIs that measure the outcomes and impacts of our various data initiatives. Conduct regular monitoring and usage to validate and improve existing solutions, and deliver outcome-oriented impact analysis to stakeholders and leadership team.
• Manage the data team, which may include data analysts, data engineers, and related data professionals.
• Provide leadership, mentorship, and professional development opportunities for team members, fostering a culture of innovation and continuous improvement.
• Spearhead the deployment and operationalization of Azure Databricks and lakehouse architecture for scalable data processing.
• Build and nurture a community of data platform users across teams to foster adoption and data-driven decision making.
• Establish and manage end-to-end data engineering practices for acquiring, processing, and delivering structured and unstructured data.

Knowledge & Experience Required:

• Bachelor’s degree in Engineering, Technology, Data Science or a related field; an advanced degree is preferred
• Track record of working with and organizing data within financial services, asset management, private markets
• Crucial to have experience in deploying, managing and optimising Azure Databricks environments, with metadata management principles, and data governance best practices
• Strong experience managing work requests using the Scrum framework and utilising Azure DevOps to facilitate that work.
• Proficient in programming languages, such as Python, R, or SQL; frameworks, such as Teraform, PySpark, Spark and analytics tools such as Azure Synapse, and reporting platforms such as PowerBI & Tableau
• Proficiency in managing processes involving acquiring, storing, processing and delivering structured and unstructured data for analytical applications
• Demonstrated expertise in data governance, data quality management, and the implementation of analytics platforms. Ideally utilising the Unity Catalog functionality in Azure Databricks.
• Strong experience in managing and developing high-performing teams, with the ability to inspire and drive cross-functional collaboration. Experience leading and developing globally distributed internal teams
• Strong analytical and strategic thinking skills, with a focus on translating data into actionable business insights
• Strong problem-solving skills, attention to detail, and an ability to navigate complex data processes and organizations
• Excellent communication and interpersonal skills, with the ability to engage and influence stakeholders at all levels of the organization
• Experience with change management and leading digital transformation initiatives within an organization
• Experience in leading cross-functional initiatives, with strong communication skills, project management abilities, and organizational skills
• Comfort with industry leading private markets software products (e.g. pipeline management, portfolio company monitoring, investment book of record, portfolio construction, planning & reporting etc.). eFront experience is a plus.
• Experience using industry standard product management tools and technologies, e.g. Aha, Miro & Visio