Lead Data Engineer

If you like providing efficient, effective solutions to complex challenges, join us at Acuity! Be at the heart of helping achieve extraordinary things with software and data for our customers!

About Us

Ascent has recently been acquired by Acuity Analytics.  This is both a significant milestone for us and a tremendous opportunity for you.  Acuity Analytics is a business with a strong global reputation, an impressive client base and ambitious growth plans.  We deliver deep insights and domain-led digital transformation to high-growth and heavily regulated organisations.  To our customers, we bring a partnership that provides the talent, technology and capability to enhance performance and operational efficiency.

About the role

We are looking for an experienced Lead Data Engineer with 7+ years' commercial experience to lead the design, delivery, and continuous evolution of our modern data platform built on Microsoft technologies.

This is a hands-on leadership role, combining technical expertise with people management, where you'll lead a team of Data Engineers while delivering scalable, secure, and high-performing data solutions across Databricks, Microsoft Fabric, Azure Synapse, and Azure.

You'll work closely with architects, stakeholders, and delivery teams to define technical direction, establish engineering best practices, and ensure successful project delivery. This is an opportunity to play a key role in shaping our Data Engineering capability while mentoring and developing a high-performing engineering team.

Key Responsibilities
 

• Lead and mentor a team of Data Engineers, supporting their technical development and delivery.
• Act as the technical SME for Data Engineering, providing guidance and support to other engineers.
• Lead technical delivery across projects, defining solutions and implementation approaches.
• Design, build, and optimise scalable data platforms using Microsoft Fabric, Databricks, Azure Synapse, and Azure technologies.
• Develop and maintain robust data pipelines using PySpark, Spark SQL, and Azure Data Factory, Synapse, or Fabric Pipelines.
• Scope technical tasks and provide accurate estimates for delivery.
• Review code and establish engineering standards and best practices across the team.
• Build and optimise SQL solutions, including complex queries, stored procedures, and database security.
• Manage Azure DevOps Boards, Repositories, and CI/CD Pipelines throughout the development lifecycle.
• Collaborate with Solution Architects, Product Owners, and stakeholders to deliver high-quality data solutions.
• Drive Agile delivery, sprint planning, technical reviews, and continuous improvement initiatives.

Skills Required

Leadership & Delivery

• Experience leading and mentoring Data Engineering teams.
• Strong stakeholder management and communication skills.
• Experience leading technical projects and Agile delivery teams.
• Ability to scope technical work and define implementation approaches.
• Ownership of code quality, code reviews, and engineering best practices.

Data Engineering

• Strong commercial experience with Microsoft Fabric, Databricks, or Azure Synapse.
• Expert knowledge of PySpark and Spark SQL, with the ability to act as a technical SME for the wider engineering team.
• Strong SQL skills, including query optimisation, stored procedures, and database security.
• Experience building and managing ETL/ELT pipelines.
• Pipeline orchestration using Azure Data Factory, Synapse Pipelines, or Microsoft Fabric.
• Experience developing scalable cloud-based data platforms.

Cloud & DevOps

• Azure DevOps (Boards, Repositories, and CI/CD Pipelines).
• Feature branching, deployment, and release management.
• CI/CD best practices for modern data platforms.

Nice to Have

Emerging Technologies

• Experience with Graph Databases (Neo4j).
• Experience with Snowflake.
• Experience using AI-assisted development tools such as Claude and GitHub Copilot.
• Experience with Azure DevOps Pipelines.
• Experience working across Microsoft data platforms including Fabric, Databricks, and Synapse.

Certifications (Preferred)

• Microsoft Certified: Fabric Analytics Engineer Associate.
• Microsoft Certified: Fabric Data Engineer Associate.
• Databricks Professional Certification.
• Snowflake Certification.

Why Join Us?
 

• Lead and grow a talented team of Data Engineers.
• Play a key role in shaping our Nearshore Data Engineering capability.
• Work across modern Microsoft technologies including Fabric, Databricks, Synapse, and Azure.
• Blend hands-on technical leadership with people management.
• Collaborate on high-impact enterprise data transformation projects with leading clients.

People are at the Heart of our Business.  By investing in people, we achieve exceptional results for our clients and create new opportunities for our teams to thrive.  Check out this page for more details.

If you have any questions contact our Talent Acquisition team on ta.admin@acuityanalytics.

For more details about Acuity Analytics please see here: Read here.