Q

Quilter

Wealth management services and investment solutions

Data Engineer

Data EngineeringPosted on Nov 13
Full-timeNo salary listed

About the Job

Requirements

  • A degree in a technical discipline such as Computer Science, Engineering, Mathematics, or Physics, or equivalent practical experience.
  • Proven experience as a Data Engineer delivering data pipelines, data integrations, and data products in a cloud-based enterprise environment.
  • Hands-on experience designing, developing, testing, and supporting data solutions using Databricks within Azure.
  • Experience developing batch and near real-time data processing solutions using modern data engineering technologies and patterns.
  • Experience building and maintaining scalable, reliable, and secure data pipelines for reporting, analytics, data science, and operational use cases.
  • Understanding of data quality, validation, monitoring, and observability concepts and their application to data engineering solutions.
  • Knowledge of data governance principles, including data lineage, metadata management, security, and access control practices.
  • Experience working in regulated environments and applying data protection, information security, and data retention requirements.
  • Experience applying engineering standards and development best practices, including source control, testing, automation, and CI/CD.
  • Experience contributing to code reviews, peer reviews, and collaborative engineering activities.
  • Knowledge of data modelling concepts, including dimensional modelling, lakehouse architectures, and modern data platform design principles.
  • Experience working within Agile delivery teams and collaborating with architects, analysts, product owners, platform teams, and business stakeholders.
  • Strong technical and analytical skills to develop high-quality, maintainable, and scalable data solutions.
  • Strong problem-solving capabilities and a structured approach to troubleshooting and resolving technical issues.
  • Strong communication and collaboration skills to work effectively within cross-functional teams.
  • Knowledge of Databricks, including Delta Lake, Unity Catalog, Workflows, Notebooks, Asset Bundles, Delta Live Tables, Spark Declarative Pipelines, and platform optimisation principles.
  • Awareness of Microsoft Fabric Lakehouse, Data Warehouse, Real-Time Intelligence, Semantic Models, and Fabric engineering concepts.
  • Knowledge of Azure data services, including ADLS Gen2, Azure Data Factory, Azure Key Vault, Azure Monitor, and Azure Functions.
  • Understanding of Spark distributed processing, performance optimisation, partitioning, and workload tuning.
  • Strong SQL and Python development skills.
  • Knowledge of CI/CD and DevOps, including Azure DevOps, Git-based development practices, automated deployments, testing frameworks, and Infrastructure as Code concepts.
  • Knowledge of orchestration tools and approaches, including Databricks Workflows, Azure Data Factory, Fabric Pipelines, and enterprise scheduling frameworks.
  • Understanding of data architecture and modelling, including dimensional modelling, Data Vault concepts, medallion architecture, and lakehouse design approaches.
  • Understanding of secure-by-design principles, data governance controls, access management, and operational monitoring practices.

Responsibilities

  • Design, develop, test, and maintain data pipelines, integrations, and data products across the enterprise data platform.
  • Build scalable and reusable solutions for data ingestion, transformation, enrichment, and consumption.
  • Develop and optimise batch, near real-time, and streaming data processing solutions where required.
  • Ensure data solutions are robust, performant, secure, and maintainable.
  • Contribute to delivery of analytical, operational, and regulatory data capabilities.
  • Develop and maintain solutions using Azure Data Factory, Azure Data Lake Storage, Databricks, Delta Lake, Azure Key Vault, and related Azure technologies.
  • Support ongoing enhancement of platform capabilities, standards, frameworks, and reusable components.
  • Implement infrastructure and configuration changes through automated deployment processes.
  • Contribute to platform modernisation initiatives and technology adoption activities.
  • Follow established engineering standards, coding practices, and architectural principles.
  • Maintain source control and deployment processes using Azure DevOps and Git-based workflows.
  • Create and maintain automated testing and validation processes.
  • Participate in code reviews and peer reviews to promote quality, consistency, and knowledge sharing.
  • Ensure solutions are appropriately documented and supportable.
  • Apply data governance, security, privacy, and retention requirements throughout the engineering lifecycle.
  • Ensure data solutions comply with internal policies, regulatory obligations, and security standards.
  • Support implementation of role-based access controls, data classification requirements, and auditability standards.
  • Contribute to maintaining accurate metadata, lineage, and data quality controls.
  • Identify and escalate operational, security, and data risks where appropriate.
  • Monitor and support production data products and services.
  • Investigate and resolve incidents, defects, and performance-related issues.
  • Participate in root cause analysis and implement preventative improvements.
  • Support service resilience through monitoring, alerting, automation, and operational best practices.
  • Contribute to capacity planning, performance tuning, and cost optimisation activities.
  • Work with Product Owners, Analysts, Architects, Platform Engineers, and business stakeholders to understand requirements and priorities.
  • Interpret business requirements and translate them into technical solutions.
  • Provide updates on delivery progress, risks, issues, and dependencies.
  • Support user adoption by ensuring solutions are reliable, well-documented, and aligned with business needs.
  • Identify opportunities to improve engineering practices, tooling, automation, and delivery processes.
  • Stay current with developments across Azure, Databricks, data engineering, and cloud technologies.
  • Contribute ideas and recommendations to improve platform efficiency, reliability, and scalability.
  • Participate in knowledge sharing, technical communities, and continuous learning activities.

Desired Qualifications

  • Relevant industry certifications in Azure, Databricks, Data Engineering, or cloud technologies are desirable but not essential.
  • Experience with additional programming languages such as Scala is desirable.
  • A proactive approach to problem solving and continuous improvement.
  • Taking ownership of work deliverables and following tasks through to completion.
  • Collaborating with teams and valuing knowledge sharing and constructive feedback.
  • Commitment to delivering high-quality, reliable, and supportable engineering solutions.
  • Curiosity and adaptability, with a desire to learn new technologies, tools, and engineering practices.
  • Balancing technical excellence with a pragmatic, delivery-focused mindset.
  • Attention to detail and appreciation for data quality, security, and governance.

About the Company

Quilter

Wealth management services and investment solutions

Quilter provides wealth management and financial planning services in the United Kingdom. It helps people manage investments, pensions, and savings across the entire investment journey, offering products such as investment portfolios, financial advice, and retirement planning through a broad set of services. The company stands out by serving as a large, established UK provider with a wide range of products tied to every stage of a client’s financial life, aiming to support clients through education and guidance as their money grows. Quilter’s goal is to create brighter financial futures for every generation.

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
London, United Kingdom
Founded
2007

Recently Posted Jobs

Explore more opportunities at Quilter

See more jobs