Data Architect - Azure Databricks, Specialty Insurance

EXL (NASDAQ: EXLS) is a global data and AI company that offers services and solutions to reinvent client business models, drive better outcomes and unlock growth with speed. Bringing together domain expertise with robust data, powerful analytics, cloud and AI to create agile, scalable solutions and execute complex operations for the world’s leading corporations.

EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect, creating value from data to ensure faster decision-making and transforming operating models. Key industries include Insurance, Healthcare, Banking and Financial Services, Media, and Retail, among others. 

Headquartered in New York, our team is over 60,000 strong, with more than 50 offices spanning six continents. For information, visit www.exlservice.com.

 Role Title:  Data Architect (Azure Databricks, Specialty Insurance)

BU/Segment: Analytics / Data Management / Insurance

Location:   London, United Kingdom (Hybrid working)

Employment Type: Permanent or Umbrella Contractor (Inside IR35)

EXL delivers data and analytics solutions for its clients. As a Data Architect, you will join an engagement with a specialty insurer in the London Market to design, implement and document the data architecture and enterprise data models for a new Azure Databricks data platform. These solutions support enterprise information management, business intelligence, regulatory and actuarial reporting, machine learning and data science.

The Data Architect will translate client business needs into technical solutions, working within the client’s established data guidelines and approaches or helping to define and mature them as required. The role covers needs identification, data modelling, source-to-target mapping, and automation and testing strategies. The Data Architect will develop conceptual, logical and physical data models and work with data engineers to implement these on a lakehouse following medallion architecture principles on Azure Databricks.

The Data Architect will guide the growth of the client’s data architecture and the optimisation of query performance, working both independently and as part of a delivery team. As a senior member of the team, they are expected to:

• Mentor and coach engineers, analysts and client staff, and provide oversight as required

• Communicate clearly with technical and business stakeholders

• Solve problems independently and raise risks early

 As part of your duties, you will be responsible for:

Architecture and modelling

• Design the lakehouse architecture on Azure Databricks using Delta Lake and the Medallion pattern (bronze, silver and gold layers), organising data by quality and purpose to keep the platform scalable and maintainable.

• Design and apply dimensional models (e.g. star and snowflake schemas) and apply other major approaches such as ODS and Data Vault modelling where they suit the client’s needs.

• Create and maintain conceptual, logical and physical data models, with corresponding metadata, that translate business needs into long-term solutions. Cover core insurance domains such as policy, premium, claims, exposure, reinsurance and finance.

• Design gold-layer (consumption) models and, where Power BI is the consumption layer, semantic models served from Databricks SQL and shaped for efficient reporting and self-service analytics.

• Select and design for the right storage and exchange formats for each workload, including row-based and columnar storage, Delta and Iceberg table formats, and file and message formats (e.g. Parquet, CSV, Excel, XML, JSON, Protobuf).

• Optimise platform performance and cost, including partitioning vs. liquid clustering, file sizing, Photon, and cluster and SQL warehouse sizing.

• Manage the model lifecycle: optimise and update models for new and existing client projects, reverse engineer physical models from existing databases and SQL scripts and recommend opportunities for reuse.

• Map business processes to conceptual models, using techniques such as bus matrices to identify dimensions, facts and conformed dimensions.

Data strategy and integration

• Help set the client’s data architecture direction, including platform and technology choices across the Azure data estate (e.g. Databricks, ADLS Gen2, Azure Data Factory, Event Hubs), and document the rationale and trade-offs.

• Design and oversee the building of metadata-driven data ingestion frameworks on Databricks (e.g. Auto Loader, Lakeflow Declarative Pipelines, Workflows), using configuration rather than bespoke code to onboard new sources, handle schema change and apply consistent patterns for loading, logging and data quality checks.

• Define how data flows from source systems (e.g. policy administration, claims, finance, bordereaux and market data feeds) through to the consumption layer, resolving integration challenges along the way.

• Perform data analysis and profiling to support mapping and transformation from legacy systems and spreadsheets to target models. Validate data against business rules and source systems for accuracy and completeness and define the testing approach for data pipelines and models.

Standards and governance

• Develop and document naming conventions, coding practices, modelling standards and metadata management procedures, and evaluate data models and physical databases for variances against them.

• Define the data governance and security approach using Unity Catalog, including catalogue and schema design, role-based access control integrated with Microsoft Entra ID, row and column-level security, data masking, lineage, tagging and data quality frameworks, appropriate to sensitive and regulated insurance data.

Client delivery and collaboration

• Produce clear architecture deliverables, such as solution designs, architecture decision records and roadmaps, designed within client constraints such as security policies, data residency requirements and regulatory obligations (e.g. Lloyd’s, PRA/FCA, Solvency II, GDPR).

• Work with application development and data engineering teams to implement data strategies and build data flows, guiding them on project limitations, capabilities, performance requirements and interfaces.

• Define CI/CD and environment promotion patterns for the platform using Git, Databricks Asset Bundles and infrastructure as code (e.g. Terraform, Azure DevOps).

• Lead design workshops and coordinate with project managers and client business and technical staff, building trusted relationships and presenting recommendations to senior stakeholders.

• Support scoping, estimating and solution input for new and existing engagements.

• Mentor and coach team members and provide line management where the engagement requires it.

 Experience and Education required: 

Essential Experience and Skills

• 8+ years of experience in data architecture, data modelling or similar, including hands-on use of a data modelling tool (Erwin, ER/Studio, PowerDesigner or similar).

• Prior experience in commercial or specialty insurance, ideally the London Market, with familiarity with Lloyd’s, bordereaux, reinsurance, or regulatory and actuarial reporting.

• Hands-on experience designing solutions on Azure Databricks, including Delta Lake, Unity Catalog, Spark (PySpark and Spark SQL) and Databricks SQL.

• Experience with core Azure data services and their integration with Databricks, such as ADLS Gen2, Azure Data Factory, Microsoft Entra ID, Key Vault and secure networking.

• Demonstrable experience of dimensional modelling and layered architecture patterns such as Medallion, with a strong understanding of data warehousing and OLTP systems from an integration perspective.

• Strong SQL skills, including writing procedures, views and functions for data analysis, profiling and quality checks.

• Proficiency in Python (or a similar language) and experience with orchestration and transformation tooling (e.g. Databricks Workflows, Lakeflow Declarative Pipelines, Azure Data Factory, dbt).

• Experience designing metadata-driven data ingestion or ELT frameworks.

• Experience working with a range of data formats, including row-based and columnar storage, Delta and Iceberg table formats, and common file formats (e.g. CSV, XML, JSON).

• Experience applying data governance and security practices on a lakehouse platform, including access control, lineage and data quality frameworks.

• Client-facing experience, including stakeholder management and presenting designs to technical and non-technical audiences.

• Technical leadership experience, such as mentoring, leading design reviews and guiding delivery teams.

• Bachelor’s degree or equivalent experience.

Desirable Experience and Skills

• Experience in a consultancy or professional services environment, working across multiple clients or engagements.

• Databricks certification (e.g. Data Engineer Associate or Professional, or Azure Databricks Platform Architect).

• Experience designing gold-layer semantic models for Power BI, including how model design affects report performance.

• Line management experience.

• Master’s degree.

• Experience with Databricks Asset Bundles, Terraform and CI/CD for data platforms and data model changes.

• Experience with Delta Sharing, Lakehouse Federation or real-time/streaming workloads.

• Exposure to machine learning and AI workloads on Databricks (e.g. MLflow, Mosaic AI).

• Experience with other modern data platforms (e.g. Microsoft Fabric, Azure Synapse, Snowflake) or other clouds (AWS, GCP). Transferable experience is welcome, but not essential.

• Experience with enterprise cataloguing and metadata management tools beyond Unity Catalog (e.g. Microsoft Purview, Collibra), and designing metadata repositories and standards.

As part of a leading global Data and AI company, you can look forward to:

• A competitive salary with a generous bonus, private healthcare, critical illness life assurance at 4 x your annual salary, income protection insurance, and a rewarding pension.
• EXL provides everyday financial well-being solutions, such as cash back cards, in which you can earn cashback while enjoying discounts, promotions, and offers from top retailers. We also offer a Cycle Scheme where you can save money on bikes and cycling accessories.
• At EXL, we are committed to providing our employees with the tools and resources they need to succeed and excel in their careers. We offer a wide range of professional and personal development opportunities. We also support a range of learning initiatives that allow our employees to build on their existing skills and knowledge. From online courses to seminars and workshops, our employees have the opportunity to enhance their skills and stay up to date with the latest trends and technologies.
• As an Equal Opportunity Employer, EXL is committed to diversity. Our company does not discriminate based on race, religion, colour, national origin, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, age, or disability status.
• EXL employees are eligible to purchase stock as part of our Employee Stock Purchase Plan (ESPP).
• At EXL, we offer a flexible hybrid working model that allows employees to live a balanced, healthy lifestyle while strengthening our culture of collaboration.

To be considered for this role, you must already be eligible to work in the United Kingdom.