Principal Data Scientist

The Role

Digital Futures is supporting a large, regulated organisation to generate product insight from its high-volume generative-AI products. You will lead analysis of AI interaction data at scale, turning questions, answers, user behaviour, feedback and model signals into clear insights that shape product strategy and delivery.

Partnering with Product Owners, engineers and business leads, you will frame broad questions as well-defined analytical problems and translate complex analysis into decision-ready outputs. As a principal-level practitioner, you will also set the standard for analytical rigour across the team. The role combines hands-on data science with strong generative-AI evaluation, experimentation and communication skills, and a disciplined, responsible approach to sensitive user and interaction data.

Key Responsibilities

Insight and analysis

• - Lead analysis of high-volume AI interaction data, including questions, answers, user behaviour, feedback and model signals, to generate clear insights for product strategy and delivery.
• - Explore, profile, cleanse and transform complex data sets, identifying data-quality issues, bias, gaps and anomalies that could affect the reliability of conclusions.
• - Partner with Product Owners, engineers and business leads to turn broad questions into well-defined analytical problems, success measures and decision-ready outputs.
• - Proactively identify meaningful patterns, emerging user needs, product opportunities and risks from large volumes of daily interactions, rather than only responding to requested analysis.
• - Develop robust analyses using Python, SQL and appropriate statistical, machine-learning and natural-language-processing techniques, selecting methods that are proportionate, interpretable and fit for purpose.

Generative-AI evaluation and experimentation

• - Analyse and evaluate generative-AI and large-language-model behaviour, including differences in outputs across models, prompts, configurations and use cases, using this insight to improve product experiences, prompt approaches and model-selection decisions.
• - Design and evaluate experiments, including A/B tests and other test-and-learn approaches, to assess the impact of product, model, prompt and experience changes.
• - Develop trusted product metrics, dashboards and monitoring views using AWS analytics tooling, including Amazon QuickSight where appropriate, helping teams understand adoption, engagement, answer quality, user outcomes and behavioural change.

Data, governance and practice

• - Work closely with data engineers to improve the analytical data environment, define reusable data assets and ensure pipelines support timely, scalable and trusted insight generation.
• - Ensure analyses and insight products meet appropriate privacy, data-protection, security and responsible-AI requirements, including careful handling of user and interaction data.
• - Communicate findings, assumptions, limitations and recommendations clearly to technical and non-technical audiences, translating complex analysis into practical product decisions.
• - Apply and champion strong data-science engineering discipline, including version control, reproducible analysis, documentation, peer review and appropriate controls for sensitive data, mentoring other data scientists along the way.

Experience and Profile

• - Strong hands-on data-science experience, with a record of turning large, noisy and complex data sets into clear insights and measurable product or business outcomes.
• - Advanced Python and SQL skills, including data exploration, cleaning, transformation, statistical analysis, visualisation and building repeatable analytical workflows.
• - Experience working with high-volume event, behavioural, interaction or text data; experience with conversational AI, search, digital-product or customer-interaction data would be particularly valuable.
• - Strong practical understanding of frontier large language models, prompt engineering and generative-AI evaluation, including why different models, prompts, retrieval approaches or configurations can produce materially different outcomes.
• - Experience defining and using evaluation approaches for generative-AI products, including quality, relevance, groundedness, consistency, safety, user feedback and task-success measures.
• - Strong understanding of statistical methods and experimental design, including hypothesis testing, A/B testing, causal inference or other approaches for evaluating product impact.
• - Experience with AWS data and analytics services used for data science and insight generation, such as Amazon S3, Athena, Glue, Redshift, SageMaker and QuickSight, or closely comparable cloud tooling.
• - Ability to operate effectively in ambiguity: framing problems, challenging assumptions, defining useful measures and identifying opportunities that stakeholders have not yet articulated.
• - Experience working alongside data-engineering teams and an understanding of scalable analytical data environments, data modelling, lineage, data quality and governance.
• - Sound knowledge of data privacy, data protection and responsible-AI practices, including minimisation, appropriate use of sensitive data, bias, model limitations, explainability and secure handling of user data.
• - Strong visualisation, storytelling, communication and collaboration skills, with the confidence to explain complex findings, challenge constructively and influence product and engineering stakeholders.
• - Evidence of disciplined, collaborative technical practice, including Git, documentation, code review and reproducible analysis.

Benefits

We offer a competitive salary. Our benefits package is designed to support your wellbeing and development. This includes:

• - Group Life Assurance
• - Virtual GP and Online Physiotherapy
• - Confidential Helpline/Employee Assistance Programme
• - Wellbeing Studio & Digital Gym
• - Discounted shopping cards, e-vouchers & cashback offers
• - Percentage discounts & savings on brands online & in store
• - Statutory pension contribution - salary exchange scheme
• - 30 days annual leave, including UK Bank Holidays