MUFG

Global bank offering diversified financial services

AI Test Analyst

Test Automation EngineeringConfirmed live in the last 24 hours

Full-timeIndiaNo salary listedIn-person

About the Job

Requirements

At least 3 years of experience in Quality Engineering, Test Automation, and AI-driven testing; 2 or more years of experience in Retirement Solutions, Financial Services, or BFSI is preferred.
Strong experience designing, developing, and executing test automation solutions using at least one of Selenium, Appium, or UFT/OpenText, as well as tools and technologies such as Python, Maven, Postman, REST APIs, SoapUI, Cortex, and Snowflake.
Hands-on experience with AI-assisted testing tools and technologies for test case generation, test optimization, defect prediction, intelligent test execution, quality analytics, and test automation.
Experience validating AI/ML-enabled applications, including model testing, data quality validation, prompt testing, response evaluation, hallucination detection, bias and fairness testing, explainability validation, adversarial testing, and assessment of model accuracy, reliability, and performance.
Strong understanding of software testing principles, methodologies, and best practices, including test planning, test design, test execution, defect management, reporting, governance, and quality assurance processes.
Understanding of software development life cycle and software quality assurance processes across Agile, Waterfall, and DevOps delivery models.
Experience working with CI/CD pipelines and DevOps practices using Jenkins, Azure DevOps, Azure Repos, Bitbucket, Git, Docker, and Kubernetes.
Experience with cloud platforms, data validation, API testing, integration testing, and end-to-end testing across complex enterprise environments.
Ability to define quality standards, automate validation processes, and ensure reliable, scalable, and secure AI-driven applications.
Strong analytical and problem-solving skills.
Attention to detail and a quality-driven mindset.
Good communication and teamwork skills.
Proactive attitude and eagerness to learn.

Responsibilities

Develop and execute testing strategies for AI, machine learning, and generative AI solutions to ensure accuracy, reliability, performance, and business value.
Establish and maintain test automation frameworks and scripts supporting functional, integration, API, system, model validation, data validation, and end-to-end AI solution testing.
Validate AI model outputs against defined acceptance criteria, business requirements, and responsible AI standards.
Ensure testing coverage across the AI lifecycle, including data ingestion, preprocessing, model training, inference, output validation, and reporting.
Assess AI solution performance for accuracy, relevance, consistency, explainability, fairness, robustness, and reliability.
Integrate AI testing activities and automation frameworks and scripts with continuous integration and CI/CD pipelines and tools.
Collaborate with developers, business analysts, product owners, and project managers to define testing requirements, success criteria, and quality measures.
Manage automated test suite execution, monitor results, and work with teams to troubleshoot and resolve issues.
Identify opportunities to use AI-driven testing tools and automation techniques to improve testing efficiency and effectiveness.
Research emerging AI testing methodologies, tools, technologies, and industry best practices, and recommend continuous improvements.
Support enterprise standards, processes, and quality controls for AI testing and validation.
Design, develop, execute, and maintain test cases and scripts for AI, machine learning, and generative AI applications.
Validate data quality, data integrity, and data lineage across AI systems and supporting platforms.
Perform functional, integration, API, regression, user acceptance, and end-to-end testing of AI-enabled solutions.
Evaluate AI model outputs for accuracy, consistency, bias, hallucinations, and compliance with business requirements and existing processes.
Execute prompt testing, response validation, and scenario-based testing for generative AI solutions.
Monitor test execution results, analyze outcomes, and produce comprehensive test reports.
Identify, document, and track defects, model performance issues, and data quality concerns, and support their resolution.
Collaborate with business analysts, developers, and other stakeholders to understand requirements and deliver high-quality AI solutions.
Support test data preparation, environment validation, and model release verification.
Identify opportunities to improve AI testing frameworks, processes, and automation capabilities.
Share knowledge and support project teams on AI testing approaches, tools, and best practices.
Identify quality, model, data, security, privacy, ethical, and compliance risks associated with AI solutions and support mitigation activities.
Validate adherence to responsible AI principles, regulatory requirements, organizational policies, and governance standards.
Support assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
Maintain test evidence, validation reports, defect records, and audit documentation for governance and compliance requirements.
Escalate risks, issues, and control gaps that may affect AI solution quality, regulatory compliance, or business outcomes.
Design, develop, and execute comprehensive test strategies, plans, and cases for functional, integration, system, regression, API, and end-to-end testing across applications, data, and reporting layers.
Collaborate with business analysts, developers, product owners, and other stakeholders to define test scenarios, automate test cases, and ensure quality throughout the software development lifecycle.
Develop and maintain automated testing solutions using traditional and AI-powered testing tools to improve efficiency, coverage, and quality.
Validate AI-powered applications, intelligent agents, and machine learning models for accuracy, reliability, performance, fairness, explainability, and compliance with responsible AI principles.
Perform prompt testing, response validation, hallucination detection, adversarial testing, and evaluation of AI outputs against business requirements and expected outcomes.
Execute API, data, integration, and end-to-end testing to ensure data integrity, system interoperability, and reliable business process outcomes.
Manage defects and quality issues using tools such as JIRA and Azure DevOps, including defect tracking, root-cause analysis, reporting, and resolution support.
Produce test evidence, quality metrics, and test execution reports to support audit, compliance, and governance requirements.
Contribute to continuous improvement of testing frameworks, methodologies, and automation practices, identifying cost-effective solutions and opportunities for innovation.
Provide guidance, support, and training to team members on AI-enabled testing tools, automation technologies, and quality engineering best practices.

Desired Qualifications

At least 2 years of experience in Retirement Solutions, Financial Services, or BFSI domains.

Overview

The AI Test Analyst is responsible for ensuring the quality, reliability, and effectiveness of AI-powered solutions, including generative AI applications, intelligent agents, and AI-enabled business processes. The role focuses on validating AI models and outputs, assessing data quality, monitoring performance, and ensuring adherence to Responsible AI principles, regulatory requirements, and organizational standards.

The role supports the adoption of AI-driven testing practices by leveraging automation, innovative testing techniques, and AI-enabled tools to improve testing efficiency and coverage. It also contributes to the development of testing capabilities across the organization by promoting best practices and supporting the upskilling of testing teams in AI and automation technologies.

As MUFG Retirement Solutions continues to integrate AI across its products, services, and testing functions, the AI Test Analyst plays a key role in embedding AI into testing processes, identifying cost-effective quality assurance solutions, and ensuring that testing strategies align with business objectives, quality standards, and the organization's AI transformation agenda.

Key Accountabilities and main responsibilities

Strategic Focus

Develop and execute testing strategies for AI, Machine Learning (ML), and Generative AI solutions to ensure accuracy, reliability, performance, and business value.
Establish and maintain test automation frameworks and scripts that support various types of testing, such as functional testing, integration testing, API testing, system testing, model validation, data validation, API testing, system testing, and end-to-end AI solution testing.
Validate AI model outputs against defined acceptance criteria, business requirements, and responsible AI standards.
Ensure comprehensive testing coverage across the AI lifecycle, including data ingestion, preprocessing, model training, inference, output validation, and reporting.
Assess AI solution performance across dimensions such as accuracy, relevance, consistency, explainability, fairness, robustness, and reliability.
Integrate AI testing activities and test automation framework & scripts with continuous integration, CI/CD pipelines and tools to support continuous testing and deployment of AI solutions.
Collaborate with developers, business analysts, product owners, and project managers to define testing requirements, success criteria, and quality measures.
Manage the execution of automated test suites, monitor the test results and collaborate with teams to troubleshoot and resolve the issues promptly.
Identify opportunities to leverage AI-driven testing tools and automation techniques to improve testing efficiency and effectiveness.
Research emerging AI testing methodologies, tools, technologies, new test automation tools and industry best practices, providing recommendations for continuous improvement.
Support the establishment and adoption of enterprise standards, processes, and quality controls for AI testing and validation.

Operational Management

Design, develop, execute, and maintain test cases and test scripts for AI, ML, and Generative AI applications.
Validate data quality, data integrity, and data lineage across AI systems and supporting platforms.
Perform functional, integration, API, regression, user acceptance, and end-to-end testing of AI-enabled solutions.
Evaluate AI model outputs for accuracy, consistency, bias, hallucinations, and compliance with business requirements and existing processes.
Execute prompt testing, response validation, and scenario-based testing for Generative AI solutions.
Monitor test execution results, analyse outcomes, and produce accurate and comprehensive test reports.
Identify, document, track, and support resolution of defects, model performance issues, and data quality concerns.
Collaborate with business analysts, developers, and other stakeholders to understand requirements and deliver high-quality AI solutions.
Support test data preparation, environment validation, and model release verification activities.
Contribute to continuous improvement initiatives by identifying opportunities to enhance AI testing frameworks, processes, and automation capabilities.
Provide knowledge sharing and support to project teams regarding AI testing approaches, tools, and best practices.

Governance & Risk

Identify quality, model, data, security, privacy, ethical, and compliance risks associated with AI solutions and proactively support mitigation activities.
Validate adherence to Responsible AI principles, regulatory requirements, organizational policies, and governance standards.
Support the assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
Maintain appropriate test evidence, validation reports, defect records, and audit documentation to support governance and compliance requirements.
Escalate risks, issues, and control gaps that may impact AI solution quality, regulatory compliance, or business outcomes.

The above list of key accountabilities is not an exhaustive list and may change from time-to-time based on business needs.

Experience & Personal Attributes

Experience

3+ years of experience in Quality Engineering, Test Automation, and AI-driven testing, with preference for candidates having 2+ years of experience in Retirement Solutions, Financial Services, or BFSI domains.
Strong experience in designing, developing, and executing test automation solutions using tools such as Selenium, Appium, UFT/OpenText (at least one mandatory), Python, Maven, Postman, REST APIs, SoapUI, Cortex, and Snowflake.
Hands-on experience with AI-assisted testing tools and technologies for test case generation, test optimization, defect prediction, intelligent test execution, quality analytics, and test automation.
Experience in validating AI/ML-enabled applications, including model testing, data quality validation, prompt testing, response evaluation, hallucination detection, bias and fairness testing, explainability validation, adversarial testing, and assessment of AI model accuracy, reliability, and performance.
Strong understanding of software testing principles, methodologies, and best practices, including test planning, test design, test execution, defect management, reporting, governance, and quality assurance processes.
Solid understanding of SDLC and SQA processes across Agile, Waterfall, and DevOps delivery models.
Experience working with CI/CD pipelines and DevOps practices using Jenkins, Azure DevOps, Azure Repos, Bitbucket, Git, Docker, and Kubernetes.
Strong analytical and problem-solving skills with the ability to define quality standards, automate validation processes, and ensure reliable, scalable, and secure AI-driven applications.
Experience with cloud platforms, data validation, API testing, integration testing, and end-to-end testing across complex enterprise environments.
Design, develop, and execute comprehensive test strategies, test plans, and test cases for functional, integration, system, regression, API, and end-to-end testing across applications, data, and reporting layers.
Collaborate closely with Business Analysts, Developers, Product Owners, and other stakeholders to define test scenarios, automate test cases, and ensure quality throughout the software development lifecycle.
Develop and maintain automated testing solutions leveraging both traditional and AI-powered testing tools to improve testing efficiency, coverage, and quality.
Validate AI-powered applications, intelligent agents, and machine learning models by assessing model accuracy, reliability, performance, fairness, explainability, and compliance with Responsible AI principles.
Perform prompt testing, response validation, hallucination detection, adversarial testing, and evaluation of AI outputs against business requirements and expected outcomes.
Execute API, data, integration, and end-to-end testing to ensure data integrity, system interoperability, and reliable business process outcomes.
Manage defects and quality issues using tools such as JIRA and Azure DevOps, including defect tracking, root-cause analysis, reporting, and resolution support.
Produce test evidence, quality metrics, and test execution reports to support audit, compliance, and governance requirements.
Contribute to the continuous improvement of testing frameworks, methodologies, and automation practices, identifying cost-effective solutions and opportunities for innovation.
Provide guidance, support, and training to team members on AI-enabled testing tools, automation technologies, and quality engineering best practices.

Personal Attributes

Strong analytical and problem-solving skills
Attention to detail and quality-driven mindset
Good communication and teamwork skills
Proactive and eager to learn

About the Company

MUFG

Global bank offering diversified financial services

MUFG is a major global financial group that provides a wide range of banking and financial services, including commercial and trust banking, securities, credit cards, and asset management. Its products are delivered through a network of domestic and international banks, digital platforms, and financial advisers, offering loans, deposits, payments, investment services, and asset management. MUFG stands out due to its size and scale as one of the world’s largest financial groups, its diversified offerings across banking and securities, and its strong international presence (notably expanding in the U.S. after investing in Morgan Stanley in 2008). Its goal is to support economic growth worldwide by expanding its global footprint and promoting sustainable finance.

Company size: 10,001+
Company stage: IPO
Headquarters: Tokyo, Japan
Founded: 2006

Recently Posted Jobs

Explore more opportunities at MUFG

See more jobs