Side

Localization, QA, audio, AI for games

AI Quality Engineer

Manual & Functional QAConfirmed live in the last 24 hours

ContractHyderabad, Telangana, IndiaNo salary listed

About the Job

Requirements

Strong understanding of artificial intelligence, machine learning, and generative artificial intelligence testing.
Hands-on experience testing large language model and retrieval-augmented generation applications.
Knowledge of large language model evaluation, hallucination detection, relevance, and response quality.
Understanding of AI agents and recommendation systems.
Strong analytical and problem-solving skills.

Responsibilities

Test and validate AI-generated insights, recommendations, and decision-making workflows.
Evaluate large language model and retrieval-augmented generation systems for accuracy, relevance, consistency, factuality, and hallucinations.
Validate retrieval quality, context relevance, grounding, and response quality in retrieval-augmented generation systems.
Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
Perform regression testing for models, prompts, retrieval-augmented generation configurations, and AI workflows.
Collaborate with AI and machine learning engineers to identify issues and improve AI system quality.

Desired Qualifications

Experience with PyTest and automated testing frameworks.
Experience building automated AI evaluation and regression frameworks.
Familiarity with RAGAS, DeepEval, LangSmith, or equivalent tools.
Experience with continuous integration and continuous delivery-based test automation, performance testing, or AI guardrails.
Familiarity with cloud platforms such as AWS, Azure, or Google Cloud Platform, and observability tools.

AI Quality Engineer (Contract)

Side · Hyderabad, India · Contract · Posted 2026-08-28

Workplace: on_site

Department: Quality Assurance

Description

**AI Quality Engineer

**Location:- Hyderabad
Work Mode:- WFO (5 days a week)
Role Type:- Contractual (3 months and extension would depend on project requirement and performance)

Key Responsibilities

Test and validate AI-generated insights, recommendations, and decision-making workflows.
Evaluate LLM and RAG systems for accuracy, relevance, consistency, factuality, and hallucinations.
Validate retrieval quality, context relevance, grounding, and response quality in RAG systems.
Test AI agents and autonomous workflows across functional, negative, and edge-case scenarios.
Define AI evaluation criteria, test datasets, quality metrics, and validation processes.
Perform regression testing for models, prompts, RAG configurations, and AI workflows.
Collaborate with AI/ML engineers to identify issues and improve AI system quality.

Requirements

Required Skills

Strong understanding of AI/ML and Generative AI testing.
Hands-on experience testing LLM and RAG-based applications.
Knowledge of LLM evaluation, hallucination detection, relevance, and response quality.
Understanding of AI agents and recommendation systems.
Strong analytical and problem-solving skills.

Good to Have

Experience with PyTest and automated testing frameworks.
Experience building automated AI evaluation and regression frameworks.
Familiarity with tools such as RAGAS, DeepEval, LangSmith, or equivalent.
Experience with CI/CD-based test automation, performance testing, or AI guardrails.
Familiarity with cloud platforms (AWS/Azure/GCP) and observability tools.

Success Metrics

High accuracy, relevance, and reliability of AI outputs.
Strong evaluation coverage across critical AI workflows.
Early detection and reduction of hallucinations and AI regressions.
Reduced production AI quality issues.
Increased confidence and trust in AI-generated insights and recommendations.

Apply

Apply at Side

About the Company

Side

Localization, QA, audio, AI for games

PTW provides specialized services for gaming and tech, including co-development, live operations, localization, quality assurance, audio production via its SIDE subsidiary, and AI dataset creation for virtual assistants, TTS, and ASR. It supports developers and publishers across the project lifecycle: localization adapts content for languages and cultures; QA tests games across platforms; live operations manages ongoing services; SIDE handles casting and post-production; and AI data services collect and evaluate data to train AI systems. PTW differentiates itself by offering end-to-end, integrated services under one umbrella, backed by SIDE’s 25 years of audio expertise and a global gaming-focused footprint. Its goal is to help clients extend game lifecycles, enter new markets, and deliver high-quality user experiences through dependable, long-term partnerships.

Company size: 1,001-5,000
Company stage: Not listed
Total funding: Not listed
Headquarters: Larkspur, CA
Founded: 1994

Recently Posted Jobs

Explore more opportunities at Side

See more jobs