Lingaro

Turns data into AI-driven business insights

Lead QA Engineer - AI Voice

Manual & Functional QAConfirmed live in the last 24 hours

Full-timeRemote in PolandNo salary listed

About the Job

Requirements

At least 6 years of experience in software quality assurance.
At least 2 years of experience testing voice agents or conversational AI in live delivery.
Has led end-to-end testing and user acceptance testing, and has worked as the lone or lead quality assurance engineer with limited guidance.
Experience in international or distributed teams.
Experience testing voice AI, including speech recognition and speech output, latency, interruptions, accents, and noise.
Experience with voice quality metrics such as word error rate, intent accuracy, and task completion.
Practical understanding of how large language models work and fail; deep large language model engineering is not needed.
Experience with test automation in Python or JavaScript/TypeScript, continuous integration/continuous delivery, and application programming interface testing, such as Postman.
Experience using cloud speech services such as Azure, Google, or AWS to generate test calls.
Practical experience with Jira or Azure DevOps and a test management tool.
Practical experience troubleshooting using logs, transcripts, and call recordings.
English proficiency at least at B2 level.

Responsibilities

Design the voice AI test strategy and process from speech input through speech recognition, the large language model, tools and integrations, to speech output.
Define the test automation approach and how it scales, including multilingual coverage of around 100 languages.
Test predictable behaviour, including intents, routing, and integrations, as well as non-deterministic AI behaviour.
Evaluate prompts and large language model responses using test datasets and scoring criteria.
Lead user acceptance testing, including planning, scenarios, test data, coordination of business testers, and a sign-off recommendation.
Gather business acceptance criteria and turn them into measurable targets.
Own defect and risk management, and provide release go/no-go recommendations.
Replan and communicate options when development is delayed.
Challenge stakeholders on readiness and quality, and mentor the regular engineer.

Desired Qualifications

Experience with voice testing platforms such as Hamming, Cekura, Coval, or Botium.
Experience with large language model evaluation tools such as Promptfoo, DeepEval, or LangSmith.
Experience with telephony basics such as SIP, Twilio, or Azure Communication Services.
ISTQB certification.
Experience in responsible AI testing, including personally identifiable information, bias, and safety.
Experience validating customer journeys across multiple markets.

Growth through diversity, equity, and inclusion. As an ethical business, we do what is right — including ensuring equal opportunities and fostering a safe, respectful workplace for each of us. We believe diversity fuels both personal and business growth. We're committed to building an inclusive community where all our people thrive regardless of their backgrounds, identities, or other personal characteristics.

What you'll be doing

Design the voice AI test strategy and process, from speech input through speech recognition (ASR), the LLM, tools and integrations, to speech output.
Define the automation approach and how it scales, including multilingual coverage of around 100 languages.
Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.
Evaluate prompts and LLM responses, using test datasets and scoring criteria.
Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.
Gather business acceptance criteria and turn them into measurable targets.
Own defect and risk management, and give release go/no-go recommendations.
Replan and communicate options when development is delayed.
Challenge stakeholders on readiness and quality, and mentor the Regular engineer.

What we're looking for

At least 6 years of experience in software QA.
At least 2 years of experience in testing voice agents or conversational AI in live delivery.
Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.
Experience in international or distributed teams.
Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.
Voice quality metrics such as word error rate, intent accuracy and task completion.
Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.
Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).
Use of cloud speech services (Azure, Google, AWS or similar) to generate test calls.
Practical experience with Jira or Azure DevOps, and a test management tool.
Practical experience with troubleshooting from logs, transcripts and call recordings.
Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus.
ISTQB certification is a plus.
Experience in Responsible-AI testing (PII, bias, safety) is a plus.
Experience validating customer journeys across multiple markets is a plus.
Structured, recommendation-first communication with technical and business stakeholders.
English at least B2.

We offer

Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
“Office as an option” model. You can choose to work remotely or in the office.
Workation.Enjoy working from inspiring locations in line with our workation policy.
Great Place to Work® certified employer.
Flexibility regarding working hours and your preferred form of contract.
Comprehensive online onboarding program with a “Buddy” from day 1.
Cooperation with top-tier engineers and experts.
Unlimited access to the Udemy learning platform from day 1.
Certificate training programs. Lingarians earn 500+ technology certificates yearly.
Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
Grow as we grow as a company. 76% of our managers are internal promotions.
A diverse, inclusive, and values-driven community.
Autonomy to choose the way you work. We trust your ideas.
Create our community together. Refer your friends to receive bonuses.
Activities to support your well-being and health.
Plenty of opportunities to donate to charities and support the environment.
Modern office equipment. Purchased for you or available to borrow, depending on your location.

About the Company

Lingaro

Turns data into AI-driven business insights

Lingaro helps large organizations turn data into business value. It provides end-to-end data and AI services, from strategy and advisory to building scalable data platforms and AI solutions. Its team turns complex data into actionable insights, advanced analytics, and AI-driven automation, enabling faster decisions and greater efficiency. Lingaro distinguishes itself with global reach rooted in Europe, strong engineering expertise, agility, and deep industry knowledge, serving 75+ brands in 30+ countries, including highly regulated sectors. As a long-term transformation partner, the company aims to make organizations data-ready, AI-ready, and future-ready, delivering measurable business impact across teams and processes.

Company size: 1,001-5,000
Company stage: Not listed
Total funding: Not listed
Headquarters: Warsaw, Poland
Founded: 2008

Recently Posted Jobs

Explore more opportunities at Lingaro

See more jobs