Conversational AI Tester-Manager

Job Title: Senior Associate, Data Science - Conversational AI Quality Assurance 

 Position: Manager

Industry: Digital Contact Solutions (DCS), Telecommunications-Media-Technology (TMT), Health Industries (Payers, Providers, Life Sciences), CPG/Retail, Financial Services, Industrial Products and Services (IPS), Deals 

Domain: Data Science – Conversational AI

About Acceleration Center Bengaluru 

At PwC, we connect people with diverse backgrounds and skill sets to solve important problems together and lead with purpose—for our clients, our communities and for the world at large. It is no surprise therefore that 429 of 500 Fortune global companies engage with PwC. Acceleration Centers (ACs) are PwC’s diverse, global talent hubs focused on enabling growth for the organization and value creation for our clients. The PwC Advisory Acceleration Center in Bengaluru is part of our Advisory business in the US. The team is focused on developing a broader portfolio with solutions for Risk Consulting, Management Consulting, Technology Consulting, Strategy Consulting, Forensics, and vertical specific solutions. PwC's high-performance culture is based on passion for excellence with a focus on diversity and inclusion. You will collaborate with and receive support from a network of people to achieve your goals. We will also provide you with global leadership development frameworks and the latest in digital technologies to learn and excel in your career. At the core of our firm's philosophy is a simple construct: We care for our people. Globally PwC is ranked the 3rd most attractive employer according to Universum. Our commitment to Responsible Business Leadership, Diversity & Inclusion, work-life flexibility, career coaching and learning & development makes our firm one of the best places to work, learn and excel  

We are looking for an experienced senior associate with a strong understanding of testing methodologies and a Conversational AI background (with overall professional experience of 4+ years) to work in our Analytics Consulting practice in Mumbai, Bengaluru. 

Senior Associate’s will work as an integral part of business analytics teams in India alongside clients and consultants in the U.S., leading teams for high-end analytics consulting engagements and providing business recommendations to project teams. 

Education: Advanced Degree in a quantitative discipline such as Computer Science, Engineering, Econometrics, Statistics, or Information Sciences such as business analytics or informatics  

Required Skills: Successful candidates will have demonstrated the following skills and characteristics: 

Must Have:

• Familiarity with the Conversational AI domain, conversational implementation and testing, customer experience metrics, and industry-specific challenges.

• Expertise in various NLU based testing methodologies, including conversational flow end to end testing, NLP and API testing, functional testing, regression testing, integration testing, performance testing, and user experience testing on Conversational AI systems

• Understanding of conversational (chats, emails, and calls) data to test and train Conversational AI systems

• Understanding the critical factors in Performance, Load, and Stress Testing for Platform Evaluation and Integration Assessment

• Understanding of cloud platforms (e.g., AWS, Azure, Google Cloud, Omilia Cloud Platform, Kore.ai, OneReach.ai, NICE, Salesforce, etc.) and their services for building and testing Conversational AI solutions for clients

• NLP and NLU Verticals Expertise: Text to Speech (TTS), Speech to Text (STT), SSML modeling, Intent Analytics, Proactive Outreach Orchestration, OmniChannel AI & IVR (incl.Testing), Intelligent Agent Assist, Contact Center as a Service (CCaaS), Modern Data for Conversational AI and Generative AI

• Familiarity with sentiment analysis, topic modeling, and text classification techniques 

• Experience with test automation tools and frameworks, such as Cyara, Botium, Postman, Qmetry, Qtest, Selenium, IBM Watson Assistant Toolkits, Cognigy Test Case Builder or similar, to streamline and automate the testing process for chatbots.

• Strong problem-solving skills with the ability to identify, analyze, and troubleshoot issues encountered during testing

• Strong understanding of diverse testing methods across different devices to verify the Conversational AI system

• Proficiency in programming languages such as JavaScript 

• Excellent communication skills 

Nice to Have:

• Proficiency in programming languages such as Python/Pyspark/R/SQL

• Proficiency in any of the data visualization tools like Tableau, Power BI, Quicksight and others 

• Experience in survey analytics, organizational functions such as pricing, sales, marketing, operations, customer insights, etc.

Roles and Responsibilities:

• Develop, evaluate, and execute on project test plans, test books to include details about unit testing, end to end testing and chatbot conversation transcripts under the guidance of project manager 

• Analyze chatbot performance metrics and generate insightful reports for client presentations

• Collaborate and address identified issues with development and coach the junior team members 

• Devise complex testing scenarios for quality delivery of products

• Identify and address potential vulnerabilities in Conversational AI system, ensuring data privacy and protection.

• Ability to test and deploy production ready code. 

• Build storylines and make presentations for the client team and/or PwC project leadership team. 

• Contribute to the knowledge and firm building activities 

What we offer:

• At PwC, we care for the mental & physical well-being of our people and respect the flexibility that is needed during challenging times. Keeping these factors in mind, we have introduced policies around Work-from-Home. Anywhere and flexible working hours. 

• Mid-year appraisal cycle to reward performance on time 

• Opportunities to solve problems which matter to our clients and have a strong, long-lasting impacts 

• Continuous learning and upskilling opportunities nurturing you to become a better leader and consultant of tomorrow. We provide you with access to Massive Online Open Courses (MOOC) at no cost. 

• A world class leadership to guide you on gaining expertise across various functions and a top notch 

• diverse peer group to support you throughout the journey 

• Interaction with the senior client leadership and a chance to meet them in person by visiting client locations (as needed) and/or permanent relocation