Quality Insights Manager

Description

Ring and Blink Customer Support assists millions of customers annually with product and account questions. Associates resolve inquiries ranging from technical support to account management, delivering consistent and exceptional customer experiences.

The Customer Support Quality Assurance (CSQA) team, as part of the broader Neighbor Excellence organization, provides data-driven insights to elevate associate performance across Ring and Blink customer service interactions. CSQA partners across Operations, Learning, Content, and Technology to implement enhancements that drive customer satisfaction.

We're building next-generation quality analytics and AI-driven measurement systems. To support this transformation, we’re seeking a highly technical professional who combines AI/ML knowledge with contact center quality experience to execute analytics and reporting that drive customer satisfaction.

As our Quality Insights Manager, you'll own dashboards, reporting, and analytics that surface trends, themes, and defects in quality performance. You'll leverage AI-enabled tools to accelerate analysis, validate AI model outputs to ensure accuracy, and ensure multilingual parity across evaluations. You'll partner with Data Science and Technology teams to support AI/ML innovation in Quality Monitoring Systems and AI-automated evaluation infrastructure. Your insights enable stakeholders to make data-driven decisions that improve associate quality and customer experience.

Success in this role means stakeholder confidence in data accuracy and reliability, timely delivery of clear insights, consistent AI-driven quality evaluations, and effective cross-functional partnerships that enable stakeholders to execute improvement actions that result in measurable improvement in associate quality results.

Key job responsibilities
Insight Generation & Root Cause Analysis
- Surface trends, themes, and systemic defects to improve associate quality and customer experience
- Translate quality insights into clear, actionable problem statements that enable stakeholders to develop improvement strategies and measurably enhance associate quality and customer satisfaction
- Ensure insights are prioritized based on criteria (risk, scale, and customer impact)
- Maintain clear documentation of insights, assumptions, and confidence levels
Data Accuracy & System Reliability
- Inspect and validate quality data inputs across human and automated evaluations
- Identify and resolve gaps in measurement accuracy, consistency, and reliability
- Ensure multilingual parity in quality evaluation across all supported languages
- Partner with automation and quality teams to improve data integrity
Scalable Analytics & AI Integration
- Own dashboards, reporting views, and analytical frameworks that support proactive quality management
- Use AI tools to accelerate analysis, trend detection, and insight generation
- Support development of predictive models to forecast performance trends and identify emerging risks
- Ensure insight outputs remain clear, explainable, credible, and actionable for non-technical stakeholders
- Support ad-hoc reporting requests for organizational and departmental initiatives, ensuring timely delivery of insights that inform strategic decisions
Leadership Enablement & Risk Management
- Present insights clearly to stakeholders with risk assessment and recommended focus areas
- Escalate appropriately when insights are not acted upon or when data indicates growing customer impact
- Partner with cross-functional teams including Data Science, Product, and Technology to refine evaluation datasets and improve model performance
- Monitor measurement systems and recommend improvements to ensure they evolve with business needs and maintain accuracy at scale
- Monitor reporting mechanism effectiveness and recommend measurement standard improvements based on outcomes, business needs, and stakeholder feedback to maintain relevance and accuracy

A day in the life
You start your day reviewing quality dashboards and data to identify emerging defects, trends, and performance patterns. You pressure-test multilingual evaluation consistency across all supported languages to ensure parity in quality measurement. You support development of predictive models to forecast performance trends and identify emerging risks before they impact customers.

You partner with stakeholders to walk through problem statements and risk areas, ensuring they have clear, actionable insights to drive improvement. When you identify a performance gap, you surface what requires attention, why it matters, and what the customer impact is if left unaddressed - enabling stakeholders to create action plans with clear owners and timelines.

Throughout the day, you work with Data Science and Technology teams to validate AI model outputs, ensuring evaluation accuracy and multilingual consistency. You provide feedback that refines evaluation datasets and improves model performance, supporting measurement systems to evolve with business needs and maintain accuracy at scale. You monitor measurement systems, identify data quality issues, and monitor reporting mechanism effectiveness, recommending measurement standard improvements based on outcomes and stakeholder feedback. You also support ad-hoc reporting requests, delivering timely insights that inform strategic decisions across the organization.

About the team
As part of the Neighbor Excellence Team (NET), CSQA operates within Ring and Blink's centralized support excellence organization. NET brings together specialized peer functions—including Neighbor Experience, Learning Experience, Training Delivery, Content and Engagement, Leadership Development, and CSQA—under one leadership structure to drive coordinated improvements in customer service quality.

What makes NET unique is our closed-loop continuous improvement approach. CSQA's quality insights don't just generate reports—they trigger coordinated action across NET's partner functions. When we identify performance trends or improvement opportunities, those insights flow to teams responsible for training, content, leadership development, and customer experience. They take action, and we measure the impact, creating a continuous cycle of improvement that compounds over time to elevate the customer experience.

Our mission is to evaluate customer interactions through our quality framework, conducting inspections across various channels. We facilitate regular calibration sessions that ensure consistent evaluation standards across all quality analysts, measuring both individual consistency and team-wide alignment. CSQA also maintains quality control through systematic review processes that verify scoring accuracy and identify opportunities for analyst development, enabling stakeholders to understand customer sentiment and drive continuous improvement.

Basic Qualifications

- Experience using data to drive root cause elimination and process improvement
- 3-5+ years in Quality Assurance with senior-level, lead, or consultative responsibilities
- Experience in contact-center quality programs, quality monitoring, or agent evaluation
- Experience using advanced analytics and statistical methods to contribute to customer satisfaction improvements
- Experience executing analytics programs, delivering insights, and communicating results to managers
- Experience with data visualization tools (AWS QuickSight, Tableau, Power BI)
- Experience running metrics reports, analyzing data to identify problems, and using data analysis to generate and explain metrics
- Active use of AI tools for productivity, analytics, and insights generation

Preferred Qualifications

- Background as QA Supervisor with hands-on evaluation experience
- Experience supporting or scaling QA programs in enterprise environments
- Experience with contact-center quality platforms (Amazon Connect, NiCE, Verint, CallMiner)
- Experience with statistical models and data visualization using AWS QuickSight, Tableau, Power BI
- Experience in at least one modern scripting or programming language (Python, Java)
- Experience in test automation and conversational AI/GenAI evaluation (semantic similarity, grounding checks, behavior prediction)
- Experience with GenAI test techniques (prompt evaluation, hallucination detection, golden datasets, guardrails, API testing)
- Experience with database management including ETL and call center metrics (AHT, FCR, CSAT, NPS)
- Experience with technology platforms, data architecture, AI-based quality systems; translating operational needs to technical requirements

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.