W

Westinghouse Electric Company

Nuclear reactor technology and services provider

Manager AI Quality - Security & Assurance

Test Automation EngineeringUpdated on Nov 14
Full-timeCranberry Township, PA, USA$130.4k - $163k /yr

About the Job

Requirements

  • A bachelor's degree in Computer Science, Software Engineering, Data Science, Engineering, or a related discipline is required.
  • At least 8 years of experience in software quality engineering, test automation, software assurance, or application security, including at least 2 years of hands-on experience testing or evaluating machine learning, generative AI, or agent-based systems.
  • At least 3 years of experience leading quality, testing, or assurance teams, including hiring and growing teams.
  • Knowledge of large language model and retrieval-augmented generation evaluation methods and metrics, hallucination and grounding analysis, agent evaluation, evaluation datasets, and large language model-as-a-judge techniques.
  • Experience with test strategy, functional, regression, and performance testing, and test automation integrated into continuous integration/continuous delivery pipelines, such as Azure DevOps and Python-based frameworks.
  • Knowledge of application security, secure software development life cycle, and AI-specific threats such as prompt injection, data leakage, and insecure tool use; experience with security scanning tools such as AppScan and red teaming.
  • Ability to define proportionate evidence requirements and make and defend independent readiness recommendations.
  • Documentation and audit-ready evidence practices; knowledge of software quality requirements in regulated environments.
  • Ability to work across Cybersecurity, AI Governance, and business owners and constructively challenge development teams.

Responsibilities

  • Define and maintain an assurance framework covering risk-based classification by intended use, assurance plans, minimum evidence requirements, Definition of Done, readiness reviews, Technology Brick integration into solutions, and exception management.
  • Integrate the assurance framework into the common AI software development life cycle and apply Westinghouse AI policies and software quality requirements without redefining them.
  • Establish methods and datasets to evaluate generative AI responses and agent behavior, including grounding, robustness, tool use, human oversight, and regression; set acceptance thresholds and automate evaluations after model, prompt, or data changes.
  • Define the test strategy for AI applications and Technology Bricks and provide test frameworks and practices integrated into continuous integration/continuous delivery pipelines for development teams to use.
  • Coordinate application security assessments and lead risk-based red teaming of generative AI systems and agents with Corporate Cybersecurity; track findings until remediation or formal acceptance.
  • Review testing, evaluation, and security evidence before release and issue an independent readiness recommendation; escalate unresolved findings and residual risks to the applicable owner for formal acceptance.
  • Document reports before turnover and integration of Technology Bricks into existing or legacy systems in accordance with Westinghouse policy.
  • Act as the primary Data & AI counterpart to IT Quality Assurance and agree on shared testing services, environments, tools, and standards to avoid duplication.
  • Engage during early-stage planning to include assurance effort and acceptance criteria in estimates for new developments and commercial projects.
  • Report quality and readiness metrics across the portfolio and improve methods, automation, and standards.
  • Lead a small primary team in AI evaluation, test automation, and security assurance, and scale it as demand grows through hiring, contractors, and specialist third parties.
  • Oversee goal setting, performance management, recognition, and succession, and develop team members through coaching, mentoring, and upskilling in AI evaluation and AI security.

Desired Qualifications

  • Familiarity with frameworks such as the NIST AI Risk Management Framework is helpful.

About the Company

Westinghouse Electric Company

Nuclear reactor technology and services provider

Westinghouse Electric Company LLC provides nuclear power technology and services to utilities, governments, and private sector clients worldwide. It designs and sells nuclear reactors and fuel, and offers maintenance, training, compliance, and consulting services. Its AP1000 PWR uses passive safety features, and it is developing Small Modular Reactors (SMRs) and Microreactors for flexible deployment. The company's goal is to enable safe, reliable, low-emission energy by delivering proven reactor technologies and a broad support ecosystem globally.

Company size
10,001+
Company stage
Grant
Total funding
$39.5M
Headquarters
Cranberry, West Virginia
Founded
1886

Recently Posted Jobs

Explore more opportunities at Westinghouse Electric Company

See more jobs