N

Netskope

Data-centric cloud security for enterprises

AI Agent Quality Engineer

Test Automation EngineeringConfirmed live in the last 24 hours
Full-timeTaguig, Metro Manila, PhilippinesNo salary listedIn-person

About the Job

Requirements

  • At least 4 years of experience in software quality, security testing, or a related discipline, including 1–2 years specifically evaluating or red-teaming large language model-based systems.
  • Strong Python skills and the ability to write production-quality code.
  • Working knowledge of REST APIs and webhook/event-driven patterns to build test harnesses that call agent tools directly and validate inputs and outputs.
  • Experience building automated test infrastructure and integrating it into continuous integration and continuous delivery pipelines.
  • Hands-on familiarity with at least one adversarial testing or large language model evaluation tool, such as DeepTeam, Garak, PyRIT, or Promptfoo, and understanding of how these map to standards such as the OWASP Large Language Model Top 10 or NIST's AI risk framework.
  • Practical understanding of prompt injection, jailbreaking, and sycophancy failure modes, including the ability to design test cases for them.
  • Willingness to block a release when an agent does not meet its evaluation bar, even under schedule pressure.

Responsibilities

  • Build and grow an automated evaluation suite that every agent runs before production approval, designed to run unattended and scale across the agent fleet.
  • Design and automate adversarial test scenarios, including prompt injection attempts, sycophancy checks, and multi-attempt attacks, and run them on relevant changes.
  • Test each new agent by attempting to extract unauthorized data, cause action outside registered tool boundaries, or make it treat a synthetic test probe as real; establish a repeatable, scriptable process as the fleet grows.
  • Partner with the Data Steward on data sensitivity classification for systems agents use so test scenarios reflect actual risk.
  • Define pass criteria for each agent capability and encode them into automated thresholds wherever possible.
  • Maintain the platform's guardrail and negative-test catalog, covering fail-closed and fail-graceful behavior, and add cases for newly discovered failure modes.
  • Produce clear, audit-ready evidence of each agent's evaluation results automatically as part of the pipeline.
  • Track drift over time across the full agent fleet.

Desired Qualifications

  • Understanding of data sensitivity and compliance classification sufficient to design risk-relevant tests; the Data Steward owns the classification system.
  • Experience in a regulated environment where evaluation results were subject to external audit.
  • Familiarity with multi-attempt or persistent-attack testing methodology, beyond single-shot adversarial prompts.
  • Exposure to how agents are built, including prompting, tool schemas, and orchestration.

About the Company

Netskope

Data-centric cloud security for enterprises

Netskope provides cloud security solutions for large enterprises, helping them secure data and protect against threats in cloud environments. It offers a cloud security platform with real-time data and threat protection, secure access service edge (SASE) capabilities, and advanced threat detection and response; all delivered through a subscription model. How it works: the platform continuously monitors cloud services and data traffic, enforcing security policies directly where data lives, and providing real-time protection and threat response across the cloud, network, and apps. How it differs from competitors: it emphasizes a data-centric approach—protecting the data itself in cloud environments—along with cloud-native, real-time protection and a comprehensive SASE bundle, aimed at large organizations undergoing digital transformation. Its goal is to redefine cloud, network, and data security for enterprises by enabling secure and seamless digital transformations.

Company size
1,001-5,000
Company stage
IPO
Headquarters
Santa Clara, CA
Founded
2012

Recently Posted Jobs

Explore more opportunities at Netskope

See more jobs