C

Cisco

Makes networking gear and security software

AI Automation Engineer - Quality

Test Automation EngineeringConfirmed live in the last 24 hours
Full-timeResearch Triangle Park, Durham, NC, USA

About the Job

Requirements

  • A Bachelor's degree with 4+ years, Master's degree with 3+ years, or PhD with 1+ year of related experience in Computer Science, AI/ML, or a related technical field is required.
  • Experience in test automation and regression framework development, with a strong understanding of functional quality assurance, is required.
  • Experience with Playwright using TypeScript or JavaScript for end-to-end and user-interface automation, and/or Python with Pytest or Robot Framework, is required; familiarity with Selenium is also required.
  • Experience with REST API testing and familiarity with API contract testing are required.
  • Practical knowledge of software development lifecycles, CI/CD pipelines such as GitHub Actions or Jenkins, and automated testing best practices is required.
  • Experience with AI-assisted coding tools such as GitHub Copilot, Cursor, or Claude Code, critical thinking, and Agile/Git workflows is required to ensure high-quality, scalable test automation.

Responsibilities

  • Architect, build, and maintain comprehensive automated regression and end-to-end test suites for the Circuit platform using Playwright with TypeScript or JavaScript, Python with Pytest, and Selenium, replacing manual quality assurance with scalable automation.
  • Design and automate REST and GraphQL API test suites that cover functional correctness, contract validation, error handling, and performance, including validation of AI/model-serving endpoints under concurrent load.
  • Plan and execute load, stress, and soak tests using k6 or JMeter, and track response-time percentiles (p95/p99), throughput, error rates, Time to First Token (TTFT), and inter-token latency.
  • Use AI agents and AI-assisted coding tools such as Cursor, Copilot, and Claude Code to generate and maintain regression suites for new and legacy features; assess generated code for stability and ensure it complies with security standards before merging.
  • Develop testing approaches for AI/large language model features, validating model response quality, behavioral consistency through stability testing, and end-to-end AI agent workflows across reasoning and action layers.
  • Integrate automated tests into CI/CD pipelines such as GitHub Actions or Jenkins for continuous validation; build dashboards and reports using Grafana or Allure; communicate quality metrics, risk assessments, and defect trends to engineering leadership.
  • Lead automation initiatives as a team captain by defining standards, mentoring peers, and collaborating with development, product, and user-experience teams to embed automation-first testing into the development lifecycle.

Desired Qualifications

  • Experience testing AI/ML-powered web applications, including large language model features, chatbots, or AI agent workflows.
  • Hands-on experience with performance and load testing using k6 or JMeter, and AI/large language model performance metrics such as Time to First Token, inter-token latency, and token throughput.
  • Experience in AI Ops or MLOps, or building or fine-tuning custom large language models.
  • Experience with observability and reporting tools such as Splunk, Grafana, and Prometheus.
  • Familiarity with Docker and Kubernetes for test environment management, and Google Cloud Platform.
  • A proven track record of leading technical projects and mentoring junior engineers.
  • Experience building and productizing agentic applications with a focus on evaluation and business acceptance.

About the Company

Cisco

Makes networking gear and security software

Cisco makes the switches and routers that connect offices, data centers, and cloud services, and it also sells cybersecurity tools and Webex video meetings to businesses and governments. It was one of the first companies to sell routers that let different kinds of computer networks exchange data, and in 2000 it briefly became the world's most valuable company. Customers run its network gear, from Wi-Fi access points to AI data center systems, with Cisco software that sets up and secures the network, while its Splunk unit searches computer logs to catch outages and attacks. Cisco earns money from hardware sales, software subscriptions, and support contracts. It was founded in 1984 and is based in San Jose, California.

Company size
10,001+
Company stage
IPO
Headquarters
San Jose, CA
Founded
1984

Recently Posted Jobs

Explore more opportunities at Cisco

See more jobs