Test Harness Engineer

We're looking for a Test Harness Engineer to join a client product team and design and build the testing systems that protect the quality of a growing technology platform. This role goes beyond writing test scripts: you will build AI-powered test harnesses where agents help plan, execute, validate, and analyze testing activities.

You'll combine strong software engineering with automation, exploratory testing, and a product-oriented mindset. You'll collaborate closely with engineering and product teams in a fast-paced environment where quality is a shared responsibility and continuous improvement is highly valued. You will make architecture and tooling decisions and take ownership of quality from design to release.

Responsibilities:

• Design and extend AI-powered test harnesses where agents plan, execute, validate, and analyze test scenarios.
• Integrate LLMs (OpenAI, Claude, Gemini, or similar) with tools, APIs, and automation frameworks, with evaluation and guardrails to prevent false positives.
• Build and maintain automated UI, API, and integration test suites that run reliably in CI/CD pipelines, keeping them fast and maintainable.
• Turn Jira acceptance criteria and Figma designs into test plans, and run exploratory, regression, and functional testing across QA and UAT.
• Identify and communicate defects with clear reproduction steps and evidence, and drive triage with developers and product.
• Use AI tools to accelerate test design and failure analysis, critically validating their output.
• Mentor engineers on harness design and automation practices.

Requirements:

• Proven background in software engineering or test automation, including designing QA automation frameworks that teams rely on in production.
• Hands-on experience building AI agents or agent-based workflows, including tool/function calling, context management, and prompt engineering.
• Experience with at least one agent orchestration framework (LangGraph, LangChain, CrewAI, or similar).
• Ability to implement evaluation, validation, and guardrail mechanisms for AI-driven test execution.
• Strong JavaScript/TypeScript skills, with working knowledge of Python or Java.
• Hands-on experience with UI and API test automation tools (e.g., Cypress, Playwright, REST Assured, Postman/Newman, Karate).
• Experience testing REST APIs and asynchronous, event-driven systems (message queues, pub/sub, or similar).
• Experience working with SQL or NoSQL databases for test data setup and validation.
• Experience integrating automated tests into CI/CD pipelines.
• Fluent English.

Nice to have:

• Experience with MCP (Model Context Protocol) or other tool-integration standards for AI agents.
• Experience with performance testing tools such as k6 or Gatling.
• Experience with accessibility testing.
• Experience testing feature-flagged releases and gradual rollouts.
• Experience working with complex SaaS or consumer-facing products.

Soft Skills:

• Senior-level autonomy: you make sound decisions with little oversight and drive solutions end to end.
• Strong analytical skills with a risk-based approach, and clear communication with developers and product.
• Pragmatic about what to automate versus what to test exploratorily.
• Dedication to continuous learning and development, for yourself and for others.