Senior Quality Assurance Engineer - Manual & Automation

We are looking for a Senior Quality Engineer to own quality end-to-end across one of our core product teams and continuously raise the quality bar of the products you support. This is a hybrid role: you are equally comfortable designing manual and exploratory test scenarios as you are writing and maintaining automated suites in CI. You think about quality from the moment a requirement is drafted - not after the code is merged.

You will work shoulder-to-shoulder with engineers, product managers, designers, and DevOps to shape what "done" means: defining acceptance criteria, designing test coverage that reflects real product risk, and acting as the final checkpoint before anything reaches production.

You bring an AI-first mindset to the craft - using AI tools to accelerate test design, automation, triage, and reporting, with the judgment to know when to trust the output and when not to. You also own quality for LLM-powered features (assistants, RAG, agents, etc.) where “pass/fail” is a judgment you have to operationalize through evals: curated datasets, graded criteria, baselines, and regression thresholds.

This is not a role for someone who waits for tickets to be marked "ready for QA". We need a quality engineer who challenges unclear requirements, asks the uncomfortable questions early, and treats production confidence as a deliverable — measured in fewer surprises in production, faster releases, and a stronger quality culture across the teams you work with

Required Qualifications

• Quality Engineering / QA experience in modern software products, with demonstrable depth in both manual and automation (not one bolted onto the other).

• Experience designing test cases, test plans, and test strategies for non-trivial systems — covering happy paths, edge cases, negative scenarios, integrations, and non-functional risks.

• Experience building and maintaining automated test suites in production-grade CI environments, with hands-on framework decisions and ownership of flakiness, reliability, and reporting.

• Strong experience with at least one modern UI automation tool (Playwright, Cypress, Selenium, Appium, or equivalent) and at least one API testing approach (Postman, REST clients, or automated API frameworks such as RestAssured, Supertest, Pytest+requests, or similar).

• Working knowledge of test case management tools (Qase, TestRail, Xray, Zephyr, or similar) and bug tracking tools (Jira, Linear, Asana, or similar).

• CI/CD fluency: comfortable wiring automated tests into pipelines using GitHub Actions, GitLab CI, Jenkins, or similar; understands quality gates, parallelization, and reporting.

• Scripting/programming proficiency in at least one of JavaScript/TypeScript, Python, Java, or equivalent—sufficient to write, debug, and refactor automation code without dependency on engineers.

• Solid understanding of software testing fundamentals: risk-based testing, equivalence partitioning, boundary analysis, exploratory testing techniques, and regression strategy.

• Strong release readiness mindset: ability to define quality gates, run release validation, and make confident go/no-go calls backed by evidence.

• Excellent written and spoken English; comfortable working in fully remote, distributed teams.

• Senior judgment: ability to operate autonomously, challenge unclear requirements, prioritize by risk, and own outcomes without micromanagement.

• AI-first working style: hands-on, daily use of AI tools (Claude, Cursor, Copilot, or similar) to accelerate test design, automation, debugging, and documentation — with the judgment to know when to trust the output and when not to.

Nice to have: 

• Performance and load testing experience (k6, Locust, JMeter, or similar) and the ability to design realistic load scenarios.
• Security testing awareness: OWASP Top 10, injection patterns, authentication/authorization edge cases.
• SQL for data validation; comfort reading logs and traces to debug failures past the UI layer.
• Experience with an eval or LLM-observability framework.
• Experience buildingLLM-as-judge graders and validating judge/human agreement.

What this person will own and do day-to-day 

Quality Ownership and Test Strategy 
Own end-to-end quality for the products and features in your scope.
Define test strategies based on product risk, release goals, and business impact — not generic checklists.
Influence quality upstream by reviewing requirements and designs to surface ambiguity, edge cases, and testability issues early.
Identify non-functional risks (performance, security, accessibility, data integrity) and ensure they're addressed or escalated.

Manual QA and Exploratory Testing
Run structured exploratory sessions guided by risk and user personas.
Cover what automation can't yet: complex flows, visual checks, new feature shakedowns, third-party integrations.
Document findings clearly: reproducible steps, environment, expected vs. actual, severity, impact.
Act as a user advocate during pre-release validation.

Test Case Design and Documentation
Design test cases covering happy paths, edge cases, negative scenarios, role/state transitions, integrations, and relevant non-functional risks.
Maintain traceability to requirements, acceptance criteria, and product risks.
Build and maintain smoke tests, regression suites, and release checklists as living assets.

LLM and AI Product Quality (Evals)
Own the evaluation strategy for LLM-powered features: what is measured, against which dataset, with what pass criteria, and what blocks a release.
Build and maintain eval datasets: golden sets, adversarial and edge-case sets, regression sets grown from real production failures.
Test for groundedness and hallucination: are answers supported by retrieved context, are citations real and relevant, does the system abstain when it should.
Test agents and tool use: correct tool selection, argument validity, recovery from tool failure, loop and cost runaway, task completion end to end.
Cover safety and misuse: prompt injection (direct and via retrieved content), jailbreaks, PII and data leakage, permission bleed between users or tenants, unsafe or out-of-policy output.
Own non-functional quality for AI features: latency (including streaming/first-token), cost per request, token usage, and rate-limit and timeout behavior.
Define the human-in-the-loop process: who reviews ambiguous outputs, how disagreements are resolved, how that feedback flows back into the eval set.

Automation QA
Design, implement, and maintain automated suites across UI, API, and integration layers.
Decide what to automate based on stability, ROI, and risk — and what not to.
Keep flakiness under control; contribute to the framework itself (utilities, fixtures, test data, reporting).

Defect Management and Quality Reporting
Triage, prioritize, and track defects to resolution with clean hygiene (title, severity, repro, environment, links).
Produce concise release quality reports: what was tested, what wasn't, known risks, recommendations.
Surface trends and turn them into actionable improvements.

Release Readiness and Quality Gates
Own release validation: smoke, regression, environment checks, sign-off criteria.
Define and enforce quality gates for staging, UAT, and production.
Coordinate with DevOps on hotfixes, rollback, and post-release monitoring.
Make confident go/no-go calls backed by evidence.