Q

Qualysoft

Builds business software and manages IT projects

Java QA Automation Engineer

Test Automation EngineeringConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • At least 3 years of QA automation experience for production systems.
  • Strong Java backend testing skills.
  • Strong frontend testing skills.
  • Experience building and maintaining Playwright, Cypress, or Selenium end-to-end test suites.
  • API testing experience, including REST, schema validation, and authentication and error scenarios.
  • Experience with SQL/PostgreSQL testing and test data management.
  • CI/CD pipeline experience with GitHub Actions, Azure Pipelines, or Jenkins.
  • Practical understanding of testing non-deterministic LLM outputs.

Responsibilities

  • Build API and integration test coverage for Java/Spring routers and database workflows.
  • Design and implement Playwright end-to-end tests for critical user journeys.
  • Create stable mocking and fixture strategies using MSW, test fixtures, and seeded test data.
  • Set up continuous integration quality gates for linting, unit, integration, and end-to-end tests, coverage, and reporting.
  • Expand coverage to untested user interface and workflow areas and prevent regressions.
  • Build an LLM evaluation discipline with golden datasets and measurable quality gates.
  • Track model output reliability, detect drift, and gate releases on quality and stability thresholds.
  • Improve test maintainability, remove flakiness, and optimize runtime.
  • Use LangChain4j for JVM-native LLM application support, including agents, retrieval-augmented generation, and guardrails.
  • Use the Langfuse Java SDK and OpenTelemetry for trace capture, prompt and version tracking, and experiment comparison.
  • Use the OpenAI Java SDK for structured outputs, function calling, webhook verification, and response validation.
  • Use Promptfoo in continuous integration for language-agnostic regression and red-team checks around the Java stack as needed.
  • Use Java schema validation and guardrail patterns with Jackson, Swagger annotations, and LangChain4j guardrails.
  • Use OpenTelemetry-based traces and Langfuse datasets as the reproducible source of truth for release gates.

Desired Qualifications

  • Familiarity with Java-compatible LLM evaluation and regression tooling such as Promptfoo and Langfuse, plus trace-based observability and output validation via API/OpenTelemetry integrations.
  • Performance and benchmark testing experience.
  • WebSocket testing experience.
  • Security and dependency scanning awareness.
  • Experience in legal-tech, fintech, or other regulated domains.

About the Company

Qualysoft

Builds business software and manages IT projects

Qualysoft is an international IT consulting and software engineering company serving enterprises and public-sector organisations. Its work spans business applications, Microsoft cloud systems, custom software, data and analytics, artificial intelligence, testing and digital tax administration. Clients engage the company for individual specialists, dedicated engineering teams or managed projects in which Qualysoft takes responsibility from requirements and design through delivery and maintenance. The group operates through European locations and an Indonesian centre of excellence, supporting projects across several industries. Founded in 1999, Qualysoft is headquartered in Vienna, Austria.

Company size
501-1,000
Company stage
Not listed
Total funding
Not listed
Headquarters
Vienna, Austria
Founded
1999

Recently Posted Jobs

Explore more opportunities at Qualysoft

See more jobs