Software Development Engineer in Test Contractor

SDET Contractor

We are looking for an experienced Software Development Engineer in Test (SDET) to help ensure the quality, reliability, and resilience of Agentic AI systems and backend services. This role will design and implement automated testing strategies across AI/LLM-driven applications, APIs, microservices, and distributed systems.

The ideal candidate brings strong Quality Engineering and test automation fundamentals, with experience testing Generative AI, LLM-based, or Agentic AI systems. They should be comfortable testing complex systems where behavior may be non-deterministic and traditional pass/fail testing alone may not be sufficient.

Responsibilities

• Develop and execute comprehensive QE strategies and test plans for Agentic AI systems, backend services, APIs, and service integrations.
• Design and maintain scalable automated test frameworks across API, functional, integration, contract, regression, and end-to-end testing.
• Develop testing approaches for non-deterministic AI behavior and establish meaningful evaluation criteria.
• Identify quality risks early and apply risk-based, shift-left testing practices throughout the development lifecycle.
• Integrate automated testing into CI/CD pipelines and establish effective quality gates.
• Partner with Software Engineers, Product Managers, and other QE teams to improve testability, observability, and overall product quality.
• Analyze defects and production issues to identify root causes, testing gaps, and opportunities for continuous QE improvement.
• Establish quality metrics and provide visibility into test coverage, reliability, defects, and release readiness.

What You'll Do

• Test Agentic AI workflows, including multi-step execution, tool selection and invocation, task completion, and failure handling.
• Validate AI-generated responses using appropriate measures for accuracy, relevance, groundedness, consistency, and adherence to expected behavior.
• Test AI guardrails, unexpected inputs, and failure scenarios, including incorrect tool usage and prompt-based attacks.
• Validate interactions between AI agents, APIs, tools, data sources, and downstream services.
• Develop reusable automation, test data, and evaluation scenarios for repeatable AI and services testing.
• Validate API contracts, schemas, authentication, business rules, error handling, and service integrations.
• Investigate failures across distributed systems and distinguish application, integration, environmental, automation, and AI-related issues.
• Participate in defect triage, root-cause analysis, regression testing, and release-readiness activities.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience.
• 4+ years of experience in Software Development Engineering in Test, Quality Engineering, Software Engineering, or a related discipline.
• Strong experience testing REST APIs, backend services, microservices, or distributed systems.
• Hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language.
• Experience with functional, integration, regression, contract, and end-to-end testing.
• Experience integrating automated tests into CI/CD pipelines.
• Strong analytical, debugging, and root-cause-analysis skills.
• Experience working in an Agile development environment and collaborating across engineering teams.

Required Skills

• Strong understanding of software testing methodologies and Quality Engineering practices.
• Strong API and services testing experience, including REST, JSON, schemas, authentication, and error handling.
• Experience designing and maintaining automated test frameworks.
• Experience with Git and CI/CD development workflows.
• Ability to develop risk-based test strategies and identify appropriate testing layers.
• Experience testing distributed systems and service-to-service integrations.
• Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems.
• Understanding of testing non-deterministic AI behavior and evaluating outputs beyond exact-match assertions.

Preferred Skills (Good to have)

• Experience testing Agentic AI workflows, including planning, multi-step execution, tool calling, memory, or state management.
• Experience with AI evaluation techniques such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection.
• Experience with AI/LLM evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar technologies.
• Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools.
• Experience with contract testing tools such as Pact and API specifications such as OpenAPI/Swagger.
• Experience with event-driven architectures, Kafka, Docker, Kubernetes, or cloud platforms such as AWS.
• Experience with performance, scalability, reliability, or resiliency testing.
• Experience establishing QE metrics, quality gates, or cross-team testing standards.

Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to race, color, religion, gender, pregnancy, national origin, ancestry, citizenship, age, legally protected physical or mental disability, covered veteran status, status in the U.S. uniformed services, sexual orientation, marital status, genetic information or membership in any other legally protected category.

We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond.

We sincerely appreciate the time and effort you spent in contacting us and we thank you for your interest in PlayStation.

The estimated base pay range for this role is listed below, this is an hourly rate. 58 - 72 USD