Tricentis AgentScore alternative: vendor-neutral AI agent quality scoring (2026). Looking for an AgentScore alternative? What Tricentis AgentScore does, where it fits, and an open, vendor-neutral rubric for scoring AI agents before you ship. If you want an AgentScore alternative, you do not need another black box. You need a written, vendor-neutral rubric (task success, tool-call accuracy, safety, cost and latency, regression) with hard thresholds, scored by open-source tools against a golden set built from your own traffic. That gives you the same ship, block, or review decision, and you can explain every point of it. First, credit where it is due. Tricentis is right about the problem, and AgentScore is a serious attempt to solve it. Here is what it actually is, who should consider it, and how to build an open equivalent if you cannot wait or do not want to. What is Tricentis AgentScore? Tricentis announced AgentScore on 20 August 2026 at its Transform conference in Dallas, alongside two other launches: Aida, which autonomously explores web and Windows desktop apps to find defects, and Release Risk Intelligence, which surfaces release-scoped coverage gaps. The announcement came a month after Tricentis acquired Tabnine. Based on the launch announcement and coverage, AgentScore: * Observes agents in real-world workflows rather than only in controlled test scenarios. * Recommends which metrics matter for a given agent type. * Produces a composite quality score with a recommendation to ship, block, or review. Tricentis frames this as a move from deterministic testing (same input, same output) to probabilistic evaluation, which is the right framing for agents. The ship, block, or review output is also smart product design. QA leads do not want another dashboard. They want a decision. What aiml could not find at the time of writing: public pricing, a general availability date, or detail on how the composite score is calculated. AgentScore came out through Tricentis Labs, the company's early-access incubator, so check tricentis.com for current status before planning around it. Who should consider AgentScore, and who needs an alternative? | Your situation | Likely best fit | | Already standardised on Tricentis (Tosca, qTest) and happy to join early access | Try AgentScore through Tricentis Labs | | Need a release decision on an agent this quarter | Open rubric plus open-source tooling | | Regulator, auditor, or enterprise customer wants to see how the score is computed | Open rubric (transparent by design) | | Agents built on several model vendors, and you want vendor-neutral comparison | Open rubric | | Small team, no enterprise QA platform contract | Open rubric | None of this is a knock on Tricentis. Composite scores from any vendor, including an internal one, raise the same question: what is inside the number? If you need to answer that for a CBUAE review, an EU AI Act file, or a bank's vendor risk team, a rubric you wrote and can show line by line is easier to defend. What does a vendor-neutral agent quality rubric look like? This is the scorecard aiml use as a starting point. The weights are a default, not a law. Tune them to your risk profile, but keep the floors. | Dimension | What you measure | How to measure it | Default weight | Hard floor (blocks on its own) | | Task success | Did the agent reach the correct end state? Golden set with expected outcomes; multiple runs per case | 35% | Below last approved release | | Tool-call accuracy | Right tool, valid arguments, sensible order, no redundant calls | Trace comparison against reference trajectories; schema validation | 25% | Any write action with invalid arguments | | Safety and policy | Prompt injection resistance, data leakage, unconfirmed irreversible actions | Adversarial suite plus policy checks on every trace | 20% | Any critical violation | | Cost and latency | Tokens, tool calls, and wall-clock time per completed task | Trace metadata from your observability tool | 10% | Exceeds agreed budget per task | | Regression | Change versus the previous approved version on the same set | Same suite, pinned judge model, side-by-side diff | 10% | Statistically meaningful drop on tier-1 cases | Turning the score into ship, block, or review. * Ship: composite at or above your target and no floor breached. * Review: composite in a grey band, or a non-critical floor close to breach. A human reads the failing traces before release. * Block: any hard floor breached, regardless of composite. A 92% score with one unconfirmed refund is a block. That floor rule is the most important line on the page. Composite scores average away exactly the failures that hurt customers. Which open-source tools cover each dimension? | Tool | What it covers | Notes | | DeepEval | Task completion and tool correctness metrics, pytest-style CI gates | Apache 2.0; good default for the CI gate | | Ragas | Agent goal accuracy and tool call accuracy against references, plus retrieval metrics | Apache 2.0; reference-based tool scoring is strict about call order by default | | Inspect | Agent and model evaluations, including tool-using agents | MIT; originally released by the UK AI Safety Institute (now AI Security Institute) | | Langfuse | Traces, datasets, scores, and cost data in one place | MIT core; acquired by ClickHouse in January 2026 | | Arize Phoenix | OpenTelemetry-native tracing with eval primitives | Free to self-host | | Promptfoo | Red-team and safety suites, multi-model comparison | MIT, but OpenAI announced its acquisition in March 2026; weigh that if vendor neutrality is the point | A typical stack: Langfuse or Phoenix for traces, DeepEval for the CI gate, Ragas or custom checks for tool-call accuracy, and an adversarial suite for the safety floor. For a deeper look at the frameworks, see its LLM evaluation framework benchmark. How do you build the scorecard in practice? * Write the rubric first, before touching tools. Get product, QA, and risk to agree on the floors. * Build a golden set from real traces, including every past incident as a permanent case. Keep multi-turn sessions intact. * Prefer deterministic checks for tool calls (names, schemas, ordering rules) and reserve LLM judges for what code cannot check. * Calibrate every LLM judge against human labels on a sample of real outputs, then pin the judge model version. * Run each case several times, because one successful run shows the agent can do a task, not that it reliably will. * Publish the verdict with the failing traces attached, so a reviewer can see why it says block. The tooling is a week of work. The golden set and calibration are where teams stall, and where an outside pair of eyes helps most. Its guide to QA-ing an AI agent covers the failure modes your safety cases should target, and the AI QA Scorecard shows how to track these numbers over time. If you are building the harness behind this scorecard, genai.qa has a companion piece on eval harnesses that match production reality. What are the limits of any agent quality score? Whether it comes from Tricentis, another vendor, or your own spreadsheet, an agent score has the same blind spots. Be honest about them in the report. * It only knows the cases you gave it. A golden set written six months ago scores yesterday's users. Refresh it from production traces every release cycle. * LLM judges drift. If the judge model or its prompt changes, last month's 88% and this month's 88% are not the same number. * Composite scores hide outliers. That is why the floors exist, and why the report should list every floor-level failure by name. * Offline is not online. A ship verdict is a prediction. Sample live sessions after release and score them with the same rubric, so you find out quickly if the prediction was wrong. None of these are reasons to avoid scoring. They are reasons to keep the scoring logic where your team can see it, question it, and fix it. The bottom line. AgentScore validates something many QA teams already know: agents need probabilistic, behaviour-level evaluation with a clear release decision at the end. You can get that decision today without waiting for early access, and without accepting a score you cannot open up. Write the rubric, set the floors, score with open tools, and keep the reasoning visible. Want an independent ship or block call on your agent? Its AI QA Readiness Assessment is a fixed-scope, 3-day agent readiness audit: aiml score your agent against this rubric on your own traces and hand you a written ship, block, or review report with the failing cases attached. Book a readiness audit.

Tricentis opens Riyadh office to support growing demand across Saudi Arabia. New office marks the company's second location in the Middle East and supports enterprise and public-sector quality engineering programs across the Gulf RIYADH, SAUDI ARABIA - September 14, 2026 - Tricentis, the global leader in agentic quality engineering, today announced the opening of a new sales office in Riyadh, Kingdom of Saudi Arabia. Alongside the office, Tricentis has established a local legal entity, creating a permanent presence in the Kingdom and supporting the company's continued investment in the Saudi market, including local hiring, customer engagement, and partner development. The move follows the opening of Tricentis' Dubai office in December 2024. The Riyadh office supports sustained customer growth across the Middle East over the past three years, with organizations in telecommunications, banking and government adopting Tricentis to accelerate delivery of critical transformation programs. Of note, Tricentis has seen: * 440% growth in customers ranging across all major industries, including (but not limited to): * The largest telco provider in the region * Two of the Kingdom's leading banks * Two of Saudi Arabia's largest giga projects * Strategic government entities supporting critical programs * Annual Recurring Revenue (ARR) growth rate of 13X * 132% Compound Annual Growth Rate (CAGR) Tricentis is pursuing a partner-led go-to-market strategy in the Middle East, working with global partners and systems integrators, including Deloitte, Accenture, Capgemini, IBM, DXC, LTM, Infosys, and Wipro, alongside established regional partners such as Testcrew, QPros, TTC Global, Adree, and more. "Saudi Vision 2030 set one of the most ambitious digital transformation agendas in the world, and with 2026 being designated the Year of AI, the Kingdom is aiming to set the global pace," said Ahmad Al Sawafta, Head of Saudi Arabia, Kuwait, and Bahrain at Tricentis. "As AI moves from pilot into production across the Kingdom's enterprises and government entities, the systems behind it have to be dependable, and quality engineering is what makes transformation at this speed and scale possible." "Tricentis is helping organizations build, validate, and deliver high-quality software at scale and speed for the critical programs the country's citizens and customers depend on," added Dan Melville, VP of Tricentis EMEA. "Being on the ground in Riyadh is a huge positive step for Tricentis enabling us to be closer to our customers and partners as we continue to shape and assure many of the largest and most critical programs in the Kingdom." "As organizations across the Kingdom accelerate cloud and AI adoption, quality and resilience are essential to successful transformation," said Ahmed Al-Faifi, Senior Vice President and Managing Director, SAP Middle East and Africa - North. "A strong partner ecosystem plays an important role in supporting customers on this journey, and Tricentis brings valuable quality engineering expertise that complements our work with customers across Saudi Arabia." As AI reshapes software development, enterprises face the challenge of ensuring that AI-generated code and AI-powered systems are reliable, secure, and production-ready. Tricentis works with customers in the region to validate these systems at the pace their programs demand. Resources: About Tricentis: Tricentis is the global leader in agentic quality engineering. The Tricentis Agentic Quality Engineering Platform leverages the power of AI and decades of Tricentis technology and expertise to provide a new and fundamentally different way to ensure software quality across large and complex enterprise application environments. An approach that's automated, fully codeless, and intelligently driven by AI. It addresses both agile development and complex enterprise apps, enabling enterprises to accelerate their digital transformation by dramatically increasing software release speed, reducing costs, and improving software quality. Widely credited for reinventing software testing for DevOps, cloud, and enterprise applications, Tricentis has been recognized as a leader by all major industry analysts, including Forrester, Gartner, and IDC. Tricentis has more than 3,000 customers, including some of the globe's most respected brands such as Experian, T-Mobile, Jaguar Land Rover, Allianz, Telstra, Dolby, and VodafoneZiggo. To learn more, visit www.tricentis.com.

Tricentis unveils AI innovations for enterprise development. Last updated: September 3, 2026 6:39 am Tricentis, a global leader in agent-based quality engineering, announced new AI-powered innovations at its annual industry conference, Tricentis Transform, to help companies build, validate, and deliver high-quality software at scale. This announcement includes three new technologies developed through Tricentis Labs, an innovation incubator. Tricentis Labs provides users with early access to AI technologies under development by Tricentis, as well as opportunities to participate in development and co-shape its direction through close collaboration with the company. This allows new ideas to be developed into practical solutions suitable for enterprise use. Tricentis Labs and its innovative technologies demonstrate Tricentis' continued investment and commitment to the future of agent-based quality engineering. As AI continues to transform software development, businesses face the challenge of ensuring the reliability, security, and readiness for production deployment of AI-generated applications and AI-powered systems. Furthermore, the rapid pace of AI innovation demands even greater speed in developing and validating new technologies. Tricentis Labs aims to deliver these new AI technologies to customers and partners more quickly. This allows Tricentis and participating companies to leverage insights gained from validating ideas based on real-world business demands and learning from actual use, enabling them to rapidly evolve promising technologies and develop solutions suitable for enterprise use.