Founded in 2019, Arthur has secured over $60M in funding from several firms, including Acrew, Greycroft, Index Ventures, BAM Elevate, Work-Bench, and Plexo Capital.

Arthur Bench introduces an extensive suite of evaluation features.

Arthur also unveiled The Generative Assessment Project (GAP) — a research initiative tracking the strengths and weaknesses of language model offerings of OpenAI, Anthropic, Meta, and others as they evolve over timeNEW YORK, Aug. 17, 2023 /PRNewswire/ -- Arthur , an AI performance platform trusted by some of the largest organizations in the world to ensure that their AI systems are well-managed and deployed in a responsible manner, today introduced Arthur Bench, an open-source evaluation tool for comparing large language models (LLMs), prompts, and hyperparameters for generative text models. This open-source tool will enable businesses to evaluate how different LLMs will perform in real-world scenarios so they can make informed, data-driven decisions when integrating the latest AI technologies into their operations.In conjunction with Arthur Bench, Arthur also unveiled The Generative Assessment Project (GAP), a research initiative ranking the strengths and weaknesses of language model offerings from industry leaders like OpenAI, Anthropic, and Meta. Notably, Arthur's research suggests that Anthropic may be gaining a slight competitive edge against OpenAI's GPT-4 on measures of "reliability" within specific domains. For example, while GPT-4 was the most successful when answering math questions, Anthropic's Claude-2 model was stronger at avoiding hallucinated factual mistakes and answering "I don't know" at appropriate times when answering history questions. Through GAP, Arthur will continue to share discoveries about behavior differences and best practices with the public in its journey to make LLMs work for everyone."As our GAP research clearly shows, understanding the differences in performance between LLMs can have an incredible amount of nuance