Artificial Analysis' AA-Omniscience v1.1 catches AI models guessing instead of abstaining. Artificial Analysis refreshed its knowledge and hallucination benchmark, cutting it to 3,000 questions and exposing which models were quietly overfitting. · 21 hrs ago Read 5 min * AA-Omniscience v1.1 shrinks to 3,000 questions across 6 domains, removing saturated and ambiguous items. * Gemini 4 Carbon leads with Net Score 54, ahead of Claude Opus 5.5 (48) and Claude Fable 5.1 (46). * Rank order holds with 0.98 Spearman correlation to v1, but median Net Score drops 0.6 points. * MiniMax M3.1 Flash collapses 22 points to -6; MiMo V2.6 Pro loses 13, suggesting contamination. * Claude Fable 5.1 tops accuracy at 68%; Gemini 4 Argon has lowest hallucination rate at 4%. * New public 300-question set on Hugging Face correlates 0.99 with the private benchmark. Artificial Analysis has released AA-Omniscience v1.1, an updated benchmark for factual knowledge and calibration. It measures whether a model answers correctly, declines when uncertain, or invents an answer. The leading models retained similar positions, while several individual scores fell sharply after the question set changed. A smaller test with sharper questions. Version 1.1 contains 3,000 private questions across six domains, down from the 6,000 questions described for v1. Artificial Analysis removed questions that most models had mastered, revised ambiguous items, and added new material. AA-Omniscience v1.1 question distribution | Domain | Questions | Share | | Science, Engineering, and Mathematics | 600 | 20% | | Software Engineering | 600 | 20% | | Finance | 450 | 15% | | Health | 450 | 15% | | Humanities and Social Sciences | 450 | 15% | | Law | 450 | 15% | Artificial Analysis also released a recalibrated 300-question public set. Across evaluated models, its Net Scores have Pearson and Spearman correlations of 0.99 with the private set, while the median public-private difference is 0.0 points. Pearson correlation measures agreement between scores; Spearman correlation measures whether the model rankings stay aligned. Scoring rewards restraint. Each question contributes to the AA-Omniscience Index according to a simple rule: * Correct answer: +1 * Incorrect answer: -1 * Abstention: 0 The normalized Net Score ranges from -100 to 100 and can be expressed as accuracy − hallucination rate. A score of 0 means the model produced equal numbers of correct and incorrect answers. Negative scores indicate more incorrect answers than correct ones. A model that answers 40% of questions correctly, answers 15% incorrectly, and abstains on the remaining 45% receives a Net Score of 25. This structure gives models credit for useful knowledge while penalizing unsupported guesses. Frontier models keep the lead. Leading v1.1 results by metric | Metric | First | Second | Third | | Net Score | Gemini 4 Carbon (High), 54 | Claude Opus 5.5 (Max, Default Fallback), 48 | Claude Fable 5.1 (Max, Default Fallback), 46 | | Accuracy | Claude Fable 5.1 (Max, Default Fallback), 68% | Claude Fable 5.1 (Xhigh, Default Fallback), 67% | Claude Opus 5.5 (Xhigh, Default Fallback), 66% | | Lowest hallucination rate | Gemini 4 Argon (High), 4% | Gemini 4 Carbon (High), 11% | Command A+, 14% | Muse Spark Open leads the open-weights entries and trails the overall leader by roughly 28 points. The result establishes a gap on this benchmark; coding ability, reasoning, latency, cost, and deployment flexibility require separate evaluation. Configuration labels such as High, Max, Xhigh, and Default Fallback identify distinct leaderboard entries. Reproductions and vendor comparisons should preserve the exact model version, inference configuration, system prompt, and decoding settings. The refresh moves four models sharply. Across evaluated models, the average Net Score declined by 1.9 points and the median declined by 0.6 points. Most rankings remained stable, but four reported changes were substantially larger. Largest reported Net Score declines | Model | Change | Reported movement | | MiniMax M3.1 Flash | -22 points | +16 to -6 | | Xiaomi MiMo V2.6 Pro | -13 points | Not specified | | GPT-6 Luna | -9 points | Not specified | | Z AI GLM-5.3 | -8.3 points | Not specified | MiniMax M3.1 Flash's move from +16 to -6 crosses a meaningful scoring boundary: under v1.1, it produced more incorrect answers than correct ones after abstentions were excluded from the penalty. Large drops raise contamination questions. An overall rank correlation of 0.98 means the ordering remained similar across releases. Large changes for a small number of models are consistent with exposure to old public questions during training, especially when benchmark items have circulated widely. Score changes alone cannot establish contamination. Revised wording, domain composition, grading behavior, model updates, sampling settings, and sensitivity to particular questions can also produce large deltas. A controlled contamination audit would require training-data evidence or experiments that compare exposed and unexposed items. The close alignment between the new public and private sets reduces the need to expose private questions. Its reported calibration applies across the models tested by Artificial Analysis, while individual models may still diverge because a 300-question sample cannot capture every behavior. Choose metrics by failure cost. Benchmarks that require an answer to every question can reward guessing. AA-Omniscience instead exposes the trade-off among accuracy, hallucination, and abstention, which matters when an unsupported answer carries more risk than a refusal. The median hallucination rate fell by 1.4 percentage points on v1.1. Because the questions changed, that decline should be interpreted as a property of performance on the refreshed set rather than a general improvement in model honesty. How to use the benchmark in model selection | Selection goal | Prioritize | Validate separately | | Reduce fabricated answers | Hallucination rate and Net Score | Refusal behavior under production prompts | | Maximize answer coverage | Accuracy and abstention frequency | Task utility, latency, and cost | | Build legal or medical systems | Relevant domain scores | Expert review, citation validity, and safety | | Build retrieval-augmented systems | Net Score and hallucination rate | Retrieval recall, grounding, and source attribution | AA-Omniscience covers parametric knowledge under benchmark prompts. Retrieval quality, citation accuracy, tool use, long-context behavior, prompt-injection resistance, and domain-specific workflows remain outside its scope. Teams can use the benchmark as an initial screening signal and follow it with evaluations built from representative production tasks. Run the public set carefully. The 300-question public set is available on Hugging Face, while the underlying research paper describes question generation and grading. * Pin the model version, provider, system prompt, inference configuration, and decoding settings. * Run all 300 questions using the published prompts and grading method. * Report Net Score, accuracy, hallucination rate, abstention frequency, and domain-level results. * Repeat a sample when outputs are stochastic and record the resulting variance. * Follow the benchmark with a private evaluation drawn from the intended application. The reported 0.99 correlations and median 0.0-point difference support using the public set as a preliminary estimate of private leaderboard performance. Exact results can still vary with model-specific behavior, serving infrastructure, and evaluation settings.

Before you call Gemini 4 Carbon the top AI model, check the ranking details. Gemini 4 Carbon tops Artificial Analysis' Intelligence Index v5 with 59 points, but its results vary by task and it is not publicly available yet. Up next. Published on 29 October 2026 AI This post was created with the assistance of artificial intelligence (AI). Before you order Offer from Amazon Get hardware and tech essentials delivered free with Prime * Fast, free delivery on millions of items * Prime Video, Amazon Music and more included * Member-only deals all year As an affiliate, One2crypto earn on qualifying purchases. Gemini 4 Carbon scored 59 on Artificial Analysis' Intelligence Index v5, one point ahead of Claude Opus 5.5 and two ahead of GPT-6 Astra. It leads four of nine tests, but the benchmark does not establish an overall best model for every task, and Google has not made Carbon publicly available. Gemini 4 Carbon has taken first place on the Artificial Analysis Intelligence Index v5, scoring 59 points - one ahead of Claude Opus 5.5 and two ahead of GPT-6 Astra. The result makes Carbon the leader on this particular nine-test benchmark, not a universal winner: it places first on four tests, trails rivals on five, and is not yet publicly available. Artificial Analysis, an independent benchmarking company, introduced version 5 on October 27. Its index gives equal weight to four areas - knowledge work, coding, science and general capability - and combines results from nine tests. The reported scores put Claude Opus 5.5 at 58, GPT-6 Astra at 57, GPT-6.1 Sol at 55 and Google's previous model, Gemini 4 Argon, at 50. Carbon leads on four of the nine tests: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. Its reported scores include 69% on HLE-Diamond, against 68% for Claude Opus 5.5, and 65% on Terminal-Bench-Science, against 63% for GPT-6 Astra. On AutomationBench-AA, which evaluates business workflows using software tools, Carbon narrowly outscored its predecessor. It falls behind competitors on the other five tests, including office-style knowledge work, real-world coding, physics research, terminal tasks and document reasoning. On the index's office-work measure, Carbon scored 1,609 Elo, compared with 1,812 for Claude Sonnet 5.5 and 1,800 for Claude Opus 5.5. On FrontierCode 2.0, it scored 41%, while Claude Opus 5.5 scored 46%. At a glance report When: Index v5 launched October 27, 2026; Car... The development Artificial Analysis' new Intelligence Index v5 ranks Gemini 4 Carbon first with 59 points, narrowly ahead of Claude Opus 5.5. Crypto market snapshot Fear & Greed Index 51/100 - Neutral Bitcoin BTC $80,182 1.5% Ethereum ETH $2,323 1.9% Tether USDT $0.999 0.0% BNB BNB $740.83 1.9% XRP XRP $1.29 2.9% USDC USDC $0.9995 0.0% Solana SOL $100.34 2.5% TRON TRX $0.3311 0.2% Live data · CoinGecko · alternative.me (24h change) Thorsten Meyer AI Infographic 01 · What the #1 is made of Number one overall. First in four of nine tests. The Intelligence Index is a decathlon score: nine tests, one number. Carbon wins on reasoning, science and careful knowledge, and trails on office work and coding. 59 Gemini 4 Carbon on Intelligence Index v5 1 point ahead of Claude Opus 5.5 (58) 4 / 9 tests where Carbon scores highest 60% of the index runs on private tests or held-out answers The nine tests behind the score. Carbon at its highest setting (high) against the best model on each test. | Test | Weight | Carbon | Leader or runner-up | | HLE-Diamond Hard reasoning + knowledge questions | 10% | 69% | Carbon leads Next: Claude Opus 5.5, 68% | | Terminal-Bench-Science Science research work in a terminal | 15% | 65% | Carbon leads Next: GPT-6 Astra, 63% | | AA-Omniscience v1.1 Knowledge, wrong answers penalized | 15% | 54 | Carbon leads Next: Claude Opus 5.5, 48 | | AutomationBench-AA Business workflows across apps | 5% | 78% | Carbon leads Next: Gemini 4 Argon, 78% | | AA-Briefcase Office-style knowledge work | 15% | 1,609 Elo | Behind Claude Sonnet 5.5, 1,812 | | FrontierCode 2.0 Code a maintainer would merge | 15% | 41% | Behind Claude Opus 5.5, 46% | | CritPt v2 (hard) Research-level physics | 10% | 63% | Behind GPT-6 Astra, 71% | | Terminal-Bench 4.0 Agentic terminal tasks | 10% | 65% | Behind Kimi K3.1 Pro, 67% | | GDP.pdf Reasoning over documents | 5% | 28% | Behind Muse Wonder Full, 33% | The surprise in the knowledge test. * Accuracy: Claude Fable 5.1 68%, Claude Opus 5.5 66%, GPT-6.1 Sol and GPT-6 Astra 62%, Carbon 58% * Hallucination rate: Carbon 11%, Claude Opus 5.5 and GPT-6 Astra about 50% * Carbon wins because it rarely bluffs, not because it knows the most Read the weights. * Each of the four blocks counts 25%: knowledge work, coding, science, general * Carbon trails in two of the four heaviest tests (15% each): office-style work and real-world code * Hallucination rate = wrong answers among all questions not answered correctly Why the one-point lead has limits. The ranking can help organizations compare models, but the 59-point total compresses different strengths into one number. A one-point margin over Claude Opus 5.5 is small, and the result depends on the index's selected tasks, their weights and the model settings tested. It should not be read as proof that Carbon performs best across every product, workflow or user's needs. Cost and access also affect the practical value of the result. The source reports a standard-price estimate of $7.76 per index task for Carbon - less than Claude Opus 5.5's $10.04, but above GPT-6 Astra's $6.24 and well above GPT-6.1 Sol's $1.27. These figures describe the benchmark's task costs; they do not show what a particular organization would pay for its own workload. For people choosing a system, the useful comparison is performance on their actual tasks alongside cost, speed and reliability. A model that leads an aggregate benchmark may not be the strongest or least expensive option for a company's coding, document or office work. The ranking is a comparison point, not a recommendation to adopt a particular model.

AA-Robotics v0.1 puts frontier language models in charge of a robot arm. Artificial Analysis finds GPT-6 Astra leads its first embodied benchmark, but every model remains far slower than a human Sponsored: Hyperlucky - Hyperlucky Welcome Offer Tap To Claim Large language models can write essays, debug code and summarize earnings calls. Now someone has handed them a robot arm and asked them to write "hello world" with a pen. On October 14, 2026, Artificial Analysis released AA-Robotics v0.1. The research preview tests how well frontier language models understand, plan and act in the physical world. How the benchmark works. The setup is intentionally simple. Each model gets control of a Hiwonder NexArm, a 6-axis robot arm. It also gets a single, plainly stated objective. The models operate through a coding-agent interface connected to a simple HTTP API. For eyes, the system relies on four cameras that feed visual information back to the model. The tasks run zero-shot. Each model attempts every task without prior practice or task-specific training. Artificial Analysis is publishing results for its first eight tasks. Examples include pouring water, stacking blocks and writing "hello world." A complete success earns 100 points, and partial credit rewards progress on attempts that stop short of the finish line. The leaderboard. OpenAI's GPT-6 Astra sits at the top with an average score of 69.4/100 across the eight tasks. Another OpenAI model, GPT-6.1 Sol, came second at 64.6. Google's Gemini 4 Argon placed third with 55.3. AI, tech, and the markets they move - in one daily briefing. Daily. Free. Join 34,000+ readers across crypto, finance, and policy. The "hello world" task offers the clearest look at how differently these models approach the same problem. GPT-6 Astra finished the writing exercise in 514 arm moves and 13 minutes. Anthropic's Claude Opus 5.5 got there too, eventually. It needed 3,270 moves and 34 minutes. That is more than six times as many moves for the same two words. The speed problem. Humans complete these tasks in about 20 seconds on average. The AI models take approximately 30 minutes. Artificial Analysis puts the gap at roughly 100 times slower than people. The models are also substantially more expensive than human labor for the same work. Why this benchmark exists. Most AI benchmarks measure what models can do on a screen. They test reasoning, coding, math and reading comprehension. Physical tasks add a layer that text benchmarks miss entirely: a model has to translate an abstract goal into a sequence of real movements, deal with imperfect sensor data and recover when a block slips or a pen skids. Artificial Analysis says progress on these embodied capabilities has been accelerating in 2026. It also says efficiency and reliability still need substantial improvement before deployment in real-world settings becomes practical. What this means. OpenAI holds the top two spots in the first release. Google sits third, and Anthropic's flagship trails noticeably on at least one task. AA-Robotics shows that using general-purpose language models as the brain in a zero-shot setting can work. It also shows the cost of that generality is measured in minutes per task rather than seconds. A system that is around 100 times slower and costlier than a person is not replacing anyone on a production line yet. Disclosure: This article was edited by Diego Almada Lopez. For more information on how Cryptobriefing create and review content, see its Editorial Policy.