AI Benchmark Digest — 2026-06-14

        June 14, 2026

AI Benchmark Digest — 2026-06-14

AI Benchmark Digest — 2026-06-14
View on AI Benchmark Hub
Daily
New Benchmarks (75)

Ramp SWE-Bench (Resolved (%)): Claude Fable 5 leads with 87.5 across 14 models.
  Ramp Labs benchmark for background coding agents on realistic financial software engineering work, scored by resolved tasks with the mini-SWE-agent harness.
CADGenBench (Aggregate CAD Score): Claude Fable 5 leads with 0.4514 across 11 models.
  CAD generation and editing benchmark scoring generated CAD artifacts on aggregate geometric and validity metrics across validated submissions.
FrontierMath - Tier 4 (v2) (Accuracy (%, 41 private v2 problems)): Claude Fable 5 (max) leads with 87.8 across 27 models.
  Current v2 private Tier 4 FrontierMath expansion set from Epoch AI, measuring accuracy on the hardest unpublished research-level mathematics problems.
FrontierMath - Tiers 1-3 (v2) (Accuracy (%, 285 private v2 problems)): GPT-5.5 Pro (xhigh) leads with 87.72 across 26 models.
  Current v2 private FrontierMath base set from Epoch AI, covering original problems from undergraduate through early-postdoc difficulty across major areas of modern mathematics.
Benchmarks.bio - SpatialBench (Pass Rate (%)): GPT-5.5 leads with 69.57 across 11 models.
  LatchBio agentic benchmark on messy real-world spatial transcriptomics data, with models writing and running analysis workflows across assays, platforms, and task categories.
Benchmarks.bio - scBench (Pass Rate (%)): Claude Mythos 5 leads with 59.3 across 13 models.
  LatchBio agentic benchmark for single-cell RNA-seq analysis, requiring models to perform realistic data cleaning, clustering, cell typing, and differential-expression workflows.
Benchmarks.bio - SpatialBench-Long (Pass Rate (%)): Gemini 3.5 Flash leads with 11.11 across 12 models.
  Long-form Benchmarks.bio spatial transcriptomics tasks that require multi-step biological data analysis, tool use, and synthesis over larger assay contexts.
Benchmarks.bio - EpiBench (Pass Rate (%)): GPT-5.5 leads with 44.97 across 11 models.
  Benchmarks.bio epigenomics benchmark covering real assays such as chromatin accessibility, binding, and methylation analyses with deterministic graders.
Agent Arena (Net Improvement (%)): Grok 4.3 xAI · Proprietary leads with 18.3 across 25 models.
  Arena.ai agent leaderboard measuring net improvement on real-world tool orchestration sessions with success, steerability, recovery, and hallucination metrics.
Agent Arena - Confirmed Success (Confirmed Success (%)): Claude Fable 5 (High) leads with 17.21 across 25 models.
  Agent Arena submetric tracking confirmed successful completion rate on real-world agent sessions.
Agent Arena - Praise vs Complaint (Praise vs Complaint (%)): Claude Fable 5 (High) leads with 27.74 across 25 models.
  Agent Arena submetric comparing user praise against complaints across agent sessions.
Agent Arena - Steerability (Steerability (%)): Nemotron 3 Ultra leads with 23.87 across 25 models.
  Agent Arena submetric measuring how well models adapt to user steering during tool-use sessions.
Agent Arena - Bash Recovery (Bash Recovery (%)): Grok 4.3 xAI · Proprietary leads with 60.23 across 25 models.
  Agent Arena submetric measuring recovery from shell or command-line failures in agent sessions.
Agent Arena - Tool Hallucination (Tool Hallucination (%)): Grok 4.3 xAI · Proprietary leads with 0.26 across 25 models.
  Agent Arena submetric measuring tool hallucination rate; lower values indicate fewer invented or invalid tool uses.
Agents' Last Exam (Pass Rate (%)): GPT-5.5 leads with 24.0 across 18 models.
  Snorkel benchmark of long-horizon economically valuable agent tasks across many industries, reporting workflow pass rate and score.
WolfBench (Average Score (%)): GPT-5.5 leads with 77.0 across 27 models.
  Agent benchmark based on Terminal-Bench 2.0 that compares harnesses and models across repeated terminal task runs using aggregate score statistics.
Appwrite Arena (With Skills) (Overall Score (%)): GPT-5.5 leads with 97.7 across 16 models.
  Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks when models can use Appwrite skills.
Appwrite Arena (Without Skills) (Overall Score (%)): Claude Fable 5 leads with 97.7 across 16 models.
  Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks without Appwrite skill assistance.
Terminal-Bench 2.1 (Accuracy (%)): GPT-5.5 leads with 83.4 across 6 models.
  Official Terminal-Bench 2.1 leaderboard measuring agent success on realistic command-line tasks, using each model best available harness row.
Terminal-Bench 2.1 (Claude Code) (Accuracy (%)): Claude Opus 4.8 leads with 78.9 across 3 models.
  Terminal-Bench 2.1 results for the Claude Code harness, measuring command-line task completion by model.
Terminal-Bench 2.1 (Terminus 2) (Accuracy (%)): GPT-5.5 leads with 78.2 across 5 models.
  Terminal-Bench 2.1 results for the Terminus 2 harness, measuring command-line task completion by model.
Vals AI Finance Agent v2 (Accuracy (%)): gemini-3.5-flash leads with 57.86 across 29 models.
  Updated Vals AI financial-research agent benchmark over SEC filings and supporting documents, measuring completion accuracy on realistic analyst workflows.
Vals AI Public Benefits Bench (Accuracy (%)): claude-fable-5 leads with 71.65 across 13 models.
  SNAP public-benefits guidance benchmark measuring whether models answer benefits questions accurately while following eligibility and documentation rules.
Vals AI Terminal-Bench 2.1 (Accuracy (%)): claude-fable-5 leads with 80.52 across 30 models.
  Updated Terminal-Bench 2.1 evaluation from Vals AI, measuring agentic command-line task completion in sandboxed software and systems environments.
Vals AI LiveCodeBench (Accuracy (%)): claude-fable-5 leads with 89.78 across 121 models.
  Vals AI run of LiveCodeBench coding problems, measuring pass rates on recent contest-style programming tasks intended to reduce contamination.
Vals AI GPQA (Accuracy (%)): gemini-3.1-pro-preview leads with 95.45 across 115 models.
  Vals AI run of GPQA graduate-level science questions, measuring difficult expert-domain reasoning accuracy.
Vals AI MMLU-Pro (Accuracy (%)): claude-fable-5 leads with 91.5 across 114 models.
  Vals AI run of MMLU-Pro multitask academic questions, using harder multi-choice problems across STEM, humanities, and professional domains.
Vals AI MMMU (Accuracy (%)): claude-fable-5 leads with 89.31 across 76 models.
  Vals AI run of MMMU multimodal college-level subject questions, measuring visual and textual academic reasoning.
Vals AI SWE-bench Verified (Resolved (%)): claude-fable-5 leads with 95.0 across 57 models.
  Vals AI SWE-bench Verified leaderboard, measuring the percentage of real GitHub issues resolved by coding agents.
GDP.pdf (Strict Pass Rate (%)): Claude Fable 5 leads with 30.0 across 12 models.
  Surge AI document-reasoning benchmark over 100 professional PDF workflows, scored by strict pass rate against expert-written rubrics.
Riemann-bench (Score (%)): Claude Fable 5 leads with 55.0 across 15 models.
  Surge AI frontier mathematics benchmark with advanced research-style problems sourced from mathematicians and scored by solution correctness.
SWE-bench Pro (Anthropic Scaffold) (Pass@1 (%)): Claude Mythos 5 leads with 80.3 across 6 models.
  Anthropic system-card run of SWE-bench Pro, measuring pass@1 on production software engineering issues using Anthropic scaffold settings.
OfficeQA Pro (Correctness (%)): Claude Fable 5 leads with 57.9 across 4 models.
  Hard OfficeQA subset for frontier document agents, requiring grounded search and numerical reasoning over U.S. Treasury Bulletin documents.
Real-World Finance v2 (Elo): Claude Fable 5 leads with 1374.0 across 4 models.
  Anthropic long-horizon finance workflow evaluation using pairwise preference grading and Elo ratings over realistic professional deliverables.
Real-World Finance v1 (Score (%)): Claude Mythos Preview leads with 70.9 across 4 models.
  Anthropic curated finance benchmark of 53 tasks evaluated against reference answers with a model-based grader.
Legal Agent Benchmark (Harvey Held-Out) (All-Pass Rate (%)): Claude Fable 5 leads with 13.3 across 5 models.
  Harvey legal-agent held-out evaluation using closed-universe matter files and expert rubrics, scored by all-pass task success.
Toolathlon (Anthropic Internal Harness) (Pass@1 (%)): Claude Fable 5 leads with 61.7 across 7 models.
  Anthropic internal Toolathlon harness over 108 tool-use tasks, reporting pass@1 for agentic workflow completion.
SWE-bench Verified (Anthropic Scaffold) (Resolved (%)): Claude Opus 4.8 leads with 88.6 across 3 models.
  Anthropic system-card run of SWE-bench Verified, measuring real GitHub issue resolution with Anthropic scaffold settings.
SWE-bench Multilingual (Anthropic Scaffold) (Resolved (%)): Claude Opus 4.8 leads with 84.4 across 2 models.
  Anthropic system-card run of SWE-bench Multilingual, measuring multilingual software issue resolution with Anthropic scaffold settings.
SWE-bench Multimodal (Anthropic Internal Harness) (Resolved (%)): Claude Opus 4.8 leads with 38.4 across 2 models.
  Anthropic internal multimodal SWE-bench harness, measuring software issue resolution that requires visual or multimodal context.
Humanity's Last Exam (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 49.8 across 4 models.
  Anthropic system-card run of Humanitys Last Exam without tools, covering expert-level academic reasoning across many domains.
Humanity's Last Exam (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 57.9 across 4 models.
  Anthropic system-card run of Humanitys Last Exam with tools, covering expert-level academic reasoning across many domains.
ChartQAPro (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 69.4 across 2 models.
  Anthropic no-tool run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ChartQAPro (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 72.3 across 2 models.
  Anthropic tool-enabled run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ScreenSpot-Pro (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 82.3 across 2 models.
  Anthropic no-tool run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
ScreenSpot-Pro (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 87.9 across 2 models.
  Anthropic tool-enabled run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
GraphWalks BFS 256K (Anthropic) (F1 Score (%)): Claude Opus 4.8 leads with 85.9 across 4 models.
  Anthropic GraphWalks long-context graph traversal evaluation using breadth-first-search tasks at 256K context.
GraphWalks Parents 256K (Anthropic) (F1 Score (%)): Claude Opus 4.8 leads with 99.3 across 4 models.
  Anthropic GraphWalks long-context graph traversal evaluation using parent-pointer recovery tasks at 256K context.
USAMO 2026 (Anthropic) (Accuracy (%)): Claude Opus 4.8 leads with 96.7 across 2 models.
  Anthropic system-card evaluation on 2026 USAMO-style olympiad math problems, scored by answer correctness.
ArXivMath Mar-Apr 2026 (Anthropic) (Accuracy (%)): Claude Opus 4.8 leads with 71.82 across 3 models.
  Anthropic system-card evaluation on recent arXiv mathematics problems from March and April 2026.
OfficeQA (Anthropic Internal Harness) (Exact Match (%)): Claude Opus 4.8 leads with 77.6 across 2 models.
  Anthropic internal OfficeQA document-agent benchmark, requiring grounded search and numerical reasoning over office documents.
OfficeQA Pro (Anthropic Internal Harness) (Exact Match (%)): Claude Opus 4.8 leads with 66.2 across 2 models.
  Anthropic internal OfficeQA Pro hard subset, requiring grounded search and numerical reasoning over office documents.
ChartMuseum (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 75.8 across 2 models.
  Anthropic no-tool run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
ChartMuseum (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 89.7 across 2 models.
  Anthropic tool-enabled run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
LAB-Bench FigQA (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 80.4 across 2 models.
  Anthropic no-tool run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
LAB-Bench FigQA (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 87.3 across 2 models.
  Anthropic tool-enabled run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
CharXiv Reasoning (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.7 leads with 81.3 across 2 models.
  Anthropic no-tool run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
CharXiv Reasoning (Anthropic Tools) (Accuracy (%)): Claude Opus 4.7 leads with 90.1 across 2 models.
  Anthropic tool-enabled run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
HealthBench Professional (Anthropic) (Length-Adjusted Score (%)): Claude Opus 4.8 leads with 55.8 across 3 models.
  Anthropic system-card run of HealthBench Professional, measuring clinical and healthcare reasoning with length-adjusted scoring.
GMMLU (Anthropic) (Average Accuracy (%)): Gemini 3.1 Pro leads with 92.2 across 5 models.
  Anthropic system-card run of Global MMLU, measuring multilingual academic and professional knowledge.
BioPipelineBench Verified (Anthropic) (Score (%)): Claude Mythos Preview leads with 88.1 across 4 models.
  Anthropic system-card run of BioPipelineBench Verified, measuring biological data-analysis workflow completion.
BioMysteryBench Verified - Human Solvable (Anthropic) (Score (%)): Claude Mythos Preview leads with 82.6 across 4 models.
  Anthropic system-card run of BioMysteryBench Verified human-solvable tasks, testing biological mystery problem solving.
BioMysteryBench Verified - Human Difficult (Anthropic) (Score (%)): Claude Opus 4.8 leads with 40.0 across 4 models.
  Anthropic system-card run of BioMysteryBench Verified human-difficult tasks, testing hard biological mystery problem solving.
LatchBio SpatialBench (Anthropic) (Score (%)): Claude Mythos Preview leads with 53.8 across 4 models.
  Anthropic system-card run of LatchBio SpatialBench, measuring spatial transcriptomics analysis workflows.
LatchBio SingleCellBench (Anthropic) (Score (%)): Claude Opus 4.8 leads with 58.2 across 4 models.
  Anthropic system-card run of LatchBio SingleCellBench, measuring single-cell RNA-seq analysis workflows.
Structural Biology (Anthropic) (Score (%)): Claude Mythos Preview leads with 81.6 across 4 models.
  Anthropic system-card structural biology evaluation, testing biomolecular structure reasoning and analysis.
ProteinGym Hard (Anthropic) (Rank Correlation (%)): Claude Mythos Preview leads with 43.1 across 4 models.
  Anthropic system-card run of the hard ProteinGym subset, measuring protein variant effect prediction via rank correlation.
Organic Chemistry (Anthropic) (Score (%)): Claude Mythos Preview leads with 86.5 across 4 models.
  Anthropic system-card organic chemistry evaluation, testing reaction and molecule reasoning.
Protocol Troubleshooting (Anthropic) (Score (%)): Claude Mythos Preview leads with 69.6 across 4 models.
  Anthropic system-card protocol troubleshooting benchmark, testing diagnosis of laboratory protocol failures.
LABBench2 - Patent Questions (Anthropic) (Score (%)): Claude Opus 4.8 leads with 68.8 across 3 models.
  Anthropic system-card LABBench2 patent-question subset, testing life-science document reasoning over patent material.
LABBench2 - Clinical Trial Questions (Anthropic) (Score (%)): Claude Mythos Preview leads with 86.3 across 3 models.
  Anthropic system-card LABBench2 clinical-trial subset, testing life-science reasoning over trial documents.
LABBench2 - Table Reading (Anthropic) (Score (%)): Claude Opus 4.8 leads with 77.2 across 2 models.
  Anthropic system-card LABBench2 table-reading subset, testing scientific table comprehension.
LABBench2 - Supplementary Materials (Anthropic) (Score (%)): Claude Opus 4.8 leads with 58.9 across 2 models.
  Anthropic system-card LABBench2 supplementary-materials subset, testing reasoning over scientific supporting files.
Agent Security League - Functional Correctness (Functional Correctness (%)): GPT-5.5 leads with 84.9 across 15 models.
  Endor Labs coding-agent benchmark measuring whether agents functionally complete security-sensitive software tasks.
Agent Security League - Security Correctness (Security Correctness (%)): GPT-5.5 leads with 24.0 across 15 models.
  Endor Labs coding-agent benchmark measuring whether completed software tasks avoid introducing or preserving security vulnerabilities.

New #1 Leaders (1)

OpenClawProBench (Overall Score (%)): GLM-5.2 (81.3) beat intern-s2-preview (76.7) by 4.6.

Weekly
New Benchmarks (86)

FrontierCode Diamond (Score (%)): Claude Opus 4.8 leads with 13.4 across 12 models.
  Hardest 50 FrontierCode production-code tasks from Cognition, measuring whether maintainers would merge model PRs using blocker criteria and quality rubrics.
FrontierCode Main (Score (%)): Claude Opus 4.8 leads with 34.3 across 12 models.
  100 hardest FrontierCode production-code tasks, including Diamond, scored by maintainer-style mergeability criteria across correctness, tests, scope, style, and codebase standards.
FrontierCode Extended (Score (%)): Claude Opus 4.8 leads with 51.8 across 12 models.
  Full 150-task FrontierCode benchmark from Cognition, evaluating production-quality coding agents on maintainer-authored open source repository work.
Ramp SWE-Bench (Resolved (%)): Claude Fable 5 leads with 87.5 across 14 models.
  Ramp Labs benchmark for background coding agents on realistic financial software engineering work, scored by resolved tasks with the mini-SWE-agent harness.
CADGenBench (Aggregate CAD Score): Claude Fable 5 leads with 0.4514 across 11 models.
  CAD generation and editing benchmark scoring generated CAD artifacts on aggregate geometric and validity metrics across validated submissions.
FrontierMath - Tier 4 (v2) (Accuracy (%, 41 private v2 problems)): Claude Fable 5 (max) leads with 87.8 across 27 models.
  Current v2 private Tier 4 FrontierMath expansion set from Epoch AI, measuring accuracy on the hardest unpublished research-level mathematics problems.
FrontierMath - Tiers 1-3 (v2) (Accuracy (%, 285 private v2 problems)): GPT-5.5 Pro (xhigh) leads with 87.72 across 26 models.
  Current v2 private FrontierMath base set from Epoch AI, covering original problems from undergraduate through early-postdoc difficulty across major areas of modern mathematics.
Benchmarks.bio - SpatialBench (Pass Rate (%)): GPT-5.5 leads with 69.57 across 11 models.
  LatchBio agentic benchmark on messy real-world spatial transcriptomics data, with models writing and running analysis workflows across assays, platforms, and task categories.
Benchmarks.bio - scBench (Pass Rate (%)): Claude Mythos 5 leads with 59.3 across 13 models.
  LatchBio agentic benchmark for single-cell RNA-seq analysis, requiring models to perform realistic data cleaning, clustering, cell typing, and differential-expression workflows.
Benchmarks.bio - SpatialBench-Long (Pass Rate (%)): Gemini 3.5 Flash leads with 11.11 across 12 models.
  Long-form Benchmarks.bio spatial transcriptomics tasks that require multi-step biological data analysis, tool use, and synthesis over larger assay contexts.
Benchmarks.bio - EpiBench (Pass Rate (%)): GPT-5.5 leads with 44.97 across 11 models.
  Benchmarks.bio epigenomics benchmark covering real assays such as chromatin accessibility, binding, and methylation analyses with deterministic graders.
Agent Arena (Net Improvement (%)): Grok 4.3 xAI · Proprietary leads with 18.3 across 25 models.
  Arena.ai agent leaderboard measuring net improvement on real-world tool orchestration sessions with success, steerability, recovery, and hallucination metrics.
Agent Arena - Confirmed Success (Confirmed Success (%)): Claude Fable 5 (High) leads with 17.21 across 25 models.
  Agent Arena submetric tracking confirmed successful completion rate on real-world agent sessions.
Agent Arena - Praise vs Complaint (Praise vs Complaint (%)): Claude Fable 5 (High) leads with 27.74 across 25 models.
  Agent Arena submetric comparing user praise against complaints across agent sessions.
Agent Arena - Steerability (Steerability (%)): Nemotron 3 Ultra leads with 23.87 across 25 models.
  Agent Arena submetric measuring how well models adapt to user steering during tool-use sessions.
Agent Arena - Bash Recovery (Bash Recovery (%)): Grok 4.3 xAI · Proprietary leads with 60.23 across 25 models.
  Agent Arena submetric measuring recovery from shell or command-line failures in agent sessions.
Agent Arena - Tool Hallucination (Tool Hallucination (%)): Grok 4.3 xAI · Proprietary leads with 0.26 across 25 models.
  Agent Arena submetric measuring tool hallucination rate; lower values indicate fewer invented or invalid tool uses.
Agents' Last Exam (Pass Rate (%)): GPT-5.5 leads with 24.0 across 18 models.
  Snorkel benchmark of long-horizon economically valuable agent tasks across many industries, reporting workflow pass rate and score.
WolfBench (Average Score (%)): GPT-5.5 leads with 77.0 across 27 models.
  Agent benchmark based on Terminal-Bench 2.0 that compares harnesses and models across repeated terminal task runs using aggregate score statistics.
Appwrite Arena (With Skills) (Overall Score (%)): GPT-5.5 leads with 97.7 across 16 models.
  Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks when models can use Appwrite skills.
Appwrite Arena (Without Skills) (Overall Score (%)): Claude Fable 5 leads with 97.7 across 16 models.
  Appwrite Arena evaluation of model knowledge and reasoning about Appwrite development tasks without Appwrite skill assistance.
Terminal-Bench 2.1 (Accuracy (%)): GPT-5.5 leads with 83.4 across 6 models.
  Official Terminal-Bench 2.1 leaderboard measuring agent success on realistic command-line tasks, using each model best available harness row.
Terminal-Bench 2.1 (Claude Code) (Accuracy (%)): Claude Opus 4.8 leads with 78.9 across 3 models.
  Terminal-Bench 2.1 results for the Claude Code harness, measuring command-line task completion by model.
Terminal-Bench 2.1 (Terminus 2) (Accuracy (%)): GPT-5.5 leads with 78.2 across 5 models.
  Terminal-Bench 2.1 results for the Terminus 2 harness, measuring command-line task completion by model.
Vals AI Finance Agent v2 (Accuracy (%)): gemini-3.5-flash leads with 57.86 across 29 models.
  Updated Vals AI financial-research agent benchmark over SEC filings and supporting documents, measuring completion accuracy on realistic analyst workflows.
Vals AI Public Benefits Bench (Accuracy (%)): claude-fable-5 leads with 71.65 across 13 models.
  SNAP public-benefits guidance benchmark measuring whether models answer benefits questions accurately while following eligibility and documentation rules.
Vals AI Terminal-Bench 2.1 (Accuracy (%)): claude-fable-5 leads with 80.52 across 30 models.
  Updated Terminal-Bench 2.1 evaluation from Vals AI, measuring agentic command-line task completion in sandboxed software and systems environments.
Vals AI LiveCodeBench (Accuracy (%)): claude-fable-5 leads with 89.78 across 121 models.
  Vals AI run of LiveCodeBench coding problems, measuring pass rates on recent contest-style programming tasks intended to reduce contamination.
Vals AI GPQA (Accuracy (%)): gemini-3.1-pro-preview leads with 95.45 across 115 models.
  Vals AI run of GPQA graduate-level science questions, measuring difficult expert-domain reasoning accuracy.
Vals AI MMLU-Pro (Accuracy (%)): claude-fable-5 leads with 91.5 across 114 models.
  Vals AI run of MMLU-Pro multitask academic questions, using harder multi-choice problems across STEM, humanities, and professional domains.
Vals AI MMMU (Accuracy (%)): claude-fable-5 leads with 89.31 across 76 models.
  Vals AI run of MMMU multimodal college-level subject questions, measuring visual and textual academic reasoning.
Vals AI SWE-bench Verified (Resolved (%)): claude-fable-5 leads with 95.0 across 57 models.
  Vals AI SWE-bench Verified leaderboard, measuring the percentage of real GitHub issues resolved by coding agents.
Icelandic LLM Leaderboard - Average (Average Score (%)): Gemini 3.1 Pro Preview leads with 88.54 across 86 models.
  Icelandic LLM leaderboard aggregating WinoGrande-IS, GED, Inflection, Belebele-IS, ARC-Challenge-IS, and WikiQA-IS for Icelandic language understanding and reasoning.
Icelandic LLM - WinoGrande-IS (Score (%)): Gemini 3.1 Pro Preview leads with 96.14 across 86 models.
  Icelandic WinoGrande common-sense reasoning score.
Icelandic LLM - GED (Score (%)): Claude Fable 5 leads with 91.5 across 86 models.
  Icelandic grammatical error detection score.
Icelandic LLM - Inflection (Score (%)): GPT-5.5 leads with 97.96 across 86 models.
  Icelandic morphological inflection score.
Icelandic LLM - Belebele-IS (Score (%)): Gemini 3.1 Pro Preview leads with 95.0 across 86 models.
  Icelandic Belebele reading-comprehension score.
Icelandic LLM - ARC-Challenge-IS (Score (%)): GPT-5.5 leads with 95.22 across 86 models.
  Icelandic ARC-Challenge science and commonsense reasoning score.
Icelandic LLM - WikiQA-IS (Score (%)): Claude Fable 5 leads with 75.39 across 86 models.
  Icelandic WikiQA question-answering score.
GDP.pdf (Strict Pass Rate (%)): Claude Fable 5 leads with 30.0 across 12 models.
  Surge AI document-reasoning benchmark over 100 professional PDF workflows, scored by strict pass rate against expert-written rubrics.
Riemann-bench (Score (%)): Claude Fable 5 leads with 55.0 across 15 models.
  Surge AI frontier mathematics benchmark with advanced research-style problems sourced from mathematicians and scored by solution correctness.
SWE-bench Pro (Anthropic Scaffold) (Pass@1 (%)): Claude Mythos 5 leads with 80.3 across 6 models.
  Anthropic system-card run of SWE-bench Pro, measuring pass@1 on production software engineering issues using Anthropic scaffold settings.
OfficeQA Pro (Correctness (%)): Claude Fable 5 leads with 57.9 across 4 models.
  Hard OfficeQA subset for frontier document agents, requiring grounded search and numerical reasoning over U.S. Treasury Bulletin documents.
Real-World Finance v2 (Elo): Claude Fable 5 leads with 1374.0 across 4 models.
  Anthropic long-horizon finance workflow evaluation using pairwise preference grading and Elo ratings over realistic professional deliverables.
Real-World Finance v1 (Score (%)): Claude Mythos Preview leads with 70.9 across 4 models.
  Anthropic curated finance benchmark of 53 tasks evaluated against reference answers with a model-based grader.
Legal Agent Benchmark (Harvey Held-Out) (All-Pass Rate (%)): Claude Fable 5 leads with 13.3 across 5 models.
  Harvey legal-agent held-out evaluation using closed-universe matter files and expert rubrics, scored by all-pass task success.
Toolathlon (Anthropic Internal Harness) (Pass@1 (%)): Claude Fable 5 leads with 61.7 across 7 models.
  Anthropic internal Toolathlon harness over 108 tool-use tasks, reporting pass@1 for agentic workflow completion.
SWE-bench Verified (Anthropic Scaffold) (Resolved (%)): Claude Opus 4.8 leads with 88.6 across 3 models.
  Anthropic system-card run of SWE-bench Verified, measuring real GitHub issue resolution with Anthropic scaffold settings.
SWE-bench Multilingual (Anthropic Scaffold) (Resolved (%)): Claude Opus 4.8 leads with 84.4 across 2 models.
  Anthropic system-card run of SWE-bench Multilingual, measuring multilingual software issue resolution with Anthropic scaffold settings.
SWE-bench Multimodal (Anthropic Internal Harness) (Resolved (%)): Claude Opus 4.8 leads with 38.4 across 2 models.
  Anthropic internal multimodal SWE-bench harness, measuring software issue resolution that requires visual or multimodal context.
Humanity's Last Exam (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 49.8 across 4 models.
  Anthropic system-card run of Humanitys Last Exam without tools, covering expert-level academic reasoning across many domains.
Humanity's Last Exam (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 57.9 across 4 models.
  Anthropic system-card run of Humanitys Last Exam with tools, covering expert-level academic reasoning across many domains.
ChartQAPro (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 69.4 across 2 models.
  Anthropic no-tool run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ChartQAPro (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 72.3 across 2 models.
  Anthropic tool-enabled run of ChartQAPro, testing chart understanding and quantitative visual reasoning.
ScreenSpot-Pro (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 82.3 across 2 models.
  Anthropic no-tool run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
ScreenSpot-Pro (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 87.9 across 2 models.
  Anthropic tool-enabled run of ScreenSpot-Pro, evaluating GUI grounding and screen element localization.
GraphWalks BFS 256K (Anthropic) (F1 Score (%)): Claude Opus 4.8 leads with 85.9 across 4 models.
  Anthropic GraphWalks long-context graph traversal evaluation using breadth-first-search tasks at 256K context.
GraphWalks Parents 256K (Anthropic) (F1 Score (%)): Claude Opus 4.8 leads with 99.3 across 4 models.
  Anthropic GraphWalks long-context graph traversal evaluation using parent-pointer recovery tasks at 256K context.
USAMO 2026 (Anthropic) (Accuracy (%)): Claude Opus 4.8 leads with 96.7 across 2 models.
  Anthropic system-card evaluation on 2026 USAMO-style olympiad math problems, scored by answer correctness.
ArXivMath Mar-Apr 2026 (Anthropic) (Accuracy (%)): Claude Opus 4.8 leads with 71.82 across 3 models.
  Anthropic system-card evaluation on recent arXiv mathematics problems from March and April 2026.
OfficeQA (Anthropic Internal Harness) (Exact Match (%)): Claude Opus 4.8 leads with 77.6 across 2 models.
  Anthropic internal OfficeQA document-agent benchmark, requiring grounded search and numerical reasoning over office documents.
OfficeQA Pro (Anthropic Internal Harness) (Exact Match (%)): Claude Opus 4.8 leads with 66.2 across 2 models.
  Anthropic internal OfficeQA Pro hard subset, requiring grounded search and numerical reasoning over office documents.
ChartMuseum (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 75.8 across 2 models.
  Anthropic no-tool run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
ChartMuseum (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 89.7 across 2 models.
  Anthropic tool-enabled run of ChartMuseum, evaluating visual chart interpretation across diverse chart types.
LAB-Bench FigQA (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.8 leads with 80.4 across 2 models.
  Anthropic no-tool run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
LAB-Bench FigQA (Anthropic Tools) (Accuracy (%)): Claude Opus 4.8 leads with 87.3 across 2 models.
  Anthropic tool-enabled run of LAB-Bench FigQA, testing scientific figure understanding and reasoning.
CharXiv Reasoning (Anthropic No Tools) (Accuracy (%)): Claude Opus 4.7 leads with 81.3 across 2 models.
  Anthropic no-tool run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
CharXiv Reasoning (Anthropic Tools) (Accuracy (%)): Claude Opus 4.7 leads with 90.1 across 2 models.
  Anthropic tool-enabled run of CharXiv Reasoning, evaluating reasoning over scientific charts from arXiv papers.
HealthBench Professional (Anthropic) (Length-Adjusted Score (%)): Claude Opus 4.8 leads with 55.8 across 3 models.
  Anthropic system-card run of HealthBench Professional, measuring clinical and healthcare reasoning with length-adjusted scoring.
GMMLU (Anthropic) (Average Accuracy (%)): Gemini 3.1 Pro leads with 92.2 across 5 models.
  Anthropic system-card run of Global MMLU, measuring multilingual academic and professional knowledge.
BioPipelineBench Verified (Anthropic) (Score (%)): Claude Mythos Preview leads with 88.1 across 4 models.
  Anthropic system-card run of BioPipelineBench Verified, measuring biological data-analysis workflow completion.
BioMysteryBench Verified - Human Solvable (Anthropic) (Score (%)): Claude Mythos Preview leads with 82.6 across 4 models.
  Anthropic system-card run of BioMysteryBench Verified human-solvable tasks, testing biological mystery problem solving.
BioMysteryBench Verified - Human Difficult (Anthropic) (Score (%)): Claude Opus 4.8 leads with 40.0 across 4 models.
  Anthropic system-card run of BioMysteryBench Verified human-difficult tasks, testing hard biological mystery problem solving.
LatchBio SpatialBench (Anthropic) (Score (%)): Claude Mythos Preview leads with 53.8 across 4 models.
  Anthropic system-card run of LatchBio SpatialBench, measuring spatial transcriptomics analysis workflows.
LatchBio SingleCellBench (Anthropic) (Score (%)): Claude Opus 4.8 leads with 58.2 across 4 models.
  Anthropic system-card run of LatchBio SingleCellBench, measuring single-cell RNA-seq analysis workflows.
Structural Biology (Anthropic) (Score (%)): Claude Mythos Preview leads with 81.6 across 4 models.
  Anthropic system-card structural biology evaluation, testing biomolecular structure reasoning and analysis.
ProteinGym Hard (Anthropic) (Rank Correlation (%)): Claude Mythos Preview leads with 43.1 across 4 models.
  Anthropic system-card run of the hard ProteinGym subset, measuring protein variant effect prediction via rank correlation.
Organic Chemistry (Anthropic) (Score (%)): Claude Mythos Preview leads with 86.5 across 4 models.
  Anthropic system-card organic chemistry evaluation, testing reaction and molecule reasoning.
Protocol Troubleshooting (Anthropic) (Score (%)): Claude Mythos Preview leads with 69.6 across 4 models.
  Anthropic system-card protocol troubleshooting benchmark, testing diagnosis of laboratory protocol failures.
LABBench2 - Patent Questions (Anthropic) (Score (%)): Claude Opus 4.8 leads with 68.8 across 3 models.
  Anthropic system-card LABBench2 patent-question subset, testing life-science document reasoning over patent material.
LABBench2 - Clinical Trial Questions (Anthropic) (Score (%)): Claude Mythos Preview leads with 86.3 across 3 models.
  Anthropic system-card LABBench2 clinical-trial subset, testing life-science reasoning over trial documents.
LABBench2 - Table Reading (Anthropic) (Score (%)): Claude Opus 4.8 leads with 77.2 across 2 models.
  Anthropic system-card LABBench2 table-reading subset, testing scientific table comprehension.
LABBench2 - Supplementary Materials (Anthropic) (Score (%)): Claude Opus 4.8 leads with 58.9 across 2 models.
  Anthropic system-card LABBench2 supplementary-materials subset, testing reasoning over scientific supporting files.
BoxPwnr CTF Bench (Average Platform Completion (%)): z-ai/glm-5.1 leads with 54.37 across 15 models.
  Aggregated BoxPwnr trace leaderboard over public CTF and security-lab platforms including CyBench, Hack The Box, picoCTF, PortSwigger, TryHackMe, Argus, and XBOW.
Agent Security League - Functional Correctness (Functional Correctness (%)): GPT-5.5 leads with 84.9 across 15 models.
  Endor Labs coding-agent benchmark measuring whether agents functionally complete security-sensitive software tasks.
Agent Security League - Security Correctness (Security Correctness (%)): GPT-5.5 leads with 24.0 across 15 models.
  Endor Labs coding-agent benchmark measuring whether completed software tasks avoid introducing or preserving security vulnerabilities.

New Models (67)

Claude Fable 5 — ELO 2697, #4/815, above GPT-5.5 Pro, below GPT-5.4 Pro
Lynchmark: 100.0 (#1/13)
Design Arena (Website): 1345.0 (#1/143)
Design Arena (Game Dev): 1382.0 (#1/129)
Design Arena (UI Components): 1417.0 (#1/123)
Design Arena (Data Viz): 1381.0 (#1/125)
Design Arena (3D): 1370.0 (#1/117)
Design Arena (SVG): 1370.0 (#1/94)
Chatbot Arena (Text): 1510.0 (#1/366)
Chatbot Arena (Code): 1665.0 (#1/86)
Blueprint-Bench 2: 0.386 (#1/14)

Claude Opus 4.8 — ELO 2449, #6/815, above GPT-5.4 Pro, below GPT-5.5
Evals for Every Language - MGSM: 96.62 (#1/70)
Evals for Every Language - Language ar: 71.58 (#1/71)
Evals for Every Language - Language be: 69.43 (#1/71)
Evals for Every Language - Language ak: 60.02 (#2/71)
Evals for Every Language - Language bem: 60.25 (#2/71)
Evals for Every Language - Language bm: 59.47 (#2/71)
Evals for Every Language - Language chm: 63.17 (#2/71)
Evals for Every Language - Language ckb: 71.59 (#2/71)
Evals for Every Language - Language crh: 69.2 (#2/71)
Evals for Every Language - Language en: 86.15 (#2/71)

GPT-5.5 — ELO 2384, #7/815, above Claude Opus 4.8, below Qwen 3.7 Max
Blueprint-Bench 2: 0.362 (#2/14)
GRAB-Lite: 71.8 (#2/38)
Evals for Every Language - Language ary: 47.34 (#2/71)
Evals for Every Language - Language doi: 71.32 (#2/71)
Evals for Every Language - Language et: 72.25 (#3/71)
Evals for Every Language - ARC: 97.82 (#4/69)
Evals for Every Language - Language ay: 59.02 (#4/71)
Evals for Every Language - Language az: 65.39 (#4/71)
Evals for Every Language - Language bho: 67.61 (#4/71)
Evals for Every Language - Language bm: 54.72 (#4/71)

Qwen 3.7 Max — ELO 2370, #8/815, above GPT-5.5, below Gemini 3.1 Pro (Preview)
Position Bias (Lechmazur): 34.8 (#10/36)
RuneBench: 2222.0 (#11/23)
Wolfram LLM Benchmarking Project: 67.5 (#14/483)

Claude Opus 4.7 — ELO 2325, #10/815, above Gemini 3.1 Pro (Preview), below GPT-5.3 Codex
Evals for Every Language - Language chm: 63.6 (#1/71)
Evals for Every Language - Language cs: 74.38 (#1/71)
Evals for Every Language - Language doi: 71.84 (#1/71)
Evals for Every Language: 66.95 (#2/71)
Evals for Every Language - MGSM: 95.57 (#2/70)
Evals for Every Language - Language am: 67.86 (#2/71)
Evals for Every Language - Language ar: 70.69 (#2/71)
Evals for Every Language - Language arz: 52.06 (#2/71)
Evals for Every Language - Language as: 68.11 (#2/71)
Evals for Every Language - Language awa: 68.23 (#2/71)

Nemotron 3 Ultra — ELO 2288, #13/815, above Gemini 3 Deep Think, below Muse Spark
YC-Bench: 326.9 (#18/26)
SimpleBench: 41.7 (#37/74)

Claude Opus 4.6 — ELO 2253, #15/815, above Muse Spark, below GPT-5.4
Android Bench: 66.6 (#5/23)

GPT-5.4 — ELO 2242, #16/815, above Claude Opus 4.6, below Gemini 3 Pro
Blueprint-Bench 2: 0.271 (#4/14)

Gemini 3.5 Flash — ELO 2219, #18/815, above Gemini 3 Pro, below GPT-5 Pro
ZeroBench: 19.0 (#4/60)
GRAB-Lite: 63.0 (#4/38)
Position Bias (Lechmazur): 29.8 (#5/36)
Android Bench: 63.7 (#6/23)
YC-Bench: 987.0 (#12/26)
SWE-rebench: 49.45 (#30/85)

GPT-5 Pro — ELO 2217, #19/815, above Gemini 3.5 Flash, below GPT-5
Epoch AI - ECI: 149.85 (#69/374)

Qwen Max — ELO 2148, #23/815, above GPT-5.2 Pro, below O3 Pro
SimpleQA Verified: 58.52 (#10/55)
OTIS Mock AIME 2024-25: 95.0 (#13/145)
Chess Puzzles (Epoch AI): 22.0 (#22/46)

DeepSeek V4 Pro — ELO 2097, #30/815, above GPT-5.2 Codex, below Qwen 3.6 Plus
RuneBench: 2939.0 (#6/23)
ProphetArena: 0.9061 (#15/46)
Position Bias (Lechmazur): 43.6 (#19/36)

Qwen 3.6 Plus — ELO 2092, #31/815, above DeepSeek V4 Pro, below Gemini 2.5 Pro (March 2025)
ProphetArena: 0.9289 (#3/46)
Evals for Every Language - Language as: 66.13 (#6/71)
Evals for Every Language - Language bm: 50.12 (#6/71)
Evals for Every Language - Language chm: 59.82 (#6/71)
Evals for Every Language - Language ckb: 68.26 (#6/71)
Evals for Every Language - Language ace: 65.27 (#7/71)
Evals for Every Language - Language cv: 61.59 (#7/71)
Evals for Every Language - Language be: 65.74 (#8/71)
Evals for Every Language - Language bjn: 45.34 (#8/71)
Evals for Every Language - Language ban: 62.52 (#9/71)

MiMo-V2.5-Pro — ELO 2059, #36/815, above Gemini 3 Flash (Preview), below MiniMax-M3
LLM Stats (CMMLU): 90.2 (#1/6)
LLM Stats (DROP): 86.3 (#3/29)
LLM Stats (TriviaQA): 81.3 (#3/18)
LLM Stats (C-Eval): 91.5 (#5/18)
LLM Stats (Claw-Eval): 64.0 (#5/11)
LLM Stats (GDPval-AA): 1581.0 (#6/13)
Vals AI ProofBench: 24.0 (#13/42)
LLM Stats (MMLU-Redux): 92.8 (#14/47)
Vals AI MedScribe: 83.73 (#14/64)
Vals AI (Vals Index): 50.74 (#16/29)

MiniMax-M3 — ELO 2054, #37/815, above MiMo-V2.5-Pro, below Gemini 3 Flash
OSWorld: 75.19 (#6/61)
WebDev Arena: 1527.75 (#9/70)
YC-Bench: 999.5 (#11/26)
Position Bias (Lechmazur): 34.9 (#11/36)
Sycophancy (Lechmazur): 3.5 (#12/32)
Design Arena (SVG): 1255.0 (#18/94)
Design Arena (Game Dev): 1273.0 (#27/129)
SWE-rebench: 45.64 (#38/85)

O3 — ELO 2049, #39/815, above Gemini 3 Flash, below Qwen 3 Max
GRAB-Lite: 40.8 (#21/38)

Kimi K2.6 — ELO 2048, #41/815, above Qwen 3 Max, below GPT-5.1
RuneBench: 1256.0 (#16/23)
Position Bias (Lechmazur): 47.3 (#24/36)

GPT-5.1 — ELO 2045, #42/815, above Kimi K2.6, below Grok 4 (0709)
GRAB-Lite: 44.4 (#17/38)

kimi-k2.7-code — ELO 2040, #45/815, above Claude Opus 4.5 (20251101), below Grok 0.1
LiveBench Python: 90.0 (#2/125)
LiveBench TypeScript: 65.0 (#3/124)
OTIS Mock AIME 2024-25: 96.39 (#6/145)
Design Arena (Website): 1322.0 (#7/143)
Design Arena (3D): 1328.0 (#11/117)
LiveBench Logic With Navigation: 74.0 (#14/125)
LiveBench Zebra Puzzle: 96.0 (#15/124)
LiveBench Olympiad: 90.3 (#17/125)
Vals AI Vibe Code Bench: 47.21 (#18/62)
LiveBench JavaScript: 55.0 (#19/125)

Claude Sonnet 4.6 — ELO 2023, #50/815, above Qwen Max (Preview), below DeepSeek V4 Flash
ZeroBench: 11.0 (#11/60)
SWE-rebench: 54.49 (#18/85)
Terminal-Bench 2.0: 53.4 (#21/58)

GLM-5.1 — ELO 2004, #55/815, above Seed 2.0 Pro, below Kimi K2.5
ProphetArena: 0.9253 (#4/46)
FrontierSWE: 32.0 (#9/13)

Grok 4.3 — ELO 1973, #64/815, above Qwen 3 VL 235B A22B Instruct, below Claude Sonnet 4.5 (20250929)
ProphetArena: 0.9188 (#6/46)

Step 3.7 Flash — ELO 1962, #71/815, above Kimi K2 Turbo, below Qwen 3.7 Plus
Design Arena (Game Dev): 1216.0 (#54/129)

Qwen 3.7 Plus — ELO 1960, #72/815, above Step 3.7 Flash, below Qwen 3.5 397B A17B
Sycophancy (Lechmazur): 5.0 (#18/32)

Qwen 3.5 Plus — ELO 1951, #77/815, above O1, below GPT-4o (May 2024)
Epoch AI - Apex Agents: 13.6 (#29/46)

Grok 4.20 — ELO 1936, #82/815, above DeepSeek V3.2 Speciale, below Claude Opus 4.1
Evals for Every Language - Language fa: 70.2 (#5/71)
Evals for Every Language - MGSM: 87.39 (#7/70)
Evals for Every Language - Language ak: 56.11 (#7/71)
Evals for Every Language - Language cy: 77.85 (#7/71)
Evals for Every Language - Language en: 84.29 (#7/71)
Evals for Every Language - Language am: 64.62 (#8/71)
Evals for Every Language - Language ba: 66.75 (#8/71)
Evals for Every Language - Language ceb: 74.99 (#8/71)
Evals for Every Language - Language es: 72.75 (#8/71)
Evals for Every Language - Language ar: 67.76 (#9/71)

GPT-5.4 Mini — ELO 1912, #91/815, above Step 3.5 Flash, below Claude Opus 4
ZeroBench: 10.0 (#13/60)

Claude Sonnet 4 (20250514) — ELO 1909, #95/815, above Gemma 4 31B (IT), below GPT-5 ChatGPT
Epoch AI - Apex Agents: 9.3 (#33/46)

Gemini 3.1 Flash Lite — ELO 1905, #97/815, above GPT-5 ChatGPT, below Intern-S1
Evals for Every Language - Language am: 68.6 (#1/71)
Evals for Every Language - Language ca: 76.29 (#1/71)
Evals for Every Language - Language ceb: 78.06 (#1/71)
Evals for Every Language - Language cy: 82.03 (#1/71)
Evals for Every Language - Language el: 73.81 (#1/71)
Evals for Every Language - Language en: 87.28 (#1/71)
Evals for Every Language - Language es: 76.16 (#1/71)
Evals for Every Language - Language aeb: 53.18 (#2/71)
Evals for Every Language - Language az: 67.76 (#2/71)
Evals for Every Language - Language eo: 76.43 (#2/71)

Qwen 3.5 122B A10B — ELO 1903, #101/815, above MamayLM-Gemma-3-12B-IT-v1.0, below MiMo-V2.5
LIBRA - ruSciPassageCount *: 21.38 (#3/13)
LIBRA - ruBABILongQA1: 66.8 (#3/13)
LIBRA - ruBABILongQA2: 53.71 (#3/13)
LIBRA - ruBABILongQA3 *: 31.85 (#3/13)
LIBRA - MatreshkaNames *: 67.39 (#4/13)
LIBRA - LibrusecHistory: 79.77 (#4/13)
LIBRA - ru2WikiMultihopQA *: 55.3 (#4/13)
LIBRA - ruSciFi: 50.29 (#4/13)
LIBRA - LibrusecMHQA *: 42.32 (#4/13)
LIBRA - ruBABILongQA4: 58.91 (#4/13)

MiMo-V2.5 — ELO 1903, #102/815, above Qwen 3.5 122B A10B, below Kimi K2
LLM Stats (Video-MME): 87.7 (#1/14)
LLM Stats (Claw-Eval): 63.2 (#6/11)
LLM Stats (CharXiv-R): 81.0 (#12/38)
Vals AI Multimodal Index: 52.77 (#12/21)
Vals AI (Vals Index): 51.57 (#15/29)
Vals AI Vibe Code Bench: 42.17 (#21/62)
Vals AI ProofBench: 16.0 (#22/42)
Vals AI SAGE: 43.27 (#26/61)
Vals AI MortgageTax: 59.26 (#49/80)
Vals AI MedScribe: 72.15 (#50/64)

qwen3.6-flash — ELO 1872, #116/815, above GPT-5.1 Codex Mini, below Mercury 2
Evals for Every Language - Language chm: 55.74 (#12/71)
Evals for Every Language - Language am: 57.31 (#19/71)
Evals for Every Language - Language ban: 58.91 (#19/71)
Evals for Every Language - ARC: 91.99 (#20/69)
Evals for Every Language - Language ckb: 62.38 (#20/71)
Evals for Every Language - Language dz: 45.14 (#20/71)
Evals for Every Language - Language en: 79.89 (#20/71)
Evals for Every Language - Language ace: 57.48 (#21/71)
Evals for Every Language - Language cv: 53.08 (#21/71)
Evals for Every Language - Language ee: 41.46 (#21/71)

MiniMax-M2.7 — ELO 1853, #124/815, above Gemma 4 26B A4B (IT), below Ring-2.6-1T
ProphetArena: 0.9215 (#5/46)

O3 Mini — ELO 1850, #127/815, above Llama 3.1 70B, below free-evo-qwen72B-v0.8-re
FinBen - FNS: 16.95 (#4/21)
FinBen - FinNum: 20.98 (#5/21)

nemotron-3-ultra-550B-a55B — ELO 1778, #168/815, above GPT-4 Turbo, below Qwen 3 235B A22B FP8
Vals AI ProofBench: 2.0 (#40/42)
Vals AI Vibe Code Bench: 7.64 (#49/62)
WeirdML: 43.45 (#63/131)
Design Arena (Website): 1144.0 (#97/143)

DeepSeek V3.1 — ELO 1763, #176/815, above Awqward2.5-32B-Instruct, below O1 Mini
Evals for Every Language - Language da: 76.78 (#2/71)
Evals for Every Language - Language ban: 65.1 (#4/71)
Evals for Every Language - ARC: 97.4 (#5/69)
Evals for Every Language - Language ay: 58.91 (#5/71)
Evals for Every Language - Language ar: 68.89 (#6/71)
Evals for Every Language - Language ca: 73.25 (#6/71)
Evals for Every Language - Language bem: 54.51 (#7/71)
Evals for Every Language - MMLU: 97.67 (#8/69)
Evals for Every Language - Language el: 70.9 (#10/71)
Evals for Every Language - Language as: 64.63 (#11/71)

GPT-4o — ELO 1712, #208/815, above Falcon-H1R-7B, below Mistral Medium 3.5
FinBen (Financial LLM): 46.01 (#1/20)
FinBen - QA: 78.22 (#1/20)
FinBen - FNS: 25.5 (#3/21)
FinBen - MultiFin: 59.26 (#4/20)
FinBen - FinNum: 9.18 (#6/21)

Mistral Medium 3.5 — ELO 1712, #209/815, above GPT-4o, below Gemini 2.0 Flash
Position Bias (Lechmazur): 72.5 (#36/36)

Mistral-Small-3.2-24B-Instruct-2506 — ELO 1708, #211/815, above Gemini 2.0 Flash, below Llama 3.1 405B Instruct FP8
Evals for Every Language - Classification: 89.59 (#24/70)
Evals for Every Language - Language en: 76.19 (#29/71)
Evals for Every Language - Language ars: 46.69 (#31/71)
Evals for Every Language - Language awa: 61.09 (#31/71)
Evals for Every Language - Language ca: 69.44 (#31/71)
Evals for Every Language - Language be: 62.6 (#32/71)
Evals for Every Language - Language cs: 66.53 (#32/71)
Evals for Every Language - Language doi: 55.87 (#36/71)
Evals for Every Language - Language eu: 59.54 (#37/71)
Evals for Every Language - Language az: 58.06 (#39/71)

Qwen 3.5 35B A3B — ELO 1707, #213/815, above Llama 3.1 405B Instruct FP8, below Qwen 2.5 72B Instruct
LIBRA - MatreshkaNames *: 68.97 (#2/13)
LIBRA - ruSciPassageCount *: 21.89 (#2/13)
LIBRA - ruSciFi: 51.47 (#2/13)
LIBRA - ruBABILongQA1: 68.38 (#2/13)
LIBRA - ruBABILongQA2: 54.97 (#2/13)
LIBRA - ruBABILongQA3 *: 32.6 (#2/13)
LIBRA - LibrusecHistory: 81.65 (#3/13)
LIBRA - ru2WikiMultihopQA *: 56.6 (#3/13)
LIBRA - LibrusecMHQA *: 43.32 (#3/13)
LIBRA - ruBABILongQA4: 60.29 (#3/13)

DeepSeek V3 — ELO 1706, #215/815, above Qwen 2.5 72B Instruct, below GPT-4.1 Mini
FinBen - FNS: 37.72 (#1/21)
FinBen - MultiFin: 61.11 (#3/20)
FinBen - FinNum: 7.43 (#7/21)
FinBen - QA: 50.0 (#7/20)
FinBen (Financial LLM): 10.2 (#13/20)

GPT-4.1 Mini — ELO 1705, #216/815, above DeepSeek V3, below Qwen2.5-32B-Instruct-CFT
GRAB-Lite: 18.6 (#32/38)

GPT-4o (2024-11-20) — ELO 1696, #224/815, above Qwen 3 VL 235B A22B, below GPT-4o (2024-08-06)
Epoch AI - Apex Agents: 1.1 (#46/46)

GLM 4.5 Air — ELO 1684, #230/815, above Qwentile2.5-32B-Instruct, below Qwen 3 VL 32B Instruct
Evals for Every Language - Language chm: 47.52 (#21/71)
Evals for Every Language - Language et: 66.43 (#21/71)
Evals for Every Language - Language ckb: 60.22 (#22/71)
Evals for Every Language - Language as: 60.21 (#23/71)
Evals for Every Language - Language az: 62.15 (#24/71)
Evals for Every Language - Language ak: 41.24 (#26/71)
Evals for Every Language - Language es: 70.3 (#26/71)
Evals for Every Language - Language ca: 70.13 (#27/71)
Evals for Every Language - Language bho: 62.66 (#28/71)
Evals for Every Language - Language ace: 51.96 (#29/71)

Hermes 4 70B — ELO 1674, #239/815, above Gemini 1.5 Pro (001), below Grok 2
Evals for Every Language - MGSM: 77.91 (#24/70)
Evals for Every Language - MMLU: 88.52 (#26/69)
Evals for Every Language - ARC: 83.16 (#38/69)
Evals for Every Language - Language chm: 34.57 (#40/71)
Evals for Every Language - Language dz: 28.93 (#40/71)
Evals for Every Language - Language cv: 31.63 (#44/71)
Evals for Every Language - Language am: 31.51 (#49/71)
Evals for Every Language - Language ckb: 41.14 (#49/71)
Evals for Every Language - Language as: 40.36 (#57/71)
Evals for Every Language - Language ba: 41.82 (#58/71)

jamba-large-1.7 — ELO 1663, #245/815, above Qwen 2.5 32B Instruct, below gemma-3-27B-pt
Evals for Every Language - Classification: 91.29 (#18/70)
Evals for Every Language - Language af: 71.78 (#24/71)
Evals for Every Language - Language fa: 65.89 (#24/71)
Evals for Every Language - Language bg: 70.55 (#25/71)
Evals for Every Language - Language ee: 30.99 (#26/71)
Evals for Every Language - Language ar: 63.0 (#27/71)
Evals for Every Language - Language be: 62.93 (#28/71)
Evals for Every Language - Language de: 70.42 (#30/71)
Evals for Every Language - Language aeb: 42.65 (#32/71)
Evals for Every Language - Language doi: 56.88 (#33/71)

Llama 3.1 70B Instruct — ELO 1658, #251/815, above GPT-4, below Qwen 3 Coder 480B A35B
FinBen - FinNum: 46.34 (#3/21)
FinBen - QA: 64.44 (#3/20)
FinBen - FNS: 13.61 (#7/21)
FinBen - MultiFin: 50.0 (#7/20)
FinBen (Financial LLM): 14.07 (#8/20)

Ministral 3 8B (2512) — ELO 1640, #263/815, above Ling-flash-2.0, below Mistral Large 3
Evals for Every Language - Language bm: 29.35 (#28/71)
Evals for Every Language - Classification: 84.43 (#39/70)
Evals for Every Language - Language cs: 65.2 (#41/71)
Evals for Every Language - Language en: 73.09 (#42/71)
Evals for Every Language - Language bn: 62.11 (#43/71)
Evals for Every Language - Language es: 67.86 (#44/71)
Evals for Every Language - Language el: 63.45 (#45/71)
Evals for Every Language - Language be: 59.13 (#46/71)
Evals for Every Language - Language ace: 43.57 (#47/71)
Evals for Every Language - Language chm: 31.67 (#47/71)

Gemma 3 27B (IT) — ELO 1639, #266/815, above trinity-large-preview, below Llama 3.1 405B
Evals for Every Language - Language el: 72.48 (#3/71)
FinBen (Financial LLM): 15.74 (#7/20)
FinBen - FinNum: 0.0 (#10/21)
FinBen - MultiFin: 38.89 (#10/20)
Evals for Every Language - Language eo: 73.18 (#10/71)
Evals for Every Language - Classification: 95.41 (#11/70)
Evals for Every Language - Language bg: 73.9 (#11/71)
Evals for Every Language - Language es: 72.28 (#11/71)
FinBen - QA: 22.67 (#13/20)
FinBen - FNS: 0.21 (#14/21)

nova-2-lite-v1 — ELO 1635, #268/815, above Llama 3.1 405B, below Llama 3.3 70B Instruct
Evals for Every Language - MMLU: 95.33 (#12/69)
Evals for Every Language - Language en: 81.54 (#12/71)
Evals for Every Language - Language be: 64.22 (#17/71)
Evals for Every Language - Language chm: 52.92 (#18/71)
Evals for Every Language - MGSM: 80.9 (#19/70)
Evals for Every Language - Language bn: 68.34 (#19/71)
Evals for Every Language - Language ak: 46.55 (#20/71)
Evals for Every Language - Language cv: 53.69 (#20/71)
Evals for Every Language - Language da: 71.55 (#21/71)
Evals for Every Language - Language bm: 36.11 (#22/71)

Qwen 3.5 9B — ELO 1628, #272/815, above Qwen 3 32B, below Gemini 2.5 Flash Lite
LIBRA - ruSciPassageCount *: 20.77 (#4/13)
LIBRA - ruBABILongQA1: 64.88 (#4/13)
LIBRA - ruBABILongQA2: 52.16 (#4/13)
LIBRA - ruBABILongQA3 *: 30.94 (#4/13)
LIBRA - MatreshkaNames *: 65.44 (#5/13)
LIBRA - LibrusecHistory: 77.47 (#5/13)
LIBRA - ru2WikiMultihopQA *: 53.7 (#5/13)
LIBRA - ruSciFi: 48.84 (#5/13)
LIBRA - LibrusecMHQA *: 41.1 (#5/13)
LIBRA - ruBABILongQA4: 57.21 (#5/13)

Qwen 3 30B A3B 2507 Instruct — ELO 1615, #280/815, above Gemma 3 12B (IT), below Llama 4 Maverick Instruct FP8
Evals for Every Language - Language ars: 50.46 (#10/71)
Evals for Every Language - Language aeb: 45.53 (#18/71)
Evals for Every Language - Language en: 78.55 (#23/71)
Evals for Every Language - Language bs: 68.53 (#30/71)
Evals for Every Language - Language bg: 69.16 (#32/71)
Evals for Every Language - Language arz: 42.54 (#33/71)
Evals for Every Language - Language dz: 32.48 (#33/71)
Evals for Every Language - Language am: 42.2 (#35/71)
Evals for Every Language - Language bn: 63.67 (#37/71)
Evals for Every Language - Language ace: 47.99 (#38/71)

Hunyuan A13B-Instruct — ELO 1579, #307/815, above Nova Premier, below Gemini 2.0 Flash (Preview)
Evals for Every Language - Language ars: 42.49 (#55/71)
Evals for Every Language - Language aeb: 35.99 (#56/71)
Evals for Every Language - Language apc: 40.5 (#56/71)
Evals for Every Language - Translation From: 22.42 (#57/71)
Evals for Every Language - Language ary: 33.19 (#57/71)
Evals for Every Language - Language ak: 25.6 (#58/71)
Evals for Every Language - Language cv: 24.99 (#58/71)
Evals for Every Language - Language arz: 36.21 (#59/71)
Evals for Every Language - Translation To: 18.34 (#60/71)
Evals for Every Language - Language bjn: 29.79 (#61/71)

GPT-4o Mini — ELO 1543, #345/815, above Granite 4.0 Micro, below QwQ-32B
GRAB-Lite: 11.4 (#38/38)

Ministral 3 14B (2512) — ELO 1532, #356/815, above DeepSeek R1 0528 Qwen3 8B, below Nanbeige4.1-3B
Evals for Every Language - Classification: 88.17 (#31/70)
Evals for Every Language - Language be: 62.67 (#31/71)
Evals for Every Language - Language el: 66.63 (#31/71)
Evals for Every Language - Language bn: 64.34 (#33/71)
Evals for Every Language - Language az: 59.29 (#34/71)
Evals for Every Language - Language af: 69.81 (#35/71)
Evals for Every Language - Language es: 68.9 (#37/71)
Evals for Every Language - Language en: 73.16 (#41/71)
Evals for Every Language - Language arz: 40.92 (#42/71)
Evals for Every Language - Language bg: 67.27 (#42/71)

GPT-OSS-20B — ELO 1515, #371/815, above Mistral-Small-Instruct-2409, below Gemma 3 12B
Evals for Every Language - Language en: 77.2 (#26/71)
Evals for Every Language - Language es: 69.42 (#30/71)
Evals for Every Language - Language awa: 60.89 (#34/71)
Evals for Every Language - Language bs: 67.0 (#37/71)
Evals for Every Language - Language da: 69.16 (#37/71)
Evals for Every Language - Language dz: 30.01 (#37/71)
Evals for Every Language - Language as: 54.72 (#38/71)
Evals for Every Language - Language bem: 33.09 (#39/71)
Evals for Every Language - Language ak: 34.16 (#40/71)
Evals for Every Language - Language cs: 65.28 (#40/71)

Llama 4 Scout Instruct — ELO 1498, #384/815, above Grok Beta, below Phi-4
FinBen - FinNum: 49.12 (#2/21)
FinBen - QA: 74.22 (#2/20)
FinBen (Financial LLM): 20.89 (#3/20)
FinBen - FNS: 16.9 (#5/21)
FinBen - MultiFin: 55.56 (#5/20)

Laguna M.1 — ELO 1491, #391/815, above Llama-3.1-Tulu-3-8B, below DeepSeek R1 Distill Llama 70B
Vals AI (Vals Index): 35.27 (#27/29)
Vals AI ProofBench: 0.0 (#42/42)
Vals AI Terminal-Bench 2.0: 31.46 (#43/68)
Vals AI Vibe Code Bench: 10.94 (#48/62)
Vals AI MedCode: 25.24 (#64/67)
Vals AI CorpFin v2: 58.16 (#68/115)
Vals AI LegalBench: 75.14 (#86/118)
Vals AI TaxEval v2: 1.64 (#121/121)

granite-4.0-h-micro — ELO 1486, #399/815, above Qwen 3 14B, below Ministral-3-3B-Reasoning-2512
Evals for Every Language - Classification: 86.11 (#36/70)
Evals for Every Language - Language ar: 60.19 (#45/71)
Evals for Every Language - Language cv: 27.5 (#52/71)
Evals for Every Language - Language ay: 29.15 (#54/71)
Evals for Every Language - Language bn: 50.72 (#55/71)
Evals for Every Language - Language ary: 33.31 (#56/71)
Evals for Every Language - Language bg: 58.7 (#57/71)
Evals for Every Language - Language eo: 59.1 (#57/71)
Evals for Every Language - Language ak: 25.2 (#59/71)
Evals for Every Language - Language da: 56.25 (#60/71)

Laguna XS.2 — ELO 1486, #401/815, above Ministral-3-3B-Reasoning-2512, below Mistral Small 3.2 (2506)
Vals AI (Vals Index): 29.15 (#28/29)
Vals AI ProofBench: 1.0 (#41/42)
Vals AI Terminal-Bench 2.0: 28.09 (#47/68)
Vals AI Vibe Code Bench: 3.84 (#53/62)
Vals AI MedCode: 20.7 (#66/67)
Vals AI CorpFin v2: 56.33 (#72/115)
Vals AI LegalBench: 71.03 (#91/118)
Vals AI TaxEval v2: 59.98 (#107/121)

Gemma 3 4B (IT) — ELO 1463, #424/815, above Nous-Hermes-2-Yi-34B, below Mistral Small 3
FinBen (Financial LLM): 12.74 (#9/20)
FinBen - FinNum: 0.0 (#9/21)
FinBen - MultiFin: 38.89 (#9/20)
FinBen - QA: 22.67 (#12/20)
FinBen - FNS: 0.24 (#13/21)

Phi-4 Mini Instruct — ELO 1451, #434/815, above Nanbeige2-16B-Chat, below Qwen 2.5 VL 32B Instruct
Evals for Every Language - Classification: 79.23 (#54/70)
Evals for Every Language - Language ckb: 31.9 (#56/71)
Evals for Every Language - Language aeb: 33.95 (#61/71)
Evals for Every Language - Language ee: 21.7 (#62/71)
Evals for Every Language - Language en: 62.96 (#62/71)
Evals for Every Language - Language es: 51.93 (#65/71)
Evals for Every Language - MGSM: 16.66 (#66/70)
Evals for Every Language - MMLU: 43.8 (#66/69)
Evals for Every Language - ARC: 41.91 (#67/69)
Evals for Every Language - Language doi: 30.09 (#67/71)

Qwen 3.5 4B — ELO 1430, #455/815, above Gemini 1.5 Flash (001), below Granite 4.0 H 1B
LIBRA - ruSciPassageCount *: 19.57 (#5/13)
LIBRA - ruBABILongQA1: 61.13 (#5/13)
LIBRA - ruBABILongQA2: 49.14 (#5/13)
LIBRA - ruBABILongQA3 *: 29.15 (#5/13)
LIBRA - MatreshkaNames *: 61.66 (#6/13)
LIBRA - LibrusecHistory: 72.99 (#6/13)
LIBRA - ru2WikiMultihopQA *: 50.6 (#6/13)
LIBRA - ruSciFi: 46.02 (#6/13)
LIBRA - LibrusecMHQA *: 38.73 (#6/13)
LIBRA - ruBABILongQA4: 53.9 (#6/13)

Qwen3.5 0.8B — ELO 1370, #528/815, above Infinity-Instruct-3M-0625-Yi-1.5-9B, below GLM-4 9B Chat
LIBRA - ruSciPassageCount *: 17.79 (#7/13)
LIBRA - ruBABILongQA2: 44.67 (#7/13)
LIBRA - MatreshkaNames *: 56.05 (#8/13)
LIBRA - ru2WikiMultihopQA *: 46.0 (#8/13)
LIBRA - LibrusecMHQA *: 35.21 (#8/13)
LIBRA - ruBABILongQA1: 55.57 (#8/13)
LIBRA - ruBABILongQA4: 49.0 (#8/13)
LIBRA - ruSciAbstractRetrieval: 56.26 (#9/13)
LIBRA - ruSciFi: 41.83 (#9/13)
LIBRA - ruBABILongQA3 *: 26.5 (#9/13)

Qwen 3.5 2B — ELO 1247, #653/815, above juud-Mistral-7B-dpo, below occiglot-7B-it-en-instruct
LIBRA - ruSciPassageCount *: 18.72 (#6/13)
LIBRA - ruBABILongQA2: 47.01 (#6/13)
LIBRA - ruBABILongQA3 *: 27.88 (#6/13)
LIBRA - MatreshkaNames *: 58.98 (#7/13)
LIBRA - ru2WikiMultihopQA *: 48.4 (#7/13)
LIBRA - ruSciFi: 44.02 (#7/13)
LIBRA - LibrusecMHQA *: 37.05 (#7/13)
LIBRA - ruBABILongQA1: 58.48 (#7/13)
LIBRA - ruBABILongQA4: 51.56 (#7/13)
LIBRA - LibrusecHistory: 69.83 (#8/13)

Qwen2.5-Omni-7B — ELO 1227, #667/815, above AFM-4.5B, below Qwen-14B-Chat
FinBen (Financial LLM): 33.53 (#2/20)
FinBen - FinNum: 0.4 (#8/21)
FinBen - QA: 48.89 (#8/20)
FinBen - FNS: 5.6 (#11/21)
FinBen - MultiFin: 38.89 (#11/20)

Gemma 4 12B — ELO 1100, #731/815, above Mixtral 8x7B, below tulu-2-dpo-7B
LLM Stats (MRCR v2): 43.4 (#3/7)
LLM Stats (FLEURS): 93.1 (#4/6)
LLM Stats (MedXpertQA): 48.7 (#8/12)
LLM Stats (MathVision): 79.7 (#9/28)
LLM Stats (AIME 2026): 77.5 (#13/16)
LLM Stats (OmniDocBench 1.5): 16.4 (#13/15)
LLM Stats (CodeForces): 55.3 (#15/16)
LLM Stats (MMMLU): 83.4 (#34/48)
ZeroEval GPQA Diamond: 78.8 (#82/223)

New Scores From Top-10 Models (186)

Claude Fable 5 on AI Chess Leaderboard (Continuation): 1092.0 Elo (#30/227)
Claude Fable 5 on AI Chess Leaderboard (Reasoning): 1711.0 Elo (#8/279)
Claude Fable 5 on Chatbot Arena (Document): 1495.0 Elo (#5/29)
Claude Fable 5 on Chatbot Arena (Vision): 1307.0 Arena Score (#2/131)
Claude Fable 5 on ClockBench: 35.0 Accuracy (%) (#4/27)
Claude Fable 5 on Epoch AI - Apex Agents: 45.0 Score (#3/46)
Claude Fable 5 on LLM Stats (GDPval-AA): 1932.0 ELO (#1/13)
Claude Fable 5 on Lynchmark: 100.0 Pass Rate (%) (#1/13)
Claude Fable 5 on MineBench: 1790.51 Elo Rating (#4/45)
Claude Fable 5 on PM-LLM-Benchmark: 35.6 Score (#13/146)
Claude Fable 5 on PinchBench: 59.61 Success Rate (%) (#44/51)
Claude Fable 5 on React Native Evals: 86.96 Overall Score (%) (#4/28)
Claude Fable 5 on SEAL - MCP Atlas: 83.3 Score (#2/23)
Claude Fable 5 on Vals AI MedCode: 56.07 Accuracy (%) (#2/67)
Claude Fable 5 on Vals AI MortgageTax: 68.92 Accuracy (%) (#5/80)
Claude Fable 5 on Vals AI SAGE: 51.89 Accuracy (%) (#5/61)
Claude Fable 5 on Vals AI TaxEval v2: 76.94 Accuracy (%) (#3/121)
Claude Fable 5 on Vellum - GPQA: 94.1 Accuracy (%) (#3/57)
Claude Fable 5 on Vellum - HumanEval: 95.0 Pass@1 (%) (#2/38)
Claude Fable 5 on Vending-Bench 2: 4529.94 Money Balance ($) (#18/46)
Claude Opus 4.7 on Android Bench: 68.7 Score (%) (#4/23)
Claude Opus 4.7 on Evals for Every Language: 66.95 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - ARC: 97.23 Average Score (%) (#6/69)
Claude Opus 4.7 on Evals for Every Language - Classification: 95.98 Average Score (%) (#7/70)
Claude Opus 4.7 on Evals for Every Language - Language ace: 69.04 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language aeb: 50.61 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language af: 76.97 Average Score (%) (#9/71)
Claude Opus 4.7 on Evals for Every Language - Language ak: 59.75 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language am: 67.86 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language apc: 55.53 Average Score (%) (#5/71)
Claude Opus 4.7 on Evals for Every Language - Language ar: 70.69 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language ars: 49.83 Average Score (%) (#13/71)
Claude Opus 4.7 on Evals for Every Language - Language ary: 44.23 Average Score (%) (#12/71)
Claude Opus 4.7 on Evals for Every Language - Language arz: 52.06 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language as: 68.11 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language awa: 68.23 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language ay: 59.38 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language az: 65.04 Average Score (%) (#8/71)
Claude Opus 4.7 on Evals for Every Language - Language ba: 67.46 Average Score (%) (#6/71)
Claude Opus 4.7 on Evals for Every Language - Language ban: 65.75 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language be: 66.48 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language bem: 59.05 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language bg: 74.44 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language bho: 67.27 Average Score (%) (#8/71)
Claude Opus 4.7 on Evals for Every Language - Language bjn: 48.88 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language bm: 58.41 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language bn: 72.35 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language bs: 70.22 Average Score (%) (#24/71)
Claude Opus 4.7 on Evals for Every Language - Language ca: 72.35 Average Score (%) (#14/71)
Claude Opus 4.7 on Evals for Every Language - Language ceb: 75.18 Average Score (%) (#6/71)
Claude Opus 4.7 on Evals for Every Language - Language ckb: 70.88 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language crh: 66.99 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language cs: 74.38 Average Score (%) (#1/71)
Claude Opus 4.7 on Evals for Every Language - Language cv: 62.92 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language cy: 79.87 Average Score (%) (#5/71)
Claude Opus 4.7 on Evals for Every Language - Language da: 74.47 Average Score (%) (#6/71)
Claude Opus 4.7 on Evals for Every Language - Language de: 75.66 Average Score (%) (#6/71)
Claude Opus 4.7 on Evals for Every Language - Language dz: 59.16 Average Score (%) (#3/71)
Claude Opus 4.7 on Evals for Every Language - Language ee: 60.86 Average Score (%) (#2/71)
Claude Opus 4.7 on Evals for Every Language - Language el: 71.56 Average Score (%) (#8/71)
Claude Opus 4.7 on Evals for Every Language - Language en: 84.79 Average Score (%) (#5/71)
Claude Opus 4.7 on Evals for Every Language - Language eo: 75.16 Average Score (%) (#4/71)
Claude Opus 4.7 on Evals for Every Language - Language es: 70.89 Average Score (%) (#19/71)
Claude Opus 4.7 on Evals for Every Language - Language et: 71.59 Average Score (%) (#5/71)
Claude Opus 4.7 on Evals for Every Language - Language eu: 68.46 Average Score (%) (#7/71)
Claude Opus 4.7 on Evals for Every Language - Language fa: 69.71 Average Score (%) (#8/71)
Claude Opus 4.7 on Evals for Every Language - MGSM: 95.57 Average Score (%) (#2/70)
Claude Opus 4.7 on Evals for Every Language - MMLU: 95.33 Average Score (%) (#13/69)
Claude Opus 4.7 on Evals for Every Language - Translation From: 40.53 Average Score (%) (#7/71)
Claude Opus 4.7 on Evals for Every Language - Translation To: 39.5 Average Score (%) (#4/71)
Claude Opus 4.7 on GRAB-Lite: 58.2 Overall Score (#10/38)
Claude Opus 4.8 on Chess Puzzles (Epoch AI): 34.0 Accuracy (%) (#13/46)
Claude Opus 4.8 on Design Arena (Game Dev): 1300.0 Elo (#17/129)
Claude Opus 4.8 on EQ-Bench Longform Writing: 80.8 Writing Score (0-100) (#3/113)
Claude Opus 4.8 on Epoch AI - Apex Agents: 42.5 Score (#4/46)
Claude Opus 4.8 on Epoch AI - ECI: 156.34 ECI Score (#14/374)
Claude Opus 4.8 on Evals for Every Language: 66.27 Average Score (%) (#3/71)
Claude Opus 4.8 on Evals for Every Language - ARC: 98.0 Average Score (%) (#3/69)
Claude Opus 4.8 on Evals for Every Language - Classification: 90.31 Average Score (%) (#21/70)
Claude Opus 4.8 on Evals for Every Language - Language ace: 66.63 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language aeb: 50.53 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language af: 78.38 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language ak: 60.02 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language am: 65.76 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language apc: 49.54 Average Score (%) (#21/71)
Claude Opus 4.8 on Evals for Every Language - Language ars: 47.35 Average Score (%) (#26/71)
Claude Opus 4.8 on Evals for Every Language - Language ary: 40.29 Average Score (%) (#25/71)
Claude Opus 4.8 on Evals for Every Language - Language arz: 49.71 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language as: 66.93 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language awa: 67.71 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language ay: 58.4 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language az: 65.38 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language ba: 67.66 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language ban: 63.8 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language bem: 60.25 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language bg: 74.44 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language bho: 67.32 Average Score (%) (#7/71)
Claude Opus 4.8 on Evals for Every Language - Language bjn: 47.35 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language bm: 59.47 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language bn: 70.4 Average Score (%) (#12/71)
Claude Opus 4.8 on Evals for Every Language - Language bs: 74.0 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language ca: 74.29 Average Score (%) (#3/71)
Claude Opus 4.8 on Evals for Every Language - Language ceb: 75.82 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language chm: 63.17 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language ckb: 71.59 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language crh: 69.2 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language cs: 73.8 Average Score (%) (#3/71)
Claude Opus 4.8 on Evals for Every Language - Language cv: 64.32 Average Score (%) (#3/71)
Claude Opus 4.8 on Evals for Every Language - Language cy: 79.83 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language da: 74.57 Average Score (%) (#5/71)
Claude Opus 4.8 on Evals for Every Language - Language de: 76.71 Average Score (%) (#3/71)
Claude Opus 4.8 on Evals for Every Language - Language doi: 70.16 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language dz: 58.51 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language ee: 57.06 Average Score (%) (#4/71)
Claude Opus 4.8 on Evals for Every Language - Language el: 70.34 Average Score (%) (#13/71)
Claude Opus 4.8 on Evals for Every Language - Language en: 86.15 Average Score (%) (#2/71)
Claude Opus 4.8 on Evals for Every Language - Language eo: 74.5 Average Score (%) (#6/71)
Claude Opus 4.8 on Evals for Every Language - Language es: 70.97 Average Score (%) (#18/71)
Claude Opus 4.8 on Evals for Every Language - Language et: 70.93 Average Score (%) (#7/71)
Claude Opus 4.8 on Evals for Every Language - Language eu: 66.0 Average Score (%) (#19/71)
Claude Opus 4.8 on Evals for Every Language - Language fa: 69.54 Average Score (%) (#9/71)
Claude Opus 4.8 on Evals for Every Language - MMLU: 98.31 Average Score (%) (#4/69)
Claude Opus 4.8 on Evals for Every Language - Translation From: 39.86 Average Score (%) (#9/71)
Claude Opus 4.8 on Evals for Every Language - Translation To: 38.22 Average Score (%) (#7/71)
Claude Opus 4.8 on GRAB-Lite: 60.6 Overall Score (#6/38)
Claude Opus 4.8 on OTIS Mock AIME 2024-25: 98.33 Accuracy (%) (#4/145)
Claude Opus 4.8 on SimpleQA Verified: 39.5 Accuracy (%) (#26/55)
Claude Opus 4.8 on WebDev Arena: 1545.05 Arena Score (#6/70)
Claude Opus 4.8 on Wolfram LLM Benchmarking Project: 65.9 Correct Functionality (%) (#18/483)
Claude Opus 4.8 on ZeroBench: 17.0 Score (%) (#7/60)
GPT-5.5 on Blueprint-Bench 2: 0.362 Connectivity Similarity Score (#2/14)
GPT-5.5 on Evals for Every Language: 65.09 Average Score (%) (#5/71)
GPT-5.5 on Evals for Every Language - ARC: 97.82 Average Score (%) (#4/69)
GPT-5.5 on Evals for Every Language - Classification: 82.73 Average Score (%) (#42/70)
GPT-5.5 on Evals for Every Language - Language ace: 67.32 Average Score (%) (#5/71)
GPT-5.5 on Evals for Every Language - Language aeb: 44.61 Average Score (%) (#22/71)
GPT-5.5 on Evals for Every Language - Language af: 77.33 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language ak: 57.86 Average Score (%) (#5/71)
GPT-5.5 on Evals for Every Language - Language am: 65.01 Average Score (%) (#6/71)
GPT-5.5 on Evals for Every Language - Language apc: 50.92 Average Score (%) (#12/71)
GPT-5.5 on Evals for Every Language - Language ar: 65.19 Average Score (%) (#18/71)
GPT-5.5 on Evals for Every Language - Language ars: 46.47 Average Score (%) (#33/71)
GPT-5.5 on Evals for Every Language - Language ary: 47.34 Average Score (%) (#2/71)
GPT-5.5 on Evals for Every Language - Language arz: 45.23 Average Score (%) (#19/71)
GPT-5.5 on Evals for Every Language - Language as: 66.04 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language awa: 66.14 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language ay: 59.02 Average Score (%) (#4/71)
GPT-5.5 on Evals for Every Language - Language az: 65.39 Average Score (%) (#4/71)
GPT-5.5 on Evals for Every Language - Language ba: 64.64 Average Score (%) (#14/71)
GPT-5.5 on Evals for Every Language - Language ban: 62.74 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language be: 64.63 Average Score (%) (#16/71)
GPT-5.5 on Evals for Every Language - Language bem: 53.46 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language bg: 71.22 Average Score (%) (#23/71)
GPT-5.5 on Evals for Every Language - Language bho: 67.61 Average Score (%) (#4/71)
GPT-5.5 on Evals for Every Language - Language bjn: 44.06 Average Score (%) (#12/71)
GPT-5.5 on Evals for Every Language - Language bm: 54.72 Average Score (%) (#4/71)
GPT-5.5 on Evals for Every Language - Language bn: 69.73 Average Score (%) (#14/71)
GPT-5.5 on Evals for Every Language - Language bs: 71.46 Average Score (%) (#13/71)
GPT-5.5 on Evals for Every Language - Language ca: 73.21 Average Score (%) (#7/71)
GPT-5.5 on Evals for Every Language - Language ceb: 74.54 Average Score (%) (#10/71)
GPT-5.5 on Evals for Every Language - Language chm: 58.46 Average Score (%) (#9/71)
GPT-5.5 on Evals for Every Language - Language ckb: 68.48 Average Score (%) (#5/71)
GPT-5.5 on Evals for Every Language - Language crh: 63.78 Average Score (%) (#15/71)
GPT-5.5 on Evals for Every Language - Language cs: 71.8 Average Score (%) (#10/71)
GPT-5.5 on Evals for Every Language - Language cv: 59.68 Average Score (%) (#10/71)
GPT-5.5 on Evals for Every Language - Language cy: 77.61 Average Score (%) (#8/71)
GPT-5.5 on Evals for Every Language - Language da: 71.48 Average Score (%) (#23/71)
GPT-5.5 on Evals for Every Language - Language de: 73.13 Average Score (%) (#20/71)
GPT-5.5 on Evals for Every Language - Language doi: 71.32 Average Score (%) (#2/71)
GPT-5.5 on Evals for Every Language - Language dz: 58.36 Average Score (%) (#6/71)
GPT-5.5 on Evals for Every Language - Language ee: 56.99 Average Score (%) (#5/71)
GPT-5.5 on Evals for Every Language - Language el: 71.64 Average Score (%) (#6/71)
GPT-5.5 on Evals for Every Language - Language en: 85.03 Average Score (%) (#4/71)
GPT-5.5 on Evals for Every Language - Language eo: 72.05 Average Score (%) (#13/71)
GPT-5.5 on Evals for Every Language - Language es: 70.48 Average Score (%) (#23/71)
GPT-5.5 on Evals for Every Language - Language et: 72.25 Average Score (%) (#3/71)
GPT-5.5 on Evals for Every Language - Language eu: 67.59 Average Score (%) (#11/71)
GPT-5.5 on Evals for Every Language - Language fa: 67.54 Average Score (%) (#12/71)
GPT-5.5 on Evals for Every Language - MGSM: 90.21 Average Score (%) (#5/70)
GPT-5.5 on Evals for Every Language - MMLU: 98.21 Average Score (%) (#5/69)
GPT-5.5 on Evals for Every Language - Translation From: 40.95 Average Score (%) (#6/71)
GPT-5.5 on Evals for Every Language - Translation To: 39.31 Average Score (%) (#5/71)
GPT-5.5 on GRAB-Lite: 71.8 Overall Score (#2/38)
Qwen 3.7 Max on Position Bias (Lechmazur): 34.8 Order Flip % (lower is better) (#10/36)
Qwen 3.7 Max on RuneBench: 2222.0 Total Peak XP Rate (XP/min) (#11/23)
Qwen 3.7 Max on Wolfram LLM Benchmarking Project: 67.5 Correct Functionality (%) (#14/483)

New #1 Leaders (92)

YC-Bench (Net Worth ($K)): Claude Fable 5 (1977.6) beat Claude Opus 4.7 (1714.5) by 263.1.
PACT (Lechmazur) (CMS Points): Claude Fable 5 (High) (2171.0) beat GPT-5.5 (High) (2016.0) by 155.0.
Chatbot Arena (Code) (Elo): Claude Fable 5 (1665.0) beat Claude Opus 4.7 (Thinking) (1567.0) by 98.0.
Chatbot Arena (Text-to-Video) (Elo): gemini-omni-flash (1527.0) beat dreamina-seedance-2.0-720p (1463.0) by 64.0.
Design Arena (UI Components) (Elo): Claude Fable 5 (1417.0) beat Claude Opus 4.7 (1360.0) by 57.0.
Multi-turn Debate (Lechmazur) (Bradley-Terry Rating): Claude Fable 5 (High) (1770.9) beat Claude Opus 4.7 (High) (1717.1) by 53.8.
AA GDPval (ELO): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (1932.47) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (1889.8) by 42.67.
Design Arena (Data Viz) (Elo): Claude Fable 5 (1381.0) beat Claude Opus 4.7 (Thinking) (1339.0) by 42.0.
Design Arena (Game Dev) (Elo): Claude Fable 5 (1382.0) beat GPT-5.5 (1355.0) by 27.0.
GSMA Open-Telco - TeleTables (Score (%)): TelecomGPT (88.0) beat OTel-LLM-8.3B-QnA (61.8) by 26.2.
LLM Stats (MCP-Mark) (Score (%)): Kimi K2.7 Code (81.1) beat Qwen 3.7 Max (60.8) by 20.3.
Design Arena (Image) (Elo): riverflow-2.5-pro (1419.0) beat gpt-image-2 (1402.0) by 17.0.
WDCD (DCD Score): Qwen 3 Max (84.38) beat Claude Opus 4.7 (70.0) by 14.38.
Evals for Every Language - Language ay (Average Score (%)): step-3.7-flash-20260528 (77.14) beat Gemini 3.1 Pro (Preview) (62.91) by 14.23.
SEAL - SWE Atlas - Test Writing (Score): Fable-5 (Claude Code) xHigh (58.52) beat GPT-5.4 (xHigh) (44.36) by 14.16.
LiveBench Python (Score): Claude Fable 5 (xHigh) (95.0) beat Claude Opus 4.5 (Thinking 64K, High) (2025-11-01) (85.0) by 10.0.
LLM Stats (FLEURS) (Score (%)): Qwen2.5-Omni-7B (95.9) beat Gemini 1.5 Flash-8B (86.4) by 9.5.
CursorBench 3.1 (Score (%)): Claude Fable 5 (Max) (72.9) beat Claude Opus 4.7 (64.8) by 8.1.
AA Omniscience - Software Engineering (SWE) - Dart (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.3 Codex (xHigh) (80.0) by 8.0.
AA Omniscience - Software Engineering (SWE) - R (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (82.0) beat GPT-5.5 (High) (74.0) by 8.0.
AA Omniscience - Software Engineering (SWE) - Swift (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (100.0) beat GPT-5.5 (xHigh) (92.0) by 8.0.
Vals AI Vibe Code Bench (Accuracy (%)): Claude Fable 5 (90.35) beat Claude Opus 4.8 (82.72) by 7.63.
AA Humanity's Last Exam (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.34) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (45.74) by 7.6.
AA Omniscience (Score): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (40.15) beat Gemini 3.1 Pro (Preview) (32.93) by 7.22.
FrontierSWE (Dominance (%)): Claude Fable 5 (90.0) beat Claude Opus 4.8 (83.0) by 7.0.
Vellum - HumanEval (Pass@1 (%)): Claude Mythos 5 (95.5) beat Claude Opus 4.8 (88.6) by 6.9.
Vellum - Humanity's Last Exam (Accuracy (%)): Claude Mythos 5 (64.5) beat Claude Opus 4.8 (57.9) by 6.6.
Evals for Every Language - Language crh (Average Score (%)): step-3.7-flash-20260528 (73.05) beat Gemini 3.1 Pro (Preview) (66.78) by 6.27.
Chatbot Arena (Text) (Elo): Claude Fable 5 (1510.0) beat Claude Opus 4.6 (Thinking) (1504.0) by 6.0.
AA Omniscience - Software Engineering (SWE) - Java (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (79.0) beat GPT-5.3 Codex (xHigh) (73.0) by 6.0.
Vals AI ProofBench (Accuracy (%)): Claude Fable 5 (77.0) beat aristotle (71.0) by 6.0.
AA Omniscience - Business (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.0) beat GPT-5.5 (xHigh) (49.1) by 5.9.
FinBen - MultiFin (Normalized Score): plutus-8B-instruct (72.22) beat Qwen 2.5 72B Instruct (66.67) by 5.55.
AA Omniscience - Science, Engineering & Mathematics (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.1) beat GPT-5.5 (High) (52.3) by 4.8.
Vals AI (Vals Index) (Accuracy (%)): Claude Fable 5 (75.14) beat Claude Opus 4.8 (70.36) by 4.78.
OpenClawProBench (Overall Score (%)): GLM-5.2 (81.3) beat intern-s2-preview (76.7) by 4.6.
Vals AI IOI (Accuracy (%)): Claude Fable 5 (72.25) beat GPT-5.4 (2026-03-05) (67.83) by 4.42.
AA Omniscience - Humanities & Social Sciences (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.9) beat Gemini 3 Pro (Preview) (High) (56.6) by 4.3.
Design Arena (Website) (Elo): Claude Fable 5 (1345.0) beat Claude Opus 4.6 (1341.0) by 4.0.
AA Omniscience - Software Engineering (SWE) - Go (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (88.0) beat GPT-5.5 (High) (84.0) by 4.0.
MathArena - ARXIV April (Accuracy (%)): Claude Fable 5 (Max) (70.73) beat GPT-5.5 (xHigh) (67.07) by 3.66.
GSMA Open-Telco LLM Leaderboard (Average Score (%)): TelecomGPT (89.64) beat OTel-LLM-8.3B-QnA (85.98) by 3.66.
FinBen - QA (Normalized Score): GPT-4o (78.22) beat GPT-4.5 (Preview) (74.67) by 3.55.
Artificial Analysis Intelligence Index (Intelligence Index): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.88) beat Claude Opus 4.8 (Adaptive Reasoning, Max Effort) (61.44) by 3.44.
Evals for Every Language - Language cv (Average Score (%)): gemma-4-31B-it-20260402 (69.3) beat Claude Opus 4.5 (65.91) by 3.39.
SEAL - SWE Atlas - Codebase QnA (Score): Opus 4.8 (Claude Code) (48.79) beat GPT-5.5 (45.43) by 3.36.
Vals AI CorpFin v2 (Accuracy (%)): Claude Fable 5 (71.83) beat Grok 4.3 (68.53) by 3.3.
Vals AI Multimodal Index (Accuracy (%)): Claude Fable 5 (74.15) beat Claude Opus 4.8 (70.89) by 3.26.
AA Omniscience - Software Engineering (SWE) (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (87.6) beat GPT-5.5 (xHigh) (84.4) by 3.2.
Design Arena (3D) (Elo): Claude Fable 5 (1370.0) beat Kimi K2.6 (1367.0) by 3.0.
GRAB-Lite (Overall Score): Claude Fable 5 (74.0) beat GPT-5.4 (71.0) by 3.0.
WeirdML (Average Score): Claude Fable 5 (High) (87.85) beat GPT-5.5 (xHigh) (84.91) by 2.94.
BIRD-SQL (Execution Accuracy (%)): Gemini-SQL2 (80.04) beat Gemini-SQL (Multitask SFT + Gemini-2.5-Pro) (77.14) by 2.9.
GSMA Open-Telco - 3GPP (Score (%)): TelecomGPT (84.22) beat OTel-LLM-8.3B-QnA (81.4) by 2.82.
GSMA Open-Telco - TeleLogs (Score (%)): TelecomGPT (98.96) beat OTel-LLM-8.3B-QnA (96.3) by 2.66.
Evals for Every Language - MGSM (Average Score (%)): Claude Opus 4.8 (96.62) beat Claude Opus 4.6 (94.26) by 2.36.
Evals for Every Language - Language ban (Average Score (%)): step-3.7-flash-20260528 (69.03) beat Claude Opus 4.5 (66.71) by 2.32.
SimpleBench (Score (AVG@5)): Claude Fable 5 (81.9) beat Gemini 3.1 Pro (Preview) (79.6) by 2.3.
AA Terminal-Bench Hard (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (62.88) beat GPT-5.5 (xHigh) (60.61) by 2.27.
Chatbot Arena (Image-to-Video) (Elo): gemini-omni-flash (1475.0) beat Grok 1.5 (1473.0) by 2.0.
LiveBench Plot Unscrambling (Score): Claude Fable 5 (xHigh) (78.09) beat GPT-5.5 (High) (76.28) by 1.81.
UGI - Writing (Writing Score): Claude Fable 5 (Adaptive Reasoning, High Effort) (74.23) beat Gemini 3.5 Flash (Thinking, Medium) (72.54) by 1.69.
GSMA Open-Telco - srsRAN-Bench (Score (%)): TelecomGPT (91.33) beat OTel-LLM-8.3B-QnA (89.68) by 1.65.
LLM Stats (OSWorld-Verified) (Score (%)): Claude Fable 5 (85.0) beat Claude Opus 4.8 (83.4) by 1.6.
AA Omniscience - Software Engineering (SWE) - Python (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.5 (xHigh) (90.5) by 1.5.
Evals for Every Language - Language chm (Average Score (%)): Claude Opus 4.7 (63.6) beat Gemini 3.1 Pro (Preview) (62.12) by 1.48.
Evals for Every Language - Language doi (Average Score (%)): Claude Opus 4.7 (71.84) beat Gemini 3 Pro (Preview) (70.38) by 1.46.
AA CritPt (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (28.57) beat GPT-5.5 (xHigh) (27.14) by 1.43.
Evals for Every Language - Language es (Average Score (%)): Gemini 3.1 Flash Lite (76.16) beat Claude Opus 4.6 (74.74) by 1.42.
AA SciCode (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (60.19) beat Gemini 3.1 Pro (Preview) (58.91) by 1.28.
Evals for Every Language - Language ace (Average Score (%)): step-3.7-flash-20260528 (72.48) beat Gemini 3.1 Pro (Preview) (71.2) by 1.28.
Evals for Every Language - MMLU (Average Score (%)): intellect-3-20251126 (100.0) beat Claude Sonnet 4.6 (98.73) by 1.27.
Evals for Every Language - ARC (Average Score (%)): intellect-3-20251126 (100.0) beat Gemini 3.1 Pro (Preview) (98.74) by 1.26.
EQ-Bench Longform Writing (Writing Score (0-100)): Claude Fable 5 (83.0) beat Claude Opus 4.7 (81.8) by 1.2.
Vals AI LegalBench (Accuracy (%)): Claude Fable 5 (88.56) beat Gemini 3.1 Pro (Preview) (87.4) by 1.16.
Evals for Every Language - Language ca (Average Score (%)): Gemini 3.1 Flash Lite (76.29) beat Gemini 3 Pro (Preview) (75.26) by 1.03.
Design Arena (SVG) (Elo): Claude Fable 5 (1370.0) beat prism (1369.0) by 1.0.
Opper TaskBench (Avg Task Score (%)): Claude Fable 5 (96.4) beat Claude Opus 4.7 (95.4) by 1.0.
Evals for Every Language - Language ar (Average Score (%)): Claude Opus 4.8 (71.58) beat Claude Opus 4.5 (70.63) by 0.95.
Evals for Every Language - Language en (Average Score (%)): Gemini 3.1 Flash Lite (87.28) beat MiniMax-M2.5 (86.51) by 0.77.
MathArena - HMMT Feb 2026 (Accuracy (%)): GPT-5.5 (xHigh) (98.48) beat GPT-5.4 (xHigh) (97.73) by 0.75.
Evals for Every Language - Language cy (Average Score (%)): Gemini 3.1 Flash Lite (82.03) beat Claude Sonnet 4.5 (81.38) by 0.65.
Evals for Every Language - Language am (Average Score (%)): Gemini 3.1 Flash Lite (68.6) beat Claude Opus 4.6 (68.01) by 0.59.
Vals AI MedScribe (Accuracy (%)): Claude Fable 5 (88.52) beat GPT-5.1 (88.09) by 0.43.
Evals for Every Language - Language af (Average Score (%)): Gemini 3.1 Pro (Preview) (79.41) beat Claude Sonnet 4 (78.98) by 0.43.
Evals for Every Language - Language be (Average Score (%)): Claude Opus 4.8 (69.43) beat Gemini 3.1 Pro (Preview) (69.11) by 0.32.
LLM Stats (Video-MME) (Score (%)): MiMo-V2.5 (87.7) beat Kimi K2.5 (87.4) by 0.3.
Evals for Every Language - Language ceb (Average Score (%)): Gemini 3.1 Flash Lite (78.06) beat Gemini 3.1 Pro (Preview) (77.77) by 0.29.
Evals for Every Language - Language el (Average Score (%)): Gemini 3.1 Flash Lite (73.81) beat Claude Opus 4.5 (73.66) by 0.15.
LLM Stats (CMMLU) (Score (%)): MiMo-V2.5-Pro (90.2) beat Qwen 2 72B Instruct (90.1) by 0.1.
Blueprint-Bench 2 (Connectivity Similarity Score): Claude Fable 5 (0.386) beat GPT-5.5 (0.37) by 0.02.
LiveBench Olympiad (Score): Claude Fable 5 (High) (92.18) beat Claude Opus 4.6 (Thinking, High) (92.17) by 0.01.

                                Don't miss what's next. Subscribe to Mikhail Doroshenko:

            Email address (required)