AI Benchmark Digest — 2026-07-19

        July 19, 2026

AI Benchmark Digest — 2026-07-19

AI Benchmark Digest — 2026-07-19
View on AI Benchmark Hub
Daily
New Benchmarks (9)

SCALE SQL Understanding (Ability Score (%)): Gemini 3 Pro leads with 86.0 across 35 models.
SCALE SQL Dialect Conversion (Ability Score (%)): Qwen 3.7 Max leads with 77.3 across 35 models.
SCALE SQL Optimization (Ability Score (%)): Gemini 3.5 Flash leads with 69.2 across 35 models.
ParseBench (Overall Score): KDL-Frontier-Parser-nano leads with 76.36 across 50 models.
  Enterprise document parsing benchmark over tables, charts, layout, and content faithfulness for multimodal extraction systems.
OmniaBench (Overall Pass@1 (%)): Claude Sonnet 5 leads with 58.54 across 22 models.
PawBench - QwenPaw (Overall Score (%)): Qwen 3.6 Max Preview leads with 78.35 across 9 models.
PawBench - OpenClaw (Overall Score (%)): Claude Opus 4.6 leads with 76.11 across 9 models.
PawBench - Hermes (Overall Score (%)): Claude Opus 4.6 leads with 78.44 across 9 models.
LoHoSearch (Accuracy (%)): GPT-5.5 leads with 34.74 across 11 models.
  LoHoSearch evaluates long-horizon search agents on 544 human-verified questions requiring multi-step open-web research across 11 topical domains; this board reports the paper\

New Scores From Top-10 Models (18)

Kimi K3 on AA APEX-Agents: 41.3 Pass@1 (%) (#2/27)
Kimi K3 on Arabic Broad Leaderboard: 8.906 Average Score (0-10) (#18/104)
Kimi K3 on HieroglyphBench: 25.9 Sign accuracy (%) (#5/19)
Kimi K3 on KernelBench Hub - Mega: 18.09 Best Speedup vs Reference (x) (#3/12)
Kimi K3 on KernelBench Hub - Mega: 18.09 Best Speedup vs Reference (x) (#3/12)
Kimi K3 on Vals AI GPQA: 92.93 Accuracy (%) (#5/123)
Kimi K3 on Vals AI Harvey Legal Agent Bench: 10.83 Accuracy (%) (#4/24)
Kimi K3 on Vals AI Legal Research Bench: 44.23 Accuracy (%) (#3/24)
Kimi K3 on Vals AI LegalBench: 86.02 Accuracy (%) (#9/126)
Kimi K3 on Vals AI LiveCodeBench: 87.19 Accuracy (%) (#9/127)
Kimi K3 on Vals AI MMLU-Pro: 87.97 Accuracy (%) (#14/121)
Kimi K3 on Vals AI MMMU: 88.15 Accuracy (%) (#6/82)
Kimi K3 on Vals AI MedScribe: 87.96 Accuracy (%) (#4/72)
Kimi K3 on Vals AI MortgageTax: 66.34 Accuracy (%) (#30/86)
Kimi K3 on Vals AI Multimodal Index: 73.42 Accuracy (%) (#2/27)
Kimi K3 on Vals AI SAGE: 54.26 Accuracy (%) (#4/68)
Kimi K3 on Vals AI TaxEval v2: 75.72 Accuracy (%) (#10/129)
Kimi K3 on You're Absolutely Right!: 3.5 Average anti-sycophancy score (1-5) (#8/22)

New #1 Leaders (6)

Design Arena (3D) (Elo): Kimi K3 (1441.0) beat Claude Fable 5 (1367.0) by 74.0.
Design Arena (Data Viz) (Elo): Kimi K3 (1429.0) beat Claude Fable 5 (1364.0) by 65.0.
Vals AI Legal Research Bench (Accuracy (%)): Claude Fable 5 (49.52) beat GPT-5.6 Sol (48.08) by 1.44.
Vals AI Excel Modeling (Accuracy (%)): Claude Fable 5 (73.67) beat GPT-5.6 Sol (72.34) by 1.33.
MMEB Leaderboard (Overall Score (%)): DME-Large (80.24) beat octen-vl-embedding-large (80.09) by 0.15.
Vals AI Public Benefits Bench (Accuracy (%)): Kimi K3 (68.27) beat Claude Opus 4.8 (68.13) by 0.14.

Weekly
New Benchmarks (9)

SCALE SQL Understanding (Ability Score (%)): Gemini 3 Pro leads with 86.0 across 35 models.
SCALE SQL Dialect Conversion (Ability Score (%)): Qwen 3.7 Max leads with 77.3 across 35 models.
SCALE SQL Optimization (Ability Score (%)): Gemini 3.5 Flash leads with 69.2 across 35 models.
ParseBench (Overall Score): KDL-Frontier-Parser-nano leads with 76.36 across 50 models.
  Enterprise document parsing benchmark over tables, charts, layout, and content faithfulness for multimodal extraction systems.
OmniaBench (Overall Pass@1 (%)): Claude Sonnet 5 leads with 58.54 across 22 models.
PawBench - QwenPaw (Overall Score (%)): Qwen 3.6 Max Preview leads with 78.35 across 9 models.
PawBench - OpenClaw (Overall Score (%)): Claude Opus 4.6 leads with 76.11 across 9 models.
PawBench - Hermes (Overall Score (%)): Claude Opus 4.6 leads with 78.44 across 9 models.
LoHoSearch (Accuracy (%)): GPT-5.5 leads with 34.74 across 11 models.
  LoHoSearch evaluates long-horizon search agents on 544 human-verified questions requiring multi-step open-web research across 11 topical domains; this board reports the paper\

New Models (72)

Claude Fable 5 — ELO 3074, #6/1475, above Gemini 3 Deep Think, below GPT-5.4 Pro
Android Bench: 84.5 (#1/25)
Terminal-Bench 2.1: 83.8 (#1/12)
Terminal-Bench 2.1 (Claude Code): 83.8 (#1/5)
SvelteBench: 100.0 (#1/152)
GeoBench ACW: 4403.7 (#1/48)
GeoBench Photos: 4313.64 (#1/32)
Vals AI Excel Modeling: 73.67 (#1/25)
Vals AI Legal Research Bench: 49.52 (#1/24)
Arabic Broad Leaderboard: 8.217 (#40/104)

Kimi K3 — ELO 2874, #8/1475, above GPT-5.4 Pro, below GPT-5.6 Sol
BridgeBench Security: 1023.1 (#1/5)
BridgeBench Debugging: 1066.7 (#1/3)
BridgeBench Refactoring: 1065.7 (#1/3)
BridgeBench Hallucination: 1030.4 (#1/3)
Design Arena (Data Viz): 1429.0 (#1/139)
Design Arena (3D): 1441.0 (#1/131)
Tau3 Banking: 33.4 (#1/159)
Chatbot Arena (Code): 1679.0 (#1/99)
LLM Stats (APEX-Agents): 37.6 (#1/7)
LLM Stats (BabyVision): 85.7 (#1/9)

GPT-5.6 Sol — ELO 2827, #9/1475, above Kimi K3, below Claude Opus 4.8
DeepSWE: 72.7 (#1/20)
Benchmarks.bio - SpatialBench-Long: 38.89 (#1/18)
ZeroEval GPQA Diamond: 94.6 (#1/232)
MineBench: 2108.79 (#1/50)
Agents' Last Exam: 30.6 (#1/20)
AgenticVBench: 38.4 (#1/10)
LLM Stats (Graphwalks BFS >128k): 90.7 (#1/11)
LLM Stats (MRCR v2 (8-needle)): 91.5 (#1/19)
LLM Stats (Terminal-Bench 2.1): 88.8 (#1/13)
AI for Education Visual Reasoning - match (process): 88.9 (#1/64)

Claude Opus 4.8 — ELO 2741, #10/1475, above GPT-5.6 Sol, below GPT-5.6 Pro Sol
BridgeBench Security: 1001.5 (#2/5)
SvelteBench: 100.0 (#4/152)
Android Bench: 72.4 (#6/25)

GPT-5.6 Pro Sol — ELO 2671, #11/1475, above Claude Opus 4.8, below GPT-5.5
Arabic Broad Leaderboard: 8.97 (#12/104)

GPT-5.5 — ELO 2620, #12/1475, above GPT-5.6 Pro Sol, below Qwen 3.7 Max
GDP.pdf: 52.8 (#2/20)
ExploitGym: 129.0 (#3/9)

Qwen 3.7 Max — ELO 2590, #13/1475, above GPT-5.5, below Inkling
Vals AI Finance Agent v2: 47.78 (#16/37)
Android Bench: 54.2 (#20/25)

Inkling — ELO 2556, #14/1475, above Qwen 3.7 Max, below Muse Spark 1.1
EQ-Bench Longform Writing: 72.5 (#22/123)
RuneBench: 1590.0 (#24/33)
ARC-AGI-2: 36.53 (#47/166)
ARC-AGI-1: 79.5 (#48/164)

Muse Spark 1.1 — ELO 2532, #15/1475, above Inkling, below Claude Opus 4.6
LLM Stats (MCP Atlas): 88.1 (#1/30)
LLM Stats (Toolathlon): 75.6 (#1/30)
LLM Stats (BabyVision): 76.3 (#2/9)
LLM Stats (Finance Agent v2): 57.2 (#2/26)
Arabic Broad Leaderboard: 9.2 (#4/104)
LLM Stats (OSWorld-Verified): 80.8 (#4/19)
OSWorld: 80.67 (#5/63)
LLM Stats (CharXiv-R): 88.4 (#5/45)
HieroglyphBench: 24.5 (#6/19)
Terminal-Bench 2.1: 76.2 (#6/12)

GPT-5.6 Terra — ELO 2489, #18/1475, above Gemini 3.1 Pro (Preview), below Gemini 3.5 Flash
Benchmarks.bio - SpatialBench-Long: 22.22 (#2/18)
WolfBench: 82.0 (#2/32)
LLM Stats (MRCR v2 (8-needle)): 89.6 (#2/19)
LLM Stats (HealthBench): 57.0 (#3/8)
LLM Stats (HealthBench Hard): 32.7 (#3/9)
LLM Stats (Terminal-Bench 2.1): 87.4 (#3/13)
Kaggle Game Arena Poker (Heads Up): 17.85 (#3/24)
Clerk LLM Leaderboard: 79.2 (#4/22)
LLM Stats (BrowseComp): 87.5 (#4/57)
LLM Stats (Graphwalks BFS >128k): 76.9 (#4/11)

Gemini 3.5 Flash — ELO 2477, #19/1475, above GPT-5.6 Terra, below Claude Sonnet 5
Chatbot Arena (Code): 1504.0 (#19/99)

Claude Sonnet 5 — ELO 2462, #20/1475, above Gemini 3.5 Flash, below Qwen 3.6 Max
Android Bench: 76.2 (#3/25)
Terminal-Bench 2.1 (Claude Code): 74.6 (#3/5)
Benchmarks.bio - SpatialBench-Long: 12.5 (#5/18)
CHI-Bench: 20.0 (#7/18)
Terminal-Bench 2.1: 74.6 (#8/12)
Benchmarks.bio - BioSecBench-Refusal: 30.1 (#11/15)
GeoBench Photos: 3713.48 (#12/32)
GeoBench ACW: 3878.41 (#19/48)
AI for Education Pedagogy: 88.21 (#22/221)
AI for Education SEND: 82.57 (#22/213)

Claude Opus 4.7 — ELO 2436, #22/1475, above Qwen 3.6 Max, below Grok 4.5
Creative Writing (Lechmazur): 2.5 (#7/38)

Grok 4.5 — ELO 2436, #23/1475, above Claude Opus 4.7, below GPT-5 Pro
Benchmarks.bio - SpatialBench-Long: 19.44 (#3/18)
Terminal-Bench 2.1: 79.3 (#3/12)
TrackingAI IQ Test (Offline): 81.25 (#5/35)
Blueprint-Bench 2: 0.273 (#6/18)
LLM Stats (Terminal-Bench 2.1): 83.3 (#6/13)
AI for Education Pedagogy - Secondary: 89.31 (#6/221)
React Native Evals: 84.58 (#7/26)
Kaggle Game Arena Poker (Heads Up): 5.18 (#7/24)
ZeroEval GPQA Diamond: 93.0 (#9/232)
HieroglyphBench: 21.2 (#9/19)

GLM-5.2 — ELO 2400, #27/1475, above Muse Spark, below Gemini 3 Pro
NatureBench: 15.56 (#2/12)
Android Bench: 72.2 (#7/25)
CHI-Bench: 18.7 (#9/18)
React Native Evals: 78.36 (#14/26)
Design Arena (SVG): 1260.0 (#19/100)

GPT-5.6 Luna — ELO 2386, #30/1475, above Grok Build 0.1, below Seed 2.1 Pro
Clerk LLM Leaderboard: 83.6 (#2/22)
LLM Stats (Graphwalks BFS >128k): 81.3 (#2/11)
Vals AI CyberBench: 83.9 (#2/17)
Benchmarks.bio - SpatialBench-Long: 18.06 (#4/18)
WolfBench: 79.0 (#4/32)
LLM Stats (HealthBench Hard): 32.0 (#4/9)
LLM Stats (Terminal-Bench 2.1): 84.7 (#4/13)
Kaggle Game Arena Poker (Heads Up): 15.95 (#4/24)
Vals AI Finance Agent v2: 55.04 (#4/37)
Vals AI SWE-bench Verified: 93.0 (#4/72)

Qwen 3.7 Plus — ELO 2326, #36/1475, above GPT-5 (2025-08-07), below Gemini 3.1 Flash
Design Arena (3D): 1324.0 (#9/131)
Android Bench: 57.7 (#18/25)
Design Arena (Data Viz): 1280.0 (#19/139)
Design Arena (Game Dev): 1276.0 (#26/141)
Design Arena (UI Components): 1279.0 (#29/136)

Gemini 2.5 Pro — ELO 2286, #44/1475, above Qwen3-VL-235B-A22B-Thinking-FP8, below Gemini 3 Flash (Preview)
FutureEval: 2.43 (#46/77)

MiniMax-M3 — ELO 2272, #48/1475, above O3 Pro (2025-06-10), below Qwen 3.6 Max Preview
NatureBench: 11.11 (#6/12)
Android Bench: 63.6 (#13/25)
Creative Writing (Lechmazur): 0.7 (#16/38)

DeepSeek V4 Pro — ELO 2251, #51/1475, above Claude Opus 4.5 (20251101), below Kimi K2.7 Code
Android Bench: 59.5 (#17/25)

Kimi K2.7 Code — ELO 2250, #52/1475, above DeepSeek V4 Pro, below Gemini 2.5 Pro (03-25)
Android Bench: 70.4 (#9/25)

MiMo-V2.5-Pro — ELO 2236, #55/1475, above Claude Opus 4.5, below GPT-5.4 (2026-03-05)
Android Bench: 60.8 (#16/25)

Hy3 — ELO 2227, #60/1475, above Grok 4.20 Multi-Agent, below Claude Sonnet 4.6
LLM Stats (SkillsBench): 55.3 (#2/6)
LLM Stats (IMO-AnswerBench): 90.0 (#3/19)
LLM Stats (WideSearch): 76.4 (#3/9)
LLM Stats (Claw-Eval): 68.5 (#4/13)
LLM Stats (DeepSearchQA): 91.0 (#4/8)
LLM Stats (MathArena Apex): 38.7 (#4/7)
LLM Stats (NL2Repo): 45.6 (#4/12)
LLM Stats (APEX-Agents): 25.6 (#7/7)
LLM Stats (MCP Atlas): 79.1 (#7/30)
LLM Stats (Terminal-Bench 2.1): 71.7 (#9/13)

DeepSeek V4 Flash — ELO 2180, #66/1475, above Doubao-1.5-Pro, below Rombos-LLM-V2.5-Qwen-72B
Lean AI Formalization Leaderboard: 3.0 (#14/41)
Android Bench: 54.7 (#19/25)

MiMo-V2-Pro — ELO 2146, #76/1475, above Seed 2.0 Lite, below Qwen 3.5 Plus
Creative Writing (Lechmazur): -0.6 (#25/38)

Qwen 3.6 Plus — ELO 2139, #81/1475, above Gemini 2.5 Pro (Preview 06-05), below Ultiima-72B
FutureEval: 7.53 (#25/77)

inkling — ELO 2118, #94/1475, above Median Human, below GPT-5 Mini (2025-08-07)
Vals AI CorpFin v2: 68.57 (#4/123)
Vals AI MedScribe: 85.41 (#12/72)
Vals AI CyberBench: 64.14 (#12/17)
Vals AI SkillsBench: 26.09 (#13/13)
Vals AI Public Benefits Bench: 58.46 (#14/19)
Vals AI Harvey Legal Agent Bench: 2.08 (#14/24)
Vals AI TaxEval v2: 75.31 (#15/129)
Vals AI Legal Research Bench: 28.36 (#15/24)
Vals AI Finance Agent v2: 46.6 (#17/37)
Vals AI SWE-bench Verified: 77.6 (#20/72)

Kimi K2.5 — ELO 2110, #96/1475, above GPT-5 Mini (2025-08-07), below DeepSeek V3.2 Speciale
Creative Writing (Lechmazur): -0.5 (#23/38)

Qwen 3 Max — ELO 2104, #99/1475, above Claude Opus 4.1 (20250805), below Grok 4
FutureEval: 0.12 (#58/77)

Grok 4 — ELO 2103, #100/1475, above Qwen 3 Max, below Step 3.7 Flash
FutureEval: 4.13 (#40/77)

Gemini 3.1 Flash Lite — ELO 2071, #114/1475, above MiMo-V2.5, below Qwen 3.6 27B
FutureEval: 3.99 (#41/77)

Claude Sonnet 4 — ELO 2047, #118/1475, above Gemini 2.5 Flash (Preview 05-20), below GPT-5 Mini
FutureEval: 0.06 (#59/77)

GPT-5 Mini — ELO 2045, #119/1475, above Claude Sonnet 4, below Grok 4 Fast
FutureEval: 0.83 (#56/77)

O4 Mini — ELO 2035, #123/1475, above LongCat-Flash-Thinking-2601, below Claude Opus 4.1
FutureEval: 3.12 (#44/77)

Grok 4.1 Fast — ELO 2023, #130/1475, above Claude Opus 4, below Claude 3.7 Sonnet (20250219)
FutureEval: 0.21 (#57/77)

KAT Coder Pro V2 — ELO 2019, #133/1475, above GPT-5.5 Instant, below Claude Opus 4 (20250514)
AA GDPval: 908.18 (#79/157)
GDPval-AA: 908.0 (#83/162)

Grok 4.20 — ELO 2014, #135/1475, above Claude Opus 4 (20250514), below Gemma 4 31B (IT)
HieroglyphBench: 16.8 (#13/19)
FrontierMath - Tier 4 (v2): 17.07 (#26/40)
FrontierMath - Tiers 1-3 (v2): 44.91 (#29/38)

Mercury 2 — ELO 2003, #140/1475, above Doubao Seed Code, below Ring-2.6-1T
AA GDPval: 688.89 (#99/157)
GDPval-AA: 689.0 (#103/162)

Kimi K2 — ELO 1991, #147/1475, above Qwen 3 Max (2025-09-23), below Qwen 3.5 122B A10B
FutureEval: 0.97 (#55/77)

Qwen 3 Max (Preview) — ELO 1979, #154/1475, above Gemma 4 26B A4B (IT), below O3 Mini
Creative Writing (Lechmazur): 0.0 (#20/38)

DeepSeek V3.2 — ELO 1957, #167/1475, above GPT-5.1 Codex Mini, below Gemini 2.5 Flash
Creative Writing (Lechmazur): -2.3 (#34/38)

Gemini 2.5 Flash — ELO 1954, #168/1475, above DeepSeek V3.2, below Gemini 2.5 Pro (Preview 03-25)
FutureEval: -8.3 (#73/77)

GPT-4.1 — ELO 1944, #173/1475, above calme-2.1-qwen2.5-72B, below MiMo-V2-Omni
FutureEval: 3.02 (#45/77)

Claude 3.5 Sonnet (20240620) — ELO 1931, #178/1475, above DeepSeek V3.2 Exp, below Grok 3 Mini
FutureEval: -2.47 (#66/77)

Grok 3 Mini — ELO 1931, #179/1475, above Claude 3.5 Sonnet (20240620), below InternVL3-78B
FutureEval: 1.95 (#51/77)

O1 Mini — ELO 1918, #189/1475, above Gemini 2.5 Flash (Preview 04-17), below Qwen 3 235B A22B FP8
FutureEval: -9.43 (#75/77)

DeepSeek V3.1 — ELO 1916, #194/1475, above GLM-4.7 FP8, below Claude 3.5 Sonnet (20241022)
FutureEval: 2.37 (#47/77)

MiniMax-M2.5 — ELO 1913, #198/1475, above Mistral-Large-3-675B-Instruct-2512, below Hermes 4 405B
FutureEval: -4.74 (#68/77)

Gemma 4 31B — ELO 1899, #205/1475, above Qwen 2 VL 72B, below Mistral Medium 3.1
LLM Stats (MRCR v2 (8-needle)): 66.4 (#5/19)

DeepSeek R1 — ELO 1862, #237/1475, above InternVL3-38B, below INTELLECT-3
FutureEval: -0.37 (#61/77)

GPT-OSS-120B — ELO 1855, #241/1475, above Qwen 3 VL 32B Instruct, below GPT-5.4 Nano
FutureEval: -0.72 (#62/77)

Claude Haiku 4.5 — ELO 1851, #244/1475, above Trinity Large, below gemma2-9B-cpt-sahabatai-v1-instruct
FutureEval: 3.25 (#43/77)

Grok 3 — ELO 1842, #250/1475, above Command A+, below DeepSeek Reasoner
FutureEval: 1.59 (#53/77)

Mistral Large 3 — ELO 1832, #258/1475, above Falcon-H1R-7B, below doubao-seed-1.6-thinking-250615
Chatbot Arena (Vision): 1203.0 (#60/135)

DeepSeek V3 — ELO 1815, #265/1475, above Llama 3.1 405B, below Hermes 4 70B
FutureEval: -5.35 (#69/77)

Claude 3.7 Sonnet — ELO 1792, #281/1475, above Qwen 3 Coder 480B A35B Instruct, below Llama 3.1 Nemotron Ultra 253B v1
FutureEval: -2.36 (#65/77)

Llama 3.1 Nemotron Ultra 253B v1 — ELO 1788, #282/1475, above Claude 3.7 Sonnet, below ko-gemma-2-9B-it
FutureEval: -10.7 (#76/77)

Claude 3.5 Sonnet — ELO 1782, #286/1475, above lapa-v0.1.2-instruct, below gemma-3-27B-pt
FutureEval: 3.59 (#42/77)

GPT-4.1 Mini — ELO 1761, #308/1475, above Ring-flash-2.0, below Mistral Small 3.2
FutureEval: -1.4 (#63/77)

Gemini 2.0 Flash — ELO 1759, #312/1475, above Command A, below Claude Haiku 4.5 (20251001)
Design Arena (Video Editing): 1417.0 (#1/15)
FutureEval: 4.18 (#37/77)

GPT-5 Nano — ELO 1754, #319/1475, above Qwentile2.5-32B-Instruct, below Llama 3.1 405B Instruct FP8
FutureEval: -6.06 (#71/77)

Laguna XS.2 — ELO 1747, #322/1475, above Qwen3 Coder, below Qwen2.5-32B-Instruct-abliterated-v2
Vals AI Harvey Legal Agent Bench: 0.0 (#24/24)
Vals AI Legal Research Bench: 0.96 (#24/24)

GPT-4o Mini — ELO 1688, #378/1475, above EXAONE 4.0 32B, below Ministral-3-8B-Instruct-2512
FutureEval: -8.97 (#74/77)

Llama 4 Maverick — ELO 1680, #383/1475, above Sonar, below Qwen 2.5 14B
FutureEval: -5.36 (#70/77)

Laguna M.1 — ELO 1642, #421/1475, above Qwen2.5-14B-Gutenberg-1e-Delta, below Chocolatine-2-14B-Instruct-v2.0
Vals AI Harvey Legal Agent Bench: 0.0 (#20/24)
Vals AI Legal Research Bench: 2.4 (#23/24)
Vals AI Finance Agent v2: 25.03 (#36/37)

Qwen 2.5 Max — ELO 1625, #440/1475, above Llama 3.1 Nemotron 70B Instruct, below GLM-4 32B
FutureEval: -3.83 (#67/77)

Gemma 4 26B A4B — ELO 1624, #442/1475, above GLM-4 32B, below Apertus-8B-Instruct-2509
LLM Stats (MRCR v2 (8-needle)): 44.1 (#7/19)

Gemma 3 27B — ELO 1613, #452/1475, above Ministral-3-3B-Reasoning-2512, below Palmyra X V3 (72B)
LLM Stats (MRCR v2 (8-needle)): 13.5 (#19/19)

Gemma 4 12B — ELO 1605, #463/1475, above Llama 3.2 90B Instruct, below Gemma 3n E4B (IT)
LLM Stats (MRCR v2 (8-needle)): 43.4 (#8/19)

GPT-4.1 Nano — ELO 1458, #699/1475, above EXAONE 3.5 7.8B Instruct, below Claude 2.1
FutureEval: -18.82 (#77/77)

Gemma 4 E4B — ELO 1316, #1050/1475, above dolphin-2.9.3-mistral-7B-32k, below salamandra-7B-instruct
LLM Stats (MRCR v2 (8-needle)): 25.4 (#15/19)

Gemma 4 E2B — ELO 1115, #1314/1475, above ReflectionCoder-DS-33B, below MagpieLM-8B-Chat-v0.1
LLM Stats (MRCR v2 (8-needle)): 19.1 (#17/19)

New Scores From Top-10 Models (120)

Claude Fable 5 on Arabic Broad Leaderboard: 8.217 Average Score (0-10) (#40/104)
Claude Fable 5 on SvelteBench: 100.0 Average pass@1 (%) (#1/152)
Claude Opus 4.8 on Android Bench: 72.4 Score (%) (#6/25)
Claude Opus 4.8 on BridgeBench Security: 1001.5 Confidence-weighted Elo (#2/5)
Claude Opus 4.8 on SvelteBench: 100.0 Average pass@1 (%) (#4/152)
GPT-5.6 Pro Sol on Arabic Broad Leaderboard: 8.97 Average Score (0-10) (#12/104)
GPT-5.6 Sol on AI for Education Pedagogy: 91.21 Accuracy (%) (#3/221)
GPT-5.6 Sol on AI for Education Pedagogy - Maths: 92.86 Accuracy (%) (#4/221)
GPT-5.6 Sol on AI for Education Pedagogy - Primary: 94.37 Accuracy (%) (#7/221)
GPT-5.6 Sol on AI for Education Pedagogy - Science: 92.35 Accuracy (%) (#17/221)
GPT-5.6 Sol on AI for Education Pedagogy - Secondary: 90.09 Accuracy (%) (#3/221)
GPT-5.6 Sol on AI for Education Pedagogy - Social studies: 90.91 Accuracy (%) (#4/221)
GPT-5.6 Sol on AI for Education Pedagogy - Technology: 88.68 Accuracy (%) (#5/221)
GPT-5.6 Sol on AI for Education SEND: 87.61 Accuracy (%) (#2/213)
GPT-5.6 Sol on AI for Education Visual Reasoning: 84.9 Accuracy (%) (#2/64)
GPT-5.6 Sol on AI for Education Visual Reasoning - match (figure): 77.0 Accuracy (%) (#3/64)
GPT-5.6 Sol on AI for Education Visual Reasoning - pattern completion (2d): 82.2 Accuracy (%) (#3/64)
GPT-5.6 Sol on AI for Education Visual Reasoning - reasoning by analogy: 86.5 Accuracy (%) (#2/64)
GPT-5.6 Sol on Agent Security League - Functional Correctness: 70.9 Functional Correctness (%) (#11/18)
GPT-5.6 Sol on Agent Security League - Security Correctness: 23.5 Security Correctness (%) (#3/18)
GPT-5.6 Sol on Agents' Last Exam: 30.6 Pass Rate (%) (#1/20)
GPT-5.6 Sol on Benchmarks.bio - BioSecBench-Refusal: 39.5 Pass Rate (%) (#5/15)
GPT-5.6 Sol on Blueprint-Bench 2: 0.336 Connectivity Similarity Score (#4/18)
GPT-5.6 Sol on CADGenBench: 0.4108 Aggregate CAD Score (#7/18)
GPT-5.6 Sol on Clerk LLM Leaderboard: 80.5 Avg score (%) (#3/22)
GPT-5.6 Sol on DeepSWE: 72.7 Pass@1 (%) (#1/20)
GPT-5.6 Sol on EQ-Bench Longform Writing: 81.7 Writing Score (0-100) (#3/123)
GPT-5.6 Sol on HieroglyphBench: 27.6 Sign accuracy (%) (#4/19)
GPT-5.6 Sol on Kaggle Game Arena Poker (Heads Up): 36.05 Mean BB/100 (#2/24)
GPT-5.6 Sol on LLM Stats (BrowseComp): 90.4 Score (%) (#2/57)
GPT-5.6 Sol on LLM Stats (GDPval-AA): 1747.8 ELO (#2/39)
GPT-5.6 Sol on LLM Stats (HealthBench Hard): 33.1 Score (%) (#2/9)
GPT-5.6 Sol on LLM Stats (HealthBench Professional): 60.5 Score (%) (#2/8)
GPT-5.6 Sol on LLM Stats (HealthBench): 57.0 Score (%) (#4/8)
GPT-5.6 Sol on LLM Stats (Toolathlon): 58.0 Score (%) (#4/30)
GPT-5.6 Sol on React Native Evals: 86.34 Overall Score (%) (#4/26)
GPT-5.6 Sol on ZeroEval GPQA Diamond: 94.6 GPQA Diamond Score (#1/232)
Kimi K3 on AA APEX-Agents: 41.3 Pass@1 (%) (#2/27)
Kimi K3 on AA CritPt: 23.43 Accuracy (%) (#9/447)
Kimi K3 on AA GDPval: 1683.65 ELO (#4/157)
Kimi K3 on AA GPQA Diamond: 93.54 Accuracy (%) (#4/541)
Kimi K3 on AA Humanity's Last Exam: 44.35 Accuracy (%) (#7/537)
Kimi K3 on AA Long Context Reasoning: 74.67 Accuracy (%) (#4/469)
Kimi K3 on AA MMMU-Pro: 80.52 Accuracy (%) (#13/226)
Kimi K3 on AA Omniscience: 18.42 Score (#13/446)
Kimi K3 on AA Omniscience - Business: 38.8 Accuracy (%) (#27/446)
Kimi K3 on AA Omniscience - Health: 37.5 Accuracy (%) (#44/446)
Kimi K3 on AA Omniscience - Humanities & Social Sciences: 43.8 Accuracy (%) (#26/446)
Kimi K3 on AA Omniscience - Law: 37.7 Accuracy (%) (#33/446)
Kimi K3 on AA Omniscience - Science, Engineering & Mathematics: 47.8 Accuracy (%) (#19/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE): 70.1 Accuracy (%) (#25/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - C: 87.0 Accuracy (%) (#11/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Dart: 62.0 Accuracy (%) (#23/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Go: 56.0 Accuracy (%) (#48/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - HTML: 76.0 Accuracy (%) (#22/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Java: 53.0 Accuracy (%) (#30/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - JavaScript: 77.27 Accuracy (%) (#21/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Julia: 48.0 Accuracy (%) (#47/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Kotlin: 66.0 Accuracy (%) (#28/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - PHP: 78.0 Accuracy (%) (#21/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Python: 70.0 Accuracy (%) (#33/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - R: 62.0 Accuracy (%) (#20/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Rust: 74.0 Accuracy (%) (#43/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Swift: 80.0 Accuracy (%) (#22/446)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - TypeScript: 74.44 Accuracy (%) (#26/446)
Kimi K3 on AA SciCode: 58.68 Accuracy (%) (#3/535)
Kimi K3 on AA-Omniscience Accuracy: 45.95 Accuracy (%) (#25/446)
Kimi K3 on AI Chess Leaderboard (Continuation): 1351.0 Elo (#18/240)
Kimi K3 on AI Chess Leaderboard (Reasoning): 1518.0 Elo (#18/296)
Kimi K3 on Arabic Broad Leaderboard: 8.906 Average Score (0-10) (#18/104)
Kimi K3 on Artificial Analysis Intelligence Index: 57.11 Intelligence Index (#4/540)
Kimi K3 on BridgeBench Debugging: 1066.7 Confidence-weighted Elo (#1/3)
Kimi K3 on BridgeBench Hallucination: 1030.4 Confidence-weighted Elo (#1/3)
Kimi K3 on BridgeBench Refactoring: 1065.7 Confidence-weighted Elo (#1/3)
Kimi K3 on BridgeBench Security: 1023.1 Confidence-weighted Elo (#1/5)
Kimi K3 on Bullshit Benchmark: 58.2 BS Detection Rate (%) (#23/162)
Kimi K3 on Chatbot Arena (Text): 1486.0 Elo (#9/376)
Kimi K3 on DeepSWE: 68.5 Pass@1 (%) (#5/20)
Kimi K3 on EQ-Bench Longform Writing: 79.6 Writing Score (0-100) (#6/123)
Kimi K3 on GDPval-AA: 1684.0 Elo (#4/162)
Kimi K3 on HieroglyphBench: 25.9 Sign accuracy (%) (#5/19)
Kimi K3 on KernelBench Hub - Mega: 18.09 Best Speedup vs Reference (x) (#3/12)
Kimi K3 on KernelBench Hub - Mega: 18.09 Best Speedup vs Reference (x) (#3/12)
Kimi K3 on LLM Stats (CharXiv-R): 91.3 Score (%) (#2/45)
Kimi K3 on LLM Stats (FrontierSWE): 81.2 Score (%) (#2/14)
Kimi K3 on LLM Stats (GDPval-AA): 1668.0 ELO (#3/39)
Kimi K3 on LLM Stats (MCP Atlas): 84.2 Score (%) (#2/30)
Kimi K3 on LLM Stats (OfficeQA Pro): 63.3 Score (%) (#4/7)
Kimi K3 on LLM Stats (Terminal-Bench 2.1): 88.3 Score (%) (#2/13)
Kimi K3 on LLM Stats (Toolathlon): 73.2 Score (%) (#2/30)
Kimi K3 on LLM Stats (ZEROBench): 41.0 Score (%) (#3/9)
Kimi K3 on PM-LLM-Benchmark: 39.2 Score (#3/160)
Kimi K3 on RuneBench: 1891.0 Total Peak XP Rate (XP/min) (#21/33)
Kimi K3 on Surface Evolver Bench: 95.0 Mean Score (%) (#2/23)
Kimi K3 on Surface Evolver Bench Pass Rate: 87.5 Pass Rate (%) (#2/23)
Kimi K3 on SvelteBench: 95.6 Average pass@1 (%) (#25/152)
Kimi K3 on Vals AI (Vals Index): 74.7 Accuracy (%) (#2/37)
Kimi K3 on Vals AI CorpFin v2: 71.56 Accuracy (%) (#2/123)
Kimi K3 on Vals AI CyberBench: 79.03 Accuracy (%) (#4/17)
Kimi K3 on Vals AI Excel Modeling: 66.4 Accuracy (%) (#6/25)
Kimi K3 on Vals AI Finance Agent v2: 54.36 Accuracy (%) (#5/37)
Kimi K3 on Vals AI GPQA: 92.93 Accuracy (%) (#5/123)
Kimi K3 on Vals AI Harvey Legal Agent Bench: 10.83 Accuracy (%) (#4/24)
Kimi K3 on Vals AI Legal Research Bench: 44.23 Accuracy (%) (#3/24)
Kimi K3 on Vals AI LegalBench: 86.02 Accuracy (%) (#9/126)
Kimi K3 on Vals AI LiveCodeBench: 87.19 Accuracy (%) (#9/127)
Kimi K3 on Vals AI MMLU-Pro: 87.97 Accuracy (%) (#14/121)
Kimi K3 on Vals AI MMMU: 88.15 Accuracy (%) (#6/82)
Kimi K3 on Vals AI MedCode: 48.88 Accuracy (%) (#16/73)
Kimi K3 on Vals AI MedScribe: 87.96 Accuracy (%) (#4/72)
Kimi K3 on Vals AI MortgageTax: 66.34 Accuracy (%) (#30/86)
Kimi K3 on Vals AI Multimodal Index: 73.42 Accuracy (%) (#2/27)
Kimi K3 on Vals AI ProofBench: 70.0 Accuracy (%) (#5/51)
Kimi K3 on Vals AI SAGE: 54.26 Accuracy (%) (#4/68)
Kimi K3 on Vals AI SWE-bench Verified: 93.4 Resolved (%) (#3/72)
Kimi K3 on Vals AI TaxEval v2: 75.72 Accuracy (%) (#10/129)
Kimi K3 on Vals AI Terminal-Bench 2.1: 80.9 Accuracy (%) (#2/42)
Kimi K3 on Vals AI Vibe Code Bench: 84.96 Accuracy (%) (#2/73)
Kimi K3 on You're Absolutely Right!: 3.5 Average anti-sycophancy score (1-5) (#8/22)
Kimi K3 on ZeroEval GPQA Diamond: 93.5 GPQA Diamond Score (#7/232)

New #1 Leaders (49)

ExploitGym (Successful Intended Exploits (#)): GPT-5.6 Sol (293.0) beat Claude Mythos Preview (157.0) by 136.0.
GeoBench Photos (Average Score): Claude Fable 5 (4313.64) beat Gemini 3 Flash (Preview) (4204.26) by 109.38.
Design Arena (Video Editing) (Elo): Gemini 2.0 Flash (1417.0) beat happy-horse-1.0 (1318.0) by 99.0.
MineBench (Elo Rating): GPT-5.6 Sol (2108.79) beat GPT-5.5 Pro (2035.79) by 73.0.
GeoBench ACW (Average Score): Claude Fable 5 (4403.7) beat Gemini 3 Flash (Preview) (4332.77) by 70.93.
Design Arena (3D) (Elo): Kimi K3 (1441.0) beat GLM-5.2 (1371.0) by 70.0.
Design Arena (Data Viz) (Elo): Kimi K3 (1429.0) beat Claude Fable 5 (1372.0) by 57.0.
GDP.pdf (Strict Pass Rate (%)): Claude Fable 5 (Thinking, Max) (70.3) beat GPT-5.6 Sol (30.7) by 39.6.
Chatbot Arena (Code) (Elo): Kimi K3 (1679.0) beat Claude Fable 5 (1649.0) by 30.0.
Benchmarks.bio - SpatialBench-Long (Pass Rate (%)): GPT-5.6 Sol (38.89) beat Gemini 3.5 Flash (11.11) by 27.78.
ClockBench (Accuracy (%)): GPT-5.6 Sol (Max) (66.7) beat GPT-5.4 (High) (50.6) by 16.1.
LLM Stats (Toolathlon) (Score (%)): Muse Spark 1.1 (75.6) beat Claude Opus 4.8 (59.9) by 15.7.
LLM Stats (MRCR v2 (8-needle)) (Score (%)): GPT-5.6 Sol (91.5) beat Claude Opus 4.6 (76.0) by 15.5.
LLM Stats (BabyVision) (Score (%)): Kimi K3 (85.7) beat Seed 2.1 Pro (73.7) by 12.0.
AI for Education Visual Reasoning - match (process) (Accuracy (%)): GPT-5.6 Sol (88.9) beat Gemini 3 Flash (77.8) by 11.1.
LLM Stats (Graphwalks BFS >128k) (Score (%)): GPT-5.6 Sol (90.7) beat Claude Mythos Preview (80.0) by 10.7.
Android Bench (Score (%)): Claude Fable 5 (84.5) beat GPT-5.5 (74.0) by 10.5.
AI for Education Visual Reasoning - odd one out (Accuracy (%)): GPT-5.6 Sol (88.3) beat Gemini 3.5 Flash (80.5) by 7.8.
Lean AI Formalization Leaderboard (Solved Problems): Seed Prover (ByteDance) (129.0) beat Aristotle (Harmonic) (122.0) by 7.0.
ARC-AGI-3 (Accuracy (%)): GPT-5.6 Sol (Max) (7.78) beat Claude Opus 4.8 (High) (1.52) by 6.26.
TrackingAI IQ Test (Offline) (Offline IQ Score (%)): GPT-Terra-Ultra (87.5) beat GPT-5 Pro (81.25) by 6.25.
Design Arena (Website) (Elo): GPT-5.6 Sol (1352.0) beat GLM-5.2 (1346.0) by 6.0.
AgenticVBench (Average Success (%)): GPT-5.6 Sol (38.4) beat Claude Fable 5 (32.4) by 6.0.
LLM Stats (Terminal-Bench 2.1) (Score (%)): GPT-5.6 Sol (88.8) beat Claude Fable 5 (84.3) by 4.5.
LLM Stats (MCP Atlas) (Score (%)): Muse Spark 1.1 (88.1) beat Seed 2.1 Pro (83.8) by 4.3.
TaxCalcBench (Correct Returns - Strict (%)): GPT-5.6 Sol Web Search (58.0) beat GPT-5.5 Web Search (54.0) by 4.0.
LLM Stats (APEX-Agents) (Score (%)): Kimi K3 (37.6) beat Seed 2.1 Pro (33.8) by 3.8.
WorldArena Track 1 - Motion Quality (Score (%)): AccioWM (93.29) beat SUSWM (89.76) by 3.53.
LLM Stats (MathVision) (Score (%)): Kimi K3 (97.8) beat Seed 2.1 Pro (94.5) by 3.3.
WeirdML (Average Score): Claude Fable 5 (Max) (91.94) beat GPT-5.6 Sol (High) (88.76) by 3.18.
SEAL - AudioMultiChallenge (Score): Inkling (Thinking) (56.64) beat Gemini 3 Pro (Preview) (Thinking) (54.65) by 1.99.
SEAL - AudioMultiChallenge - Text Output (Score): Inkling (Thinking) (56.64) beat Gemini 3 Pro (Preview) (Thinking) (54.65) by 1.99.
LLM Stats (DeepSearchQA) (Score (%)): Kimi K3 (95.0) beat Claude Opus 4.8 (93.1) by 1.9.
WorldScore - Dynamic (Dynamic Score): WorldScape-0.2(MoE) (76.23) beat World Dreamer (74.35) by 1.88.
Vals AI Legal Research Bench (Accuracy (%)): Claude Fable 5 (49.52) beat GPT-5.6 Sol (48.08) by 1.44.
WorldArena Track 1 (EWMScore (%)): UNIS (73.64) beat PAIWorld (72.31) by 1.33.
Vals AI Excel Modeling (Accuracy (%)): Claude Fable 5 (73.67) beat GPT-5.6 Sol (72.34) by 1.33.
WorldScore (Average Static/Dynamic Score): WorldScape-0.2(MoE) (80.68) beat World Dreamer (79.44) by 1.24.
LLM Stats (BrowseComp) (Score (%)): Kimi K3 (91.2) beat GPT-5.5 Pro (90.1) by 1.1.
AI for Education Visual Reasoning - pattern completion (linear) (Accuracy (%)): GPT-5.6 Sol (92.3) beat Gemini 3.5 Flash (91.5) by 0.8.
WorldArena Track 1 - Controllability (Score (%)): UNIS (91.6) beat ABot-PhysWorld (text) (90.84) by 0.76.
Terminal-Bench 2.1 (Claude Code) (Accuracy (%)): Claude Fable 5 (83.8) beat Claude 5 Fable (83.1) by 0.7.
WorldScore - Static (Static Score): WorldScape-0.2(MoE) (85.13) beat World Dreamer (84.52) by 0.61.
Tau3 Banking (Success Rate (%)): Kimi K3 (33.4) beat GPT-5.6 Sol (Max) (32.99) by 0.41.
Terminal-Bench 2.1 (Accuracy (%)): Claude Fable 5 (83.8) beat GPT-5.5 (83.4) by 0.4.
Agent Arena - Confirmed Success (Confirmed Success (%)): Claude Fable 5 (High) (17.27) beat MiniMax-M2.7 (17.12) by 0.15.
MMEB Leaderboard (Overall Score (%)): DME-Large (80.24) beat octen-vl-embedding-large (80.09) by 0.15.
Vals AI Public Benefits Bench (Accuracy (%)): Kimi K3 (68.27) beat Claude Opus 4.8 (68.13) by 0.14.
CADGenBench (Aggregate CAD Score): GPT-5.6 Sol (xHigh) (0.5319) beat GPT-5.5 (xHigh) (0.46) by 0.07.

                                Don't miss what's next. Subscribe to The Aggregate Digest:

                    Email
                    *

                    ← Newer

                AI Benchmark Digest — 2026-07-20

                    Older →

                AI Benchmark Digest — 2026-07-18