AI Benchmark Digest — 2026-06-20
AI Benchmark Digest — 2026-06-20
Daily
New Benchmarks (18)
- SQL Capability Leaderboard (Average Ability Score): SQLShift leads with 83.4 across 35 models. Monthly SQL capability leaderboard measuring SQL understanding, optimization, and dialect conversion ability scores for LLMs and SQL-focused applications.
- SQL Capability - Dialect Conversion (Ability Score): SQLShift leads with 83.4 across 34 models. SQL Capability leaderboard slice measuring conversion of SQL queries across database dialects.
- SQL Capability - SQL Optimization (Ability Score): SQLFlash leads with 72.1 across 34 models. SQL Capability leaderboard slice measuring query optimization, index advice, and semantic-preserving SQL improvements.
- SQL Capability - SQL Understanding (Ability Score): Gemini 3 Pro leads with 86.0 across 33 models. SQL Capability leaderboard slice measuring SQL execution understanding, explanation, and syntax-error detection.
- Software Engineering Arena - Model Arena (Elo Rating): MiMo-V2-Flash leads with 1002.0 across 23 models. Software Engineering Arena model-only preference leaderboard for software-engineering task performance.
- boogiebench (Elo Rating): gemini-3-pro-preview leads with 1683.0 across 13 models. Elo-ranked LLM music-composition arena where text models write Strudel JavaScript compositions from music prompts and users vote on the generated tracks.
- Lean AI Formalization Leaderboard (Solved Problems): Aristotle (Harmonic) leads with 92.0 across 29 models. Submission-based Lean formalization leaderboard for hard mathematical problems where accepted solutions must pass automated comparator verification.
- WorldCupBench (Quiniela Points): MiMo-V2.5-Pro leads with 20.0 across 11 models. Live 2026 World Cup forecasting benchmark where frontier models submitted frozen tournament predictions and are scored by match-prediction points and Brier score as results arrive.
- WorldCupBench - Brier Skill (100 - Brier Total): Gemini-3.5-Flash leads with 92.0625 across 11 models. WorldCupBench live tournament forecasting score transformed as 100 minus total Brier score, so higher is better.
- SWE-bench Multilingual (Mythos Preview System Card) (Resolved (%)): Claude Mythos Preview leads with 87.3 across 2 models. Claude Mythos Preview system-card run of SWE-bench Multilingual, measuring multilingual software issue resolution.
- SWE-bench Multimodal (Mythos Preview System Card) (Resolved (%)): Claude Mythos Preview leads with 59.0 across 2 models. Claude Mythos Preview system-card run of SWE-bench Multimodal, measuring software issue resolution when visual context is part of the task.
- Terminal-Bench 2.0 (Mythos Preview System Card) (Mean Reward (%)): Claude Mythos Preview leads with 82.0 across 4 models. Claude Mythos Preview system-card run of Terminal-Bench 2.0, measuring terminal-based agent task completion with the Terminus harness.
- USAMO 2026 (Mythos Preview System Card) (Score (%)): Claude Mythos Preview leads with 97.6 across 4 models. Claude Mythos Preview system-card evaluation on 2026 USAMO proof problems, scored with MathArena-style model judging.
- LAB-Bench FigQA (Mythos Preview No Tools) (Accuracy (%)): Claude Mythos Preview leads with 79.7 across 3 models. Claude Mythos Preview system-card LAB-Bench FigQA result without tools, measuring biology-figure visual reasoning.
- LAB-Bench FigQA (Mythos Preview Tools) (Accuracy (%)): Claude Mythos Preview leads with 89.0 across 3 models. Claude Mythos Preview system-card LAB-Bench FigQA result with Python/image tools, measuring biology-figure visual reasoning.
- ScreenSpot-Pro (Mythos Preview No Tools) (Accuracy (%)): Claude Mythos Preview leads with 79.5 across 3 models. Claude Mythos Preview system-card ScreenSpot-Pro result without tools, measuring GUI grounding in high-resolution professional screenshots.
- ScreenSpot-Pro (Mythos Preview Tools) (Accuracy (%)): Claude Mythos Preview leads with 92.8 across 3 models. Claude Mythos Preview system-card ScreenSpot-Pro result with Python/image tools, measuring GUI grounding in high-resolution professional screenshots.
- Google Gemini 3 Deep Think - GPQA Diamond (Score (%)): Gemini 3 Deep Think leads with 93.8 across 2 models. Google Gemini 3 launch result for GPQA Diamond, reporting graduate-level science reasoning accuracy for Gemini 3 Deep Think and Gemini 3 Pro.
New #1 Leaders (1)
- NYT Connections Older Models (Score (%)): GLM-5.2 (92.7) beat Sherlock Think Alpha (92.5) by 0.2.
Don't miss what's next. Subscribe to Mikhail Doroshenko: