AI Benchmark Digest — 2026-06-23

View on AI Benchmark Hub

Daily

New Benchmarks (23)

• FutureSearch BTF-3 (Pooled Brier/RPS Score): FutureSearch SOTA leads with 0.116 across 7 models. FutureSearch Bench to the Future 3 pooled pastcasting score across binary and numeric forecasting questions.
• FutureSearch BTF-3 Binary (Binary Brier Score): FutureSearch SOTA leads with 0.114 across 7 models. FutureSearch Bench to the Future 3 binary-question slice scored by Brier score.
• FutureSearch BTF-3 Numeric (Numeric RPS): GPT-5.5 leads with 0.122 across 7 models. FutureSearch Bench to the Future 3 numeric-question slice scored by ranked probability score.
• FutureSearch DRB (Average Score): Claude Opus 4.6 leads with 0.553 across 22 models. FutureSearch Deep Research Bench aggregate score for open-web research agents across task categories.
• FutureSearch DRB - Find Number (Average Score): Claude Opus 4.6 leads with 0.738 across 22 models. FutureSearch Deep Research Bench task slice for finding specific numeric answers from web evidence.
• FutureSearch DRB - Derive Number (Average Score): Claude Sonnet 4.6 leads with 0.477 across 22 models. FutureSearch Deep Research Bench task slice for deriving numeric answers from gathered evidence.
• FutureSearch DRB - Find Dataset (Average Score): Claude Opus 4.1 leads with 0.708 across 22 models. FutureSearch Deep Research Bench task slice for locating relevant datasets on the web.
• FutureSearch DRB - Compile Dataset (Average Score): Claude Sonnet 4.6 leads with 0.754 across 22 models. FutureSearch Deep Research Bench task slice for compiling structured datasets from web sources.
• FutureSearch DRB - Populate Reference Class (Average Score): Claude Opus 4.6 leads with 0.398 across 22 models. FutureSearch Deep Research Bench task slice for building reference classes from researched examples.
• FutureSearch DRB - Gather Evidence (Average Score): Claude Opus 4 leads with 0.395 across 22 models. FutureSearch Deep Research Bench task slice for gathering supporting evidence from web sources.
• FutureSearch DRB - Validate Claim (Average Score): Claude Sonnet 4.6 leads with 0.799 across 22 models. FutureSearch Deep Research Bench task slice for validating claims against web evidence.
• FutureSearch DRB - Find Original Source (Average Score): Claude Opus 4.5 leads with 0.591 across 22 models. FutureSearch Deep Research Bench task slice for tracing facts back to original sources.
• FutureSearch BTF-2 (Brier Score): FutureSearch Agent leads with 0.119 across 5 models. FutureSearch Bench to the Future 2 pastcasting benchmark scored by Brier accuracy on resolved forecasting questions.
• FutureSearch BTF-2 Calibration (Calibration Error): FutureSearch Agent leads with 0.002 across 5 models. FutureSearch Bench to the Future 2 calibration-error slice for forecasting agents.
• FutureSearch BTF-2 Refinement (Refinement): FutureSearch Agent leads with 0.081 across 5 models. FutureSearch Bench to the Future 2 refinement slice measuring forecast sharpness and discrimination.
• Epoch AI - Rli (Score): claude-opus-4-6_unknown leads with 4.17 across 8 models.
• Epoch AI - Algotune (Score): gpt-5.2-2025-12-11_medium leads with 2.05 across 18 models.
• Epoch AI - Vending Bench 2 (Score): gpt-5.5_unknown leads with 10626.96 across 45 models.
• Tau3 Banking (Success Rate (%)): GPT-5.5 (xhigh) leads with 31.34 across 64 models. Artificial Analysis Tau3 banking-domain customer-service tasks, measuring agent success at policy-grounded banking support workflows.
• AA-Omniscience Accuracy (Accuracy (%)): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) leads with 61.35 across 414 models. Artificial Analysis Omniscience accuracy on factual-recall questions across law, health, business, software engineering, humanities, and STEM.
• MathArena Arxiv False (Accuracy (%)): GPT-5.5 (xhigh) leads with 50.0 across 9 models. MathArena arXiv false-premise mathematics slice testing whether models avoid solving invalid or inconsistent research-style problems.
• MathArena Arxiv (Accuracy (%)): Claude-Fable-5 (max) leads with 86.67 across 9 models. MathArena arXiv mathematics competition slice with research-style final-answer problems from recent arXiv-derived tasks.
• HiL-Bench (Combined Pass@3 (%)): GPT-5.5 leads with 29.1 across 12 models. Scale Human-in-Loop benchmark measuring when agents should ask for help, escalate uncertainty, or continue autonomously.

New Scores From Top-10 Models (3)

• Claude Fable 5 on DeepSWE: 69.9 Pass@1 (%) (#1/10)
• Claude Opus 4.8 on DeepSWE: 59.0 Pass@1 (%) (#4/10)
• GPT-5.5 on DeepSWE: 64.4 Pass@1 (%) (#3/10)

New #1 Leaders (1)

• WebDev Arena (Arena Score): Claude 5 (1653.93) beat Claude Opus 4.7 (Unknown) (1566.85) by 87.08.