AI Benchmark Digest — 2026-06-25

View on AI Benchmark Hub

Daily

New Benchmarks (87)

• ParallelKernelBench (Fast1@3 (% of problems)): GPT-5.5 leads with 31.03 across 6 models. ParallelKernelBench evaluates multi-GPU CUDA kernel generation, asking models to replace PyTorch plus NCCL references with direct NVLink communication kernels.
• ParallelKernelBench Pass@3 (Pass@3 (% of problems)): GPT-5.5 leads with 41.38 across 6 models. ParallelKernelBench pass@3 measures best-of-three correctness on multi-GPU CUDA kernels that replace PyTorch plus NCCL references.
• ParallelKernelBench Fast1@1 (Fast1@1 (% of problems)): GPT-5.5 leads with 25.29 across 6 models. ParallelKernelBench fast1@1 measures single-shot model outputs that are both correct and faster than the PyTorch plus NCCL baseline.
• ParallelKernelBench Pass@1 (Pass@1 (% of problems)): GPT-5.5 leads with 32.18 across 6 models. ParallelKernelBench pass@1 measures single-shot correctness on production-style multi-GPU kernel generation tasks.
• Surface Evolver Bench (Mean Score (%)): gpt-5.5 (high) leads with 89.29 across 14 models. Surface Evolver Bench evaluates agentic scientific simulation writing, with models creating liquid-surface physics datafiles and using tool feedback before hidden checks.
• Surface Evolver Bench Pass Rate (Pass Rate (%)): gpt-5.5 (high) leads with 78.57 across 14 models. Surface Evolver Bench pass rate measures fully passing agentic submissions for custom liquid-surface physics simulations.
• LLM2014 Code 2025-11 (Multi-round Score): Gemini 3 Pro leads with 96.0 across 26 models.
• LLM2014 Code 2025-11 - Python (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-11 - TypeScript (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-11 - Golang (Score): GPT-5 Mini (high) leads with 9.2 across 26 models.
• LLM2014 Code 2025-11 - C# (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-11 - Java (Score): Gemini 3 Pro leads with 9.67 across 26 models.
• LLM2014 Code 2025-11 - C++ (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-12 (Multi-round Score): Gemini 3 Pro leads with 96.0 across 26 models.
• LLM2014 Code 2025-12 - Python (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-12 - TypeScript (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-12 - Golang (Score): GPT-5 Mini (high) leads with 9.2 across 26 models.
• LLM2014 Code 2025-12 - C# (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2025-12 - Java (Score): Gemini 3 Pro leads with 9.67 across 26 models.
• LLM2014 Code 2025-12 - C++ (Score): Gemini 3 Pro leads with 10.0 across 26 models.
• LLM2014 Code 2026-01 (Multi-round Score): Gemini 3 Pro leads with 96.0 across 23 models.
• LLM2014 Code 2026-01 - Python (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-01 - TypeScript (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-01 - Golang (Score): GPT-5 Mini (high) leads with 9.2 across 23 models.
• LLM2014 Code 2026-01 - C# (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-01 - Java (Score): Gemini 3 Pro leads with 9.67 across 23 models.
• LLM2014 Code 2026-01 - C++ (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-02 (Multi-round Score): Gemini 3 Pro leads with 96.0 across 23 models.
• LLM2014 Code 2026-02 - Python (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-02 - TypeScript (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-02 - Golang (Score): GPT-5 Mini (high) leads with 9.2 across 23 models.
• LLM2014 Code 2026-02 - C# (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Code 2026-02 - Java (Score): Gemini 3 Pro leads with 9.67 across 23 models.
• LLM2014 Code 2026-02 - C++ (Score): Gemini 3 Pro leads with 10.0 across 23 models.
• LLM2014 Vision 2025-11 (Median Score): Gemini 3 Pro leads with 70.47 across 20 models.
• LLM2014 Logic 2024-05 (Score (%)): GPT-4 Turbo 0409 leads with 77.05 across 22 models.
• LLM2014 Logic 2024-06 (Score (%)): GPT-4 Turbo 0409 leads with 76.53 across 30 models.
• LLM2014 Logic 2024-07 (Score (%)): GPT-4 Turbo 0409 leads with 76.65 across 27 models.
• LLM2014 Logic 2024-08 (Score (%)): GPT-4 Turbo 0409 leads with 74.86 across 25 models.
• LLM2014 Logic 2024-09 (Score (%)): O1 Preview leads with 87.52 across 28 models.
• LLM2014 Logic 2024-10 (Score (%)): O1 Preview leads with 86.55 across 28 models.
• LLM2014 Logic 2024-11 (Score (%)): O1 Preview leads with 86.55 across 29 models.
• LLM2014 Logic 2025-11 (Median Score): GPT-5 (high) leads with 83.75 across 53 models.
• LLM2014 Logic 2025-12 (Median Score): GPT-5.2 (high) leads with 81.83 across 51 models.
• LLM2014 Logic 2026-01 (Median Score): GPT-5.2 (high) leads with 80.71 across 45 models.
• LLM2014 Logic 2026-02 (Median Score): Claude Opus 4.6 (Thinking) leads with 78.02 across 46 models.
• LLM2014 Logic 2026-03 (Median Score): GPT-5.4 (high) leads with 78.85 across 42 models.
• LLM2014 Logic 2026-04 (Median Score): GPT-5.5 (xhigh) leads with 83.96 across 42 models.
• LLM2014 Logic 2026-05 (Median Score): GPT-5.5 (xhigh) leads with 80.47 across 43 models.
• LLM2014 Logic 2026-06 (Median Score): GPT-5.5 (xhigh) leads with 80.47 across 42 models.
• HAL GAIA (Accuracy (%)): Claude Sonnet 4.5 (September 2025) leads with 74.55 across 32 models. Princeton HAL cost-aware agent leaderboard for GAIA multi-step web assistance tasks, reporting overall accuracy.
• HAL GAIA Level 1 (Accuracy (%)): Claude Sonnet 4.5 (September 2025) leads with 82.07 across 32 models. Princeton HAL GAIA level-1 slice, covering the easiest GAIA web assistance tasks.
• HAL GAIA Level 2 (Accuracy (%)): Claude Sonnet 4.5 High (September 2025) leads with 74.42 across 32 models. Princeton HAL GAIA level-2 slice, covering intermediate GAIA web assistance tasks.
• HAL GAIA Level 3 (Accuracy (%)): Claude Sonnet 4.5 (September 2025) leads with 65.39 across 32 models. Princeton HAL GAIA level-3 slice, covering the hardest GAIA web assistance tasks.
• HAL SciCode (Accuracy (%)): o4-mini Low (April 2025) leads with 9.23 across 33 models. Princeton HAL cost-aware agent leaderboard for SciCode scientific programming tasks.
• Wordle Arena (Win Rate (%)): Gemini 2.5 Pro leads with 100.0 across 49 models. Wordle Arena evaluates models on daily Wordle games, measuring lexical deduction and constraint tracking from public game logs.
• Fibble Arena (Win Rate (%)): Gemini 2.5 Pro leads with 80.0 across 47 models. Fibble Arena evaluates Wordle-style play when each clue can contain one lie, testing robust lexical reasoning under corrupted feedback.
• Fibble2 Arena (Win Rate (%)): Gemini 3.1 Pro leads with 50.0 across 46 models. Fibble2 Arena evaluates Wordle-style play with two lies per clue, increasing the need to reason through inconsistent feedback.
• Fibble3 Arena (Win Rate (%)): DeepSeek-R1 leads with 33.33 across 43 models. Fibble3 Arena evaluates Wordle-style play with three lies per clue, testing resilient hypothesis search under noisy constraints.
• Fibble4 Arena (Win Rate (%)): Gemini 3.1 Pro leads with 60.0 across 43 models. Fibble4 Arena evaluates Wordle-style play with four lies per clue, stressing deduction from heavily corrupted feedback.
• Fibble5 Arena (Win Rate (%)): Gemini 3.1 Pro leads with 58.33 across 46 models. Fibble5 Arena evaluates Wordle-style play with every clue position potentially deceptive, testing adversarial constraint reasoning.
• APEX v1 (Score (%)): GPT 5 leads with 67.0 across 7 models. Mercor APEX v1 evaluates professional task performance across expert-domain work samples.
• APEX v1 Consulting (Score (%)): Gemini 3 Flash leads with 64.0 across 3 models. Mercor APEX v1 consulting slice evaluates model performance on consulting-style professional reasoning tasks.
• APEX v1 Investment Banking (Score (%)): GPT 5.2 Pro leads with 64.0 across 3 models. Mercor APEX v1 investment-banking slice evaluates finance-focused professional work tasks.
• APEX v1 Medicine (MD) (Score (%)): GPT 5 leads with 66.0 across 3 models. Mercor APEX v1 medicine slice evaluates primary-care physician style professional tasks.
• BountyBench DetectWorkflow (Success Rate (%)): claude-opus-4-6 leads with 13.04 across 1 models. BountyBench DetectWorkflow evaluates cybersecurity agents on identifying exploitable bounty-style workflows.
• CocoaBench (Accuracy): CodeX leads with 45.1 across 10 models. CocoaBench evaluates autonomous agents on computer-control tasks, measuring successful completion across released aggregate runs.
• GSM-MC (Accuracy (%)): DeepSeek-V4-Flash-FP8 leads with 99.47 across 68 models. GSM-MC evaluates grade-school math reasoning in a multiple-choice format.
• HAL SWE-bench Verified Mini (Score (%)): Claude Sonnet 4.5 High (September 2025) leads with 72.0 across 18 models. HAL SWE-bench Verified Mini evaluates software issue resolution on a compact SWE-bench Verified subset.
• Journalistic Bias Accuracy (Accuracy (%)): GPT-4o leads with 44.44 across 7 models. Journalistic Bias accuracy evaluates classification of media bias labels in news-style examples.
• Journalistic Bias F1-macro (F1-Macro (%)): GPT-4o leads with 50.24 across 7 models. Journalistic Bias F1-macro evaluates balanced classification quality across media bias categories.
• JudgeBench Coding (Accuracy (%)): DeepSeek-R1-0528 leads with 97.62 across 52 models. JudgeBench coding slice evaluates judge-model accuracy on code-answer comparisons.
• JudgeBench Knowledge (Accuracy (%)): gemini-3.1-pro-preview leads with 91.88 across 52 models. JudgeBench knowledge slice evaluates judge-model accuracy on MMLU-Pro-derived knowledge comparisons.
• JudgeBench Math (Accuracy (%)): qwen3.6-plus leads with 96.43 across 52 models. JudgeBench math slice evaluates judge-model accuracy on mathematical answer comparisons.
• JudgeBench Reasoning (Accuracy (%)): DeepSeek-V3.2-Speciale leads with 96.94 across 52 models. JudgeBench reasoning slice evaluates judge-model accuracy on reasoning comparisons from LiveBench-style tasks.
• MATH-MC Level 1 (Accuracy (%)): Kimi-K2.5 leads with 99.3 across 69 models. MATH-MC Level 1 evaluates multiple-choice mathematical reasoning on the easiest MATH difficulty tier.
• MATH-MC Level 2 (Accuracy (%)): claude-opus-4.6 leads with 99.66 across 69 models. MATH-MC Level 2 evaluates multiple-choice mathematical reasoning on low-intermediate MATH problems.
• MATH-MC Level 3 (Accuracy (%)): Kimi-K2.5 leads with 99.73 across 69 models. MATH-MC Level 3 evaluates multiple-choice mathematical reasoning on intermediate MATH problems.
• MATH-MC Level 4 (Accuracy (%)): gemini-3.1-pro-preview leads with 99.58 across 69 models. MATH-MC Level 4 evaluates multiple-choice mathematical reasoning on advanced MATH problems.
• MATH-MC Level 5 (Accuracy (%)): Qwen3.5-122B-A10B leads with 99.92 across 69 models. MATH-MC Level 5 evaluates multiple-choice mathematical reasoning on the hardest MATH difficulty tier.
• RewardBench 2 Factuality (Accuracy (%)): gpt-5.5 leads with 88.21 across 52 models. RewardBench 2 factuality slice evaluates preference-model accuracy on factual response comparisons.
• RewardBench 2 Focus (Accuracy (%)): DeepSeek-V4-Flash-FP8 leads with 93.64 across 52 models. RewardBench 2 focus slice evaluates preference-model accuracy on responses that must stay on task.
• RewardBench 2 Math (Accuracy (%)): Qwen3.5-397B-A17B leads with 91.8 across 52 models. RewardBench 2 math slice evaluates preference-model accuracy on mathematical response comparisons.
• RewardBench 2 Precise IF (Accuracy (%)): gemini-3.1-pro-preview leads with 75.78 across 52 models. RewardBench 2 precise-instruction-following slice evaluates preference accuracy on tightly constrained instructions.
• RewardBench 2 Safety (Accuracy (%)): Qwen3-VL-235B-A22B-Thinking-FP8 leads with 96.44 across 52 models. RewardBench 2 safety slice evaluates preference-model accuracy on safety-sensitive response comparisons.
• ALL Bench LLM (Average Numeric Benchmark Score (%)): DeepSeek R2 leads with 85.76 across 39 models. Composite LLM leaderboard aggregating cross-verified scores across reasoning, knowledge, coding, and instruction-following evaluations.
• ALL Bench Multimodal (Average Numeric VLM Score (%)): GPT-5.2 leads with 86.7 across 16 models. Composite multimodal leaderboard aggregating model results across VLM, image generation, video generation, and agent-style multimodal evaluations.