250K Deployment Traces Become Tests, VQS Answers Reach 94%
- Production Failures Can Become Regression Tests: TraceDance turns 252,557 real agent sessions into 107 targeted benchmarks. Both annotators confirmed the target behavior in only 84% of sampled cases, while nine frontier models averaged a 26.7% pass rate.
- YuE2 Turns Readable Scores Into a Song-Generation Interface: One model can plan, generate, and locally edit music. Its standard WildSongBench result is 6.73; the 6.96 score comes from selecting the best of eight candidates.
- Program Verification Raises Self-Generated Answer Accuracy From 76% to 94%: VQS converts images into structured records, then uses fixed programs to write questions and calculate answers. Qwen3-VL gains up to 3.18 points across ten benchmarks.
Also Notable
- 448 Reusable Services Create a Cross-Tool Task World — CompoWorld composes 448 reusable services exposing 10,130 tools to generate and verify tasks in which information flows across services.
- Reward Models Learn Multimodal Preference Distributions — DRM stops compressing judgments into one score. It uses variance, quantiles, and lower confidence bounds to support steadier RLHF decisions.
- Reconstruct Geometry Before Training Spatial Reasoning — SpatialSpeak directly supervises intermediate geometry estimates through question answering, expanding CoT-VC's gain on ReVSI from 2.6 to 6.9 points.
- 24 Matched Comparisons Separate Appearance From Motion — TT-VidT strengthens motion-sensitive representations with fewer encoder FLOPs, though results on HMDB51, IARD, and EPIC-Kitchens limit the claim's scope.
- Generalization May Jump Between Two Compute Modes — The authors hypothesize that shallow patterns and general computation compete for capacity; empirically, intermediate checkpoints can outperform final checkpoints on reasoning and alignment.
- Relational Algebra Becomes a Portable SQL Interface — The model generates a dialect-independent plan, then a deterministic compiler produces SQL. Fine-tuning on relational algebra plans avoids the small in-dialect accuracy loss seen in capable prompted models.
- Theorem Provers Find Missing and Extra Logic — SIV generates positive and contrastive probes. Under controlled perturbations, more than 99% of example pairs rank the reference above the mistranslation.
- Branching Replays Create Process Signals for Software Agents — CRR replaces some step-level advantages with terminal reward differences. “Free” means no human process labels or learned reward model, not zero replay compute.
- No Foundation Model Family Wins Across Retinal Datasets — FOCUS tests ranking, calibration, subgroup differences, and image-quality resilience across ten datasets. External transfer after fine-tuning remains uneven.
- Spatial Gene Prediction Targets Differential Rankings Directly — IDER aligns training with downstream gene discovery and pathway enrichment instead of optimizing only per-gene expression reconstruction.
Don't miss what's next. Subscribe to AI Research Brief: