AI Research Brief

Archives
Log in
Subscribe
October 3, 2026

Mean OSWorld Task Score Rises From 33.0% to 40.8%

  • Post-Training Can Improve One-Shot Accuracy While Shrinking Solution Coverage. Across 42 comparisons spanning four model families and three agent benchmarks, pass@1 and pass@K sometimes moved in opposite directions. With sufficient test-time budget, base models often surpassed their post-trained counterparts in pass@K solution coverage.
  • Image Generators Can Simulate GUI Environments. AutoGUIWorld creates interaction traces without installing software. Fine-tuning raised the average OSWorld task score from 33.0% to 40.8%, but reliable state transitions, action targets, and quality filters remain essential.
  • 4Director Turns Directorial Intent Into Executable 3D Constraints. It reuses a canonical mesh and applies rigid transformations frame by frame, improving cross-view consistency. The generative model still handles non-rigid changes.

Also Notable

  • Continuous Latents Become the Only Persistent State, While Discrete Tokens Add Scaffolding. HC-DLM targets both broken dependencies in parallel decoding and validity gaps in continuous representations.
  • More Distinct Text Embeddings Are Not Always Better for Diffusion. Soft-label distillation brings plausible substitute words closer, creating a more connected latent space that is easier to sample.
  • Long-Horizon Agents Need More Than Compressed History. PoS explicitly tracks the current world, unresolved requirements, and stalled states. It checks belief consistency and selects recovery actions for different failure modes.
  • Training Scenarios Should Change With Agent Failure Modes. ActiveSaddler's non-stationary curriculum improved test Pass@1 by 4.4 percentage points on GAIA2 and 7.5 points on Terminal-Bench 2.0.
  • Fitting a Catalog Into Long Context Does Not Make It Usable. RPTune first learns product ranking and pruning, then applies post-training with rewards relative to the current context.
  • Execution Traces Can Test Whether Videos Follow Rules. PROWBench uses program traces and timestamped events to move world-model evaluation from visual plausibility toward faithful execution.
  • Long Audio-Video Reasoning Does Not Require Watching Everything. OmniSeek decides when to look and when to listen. Its rewards discourage shortcuts that solve tasks through only one modality.
  • Compiling Lean Code Does Not Prove the Right Claim. Isolated execution, statement comparison, assumption audits, and independent review turn successful outputs into traceable evidence.
  • Hybrid Transformer-SSM Architectures Can Reuse μP Learning Rates. Experiments span widths from 256 to 2,048, depths from 4 to 32, and billion-parameter models. Empirical results currently run ahead of theory.
  • VideoLLMs Learn Temporal Order, Then Lose It Across Layers. Relevant representations peak in intermediate layers before fading. Reinjecting temporal-difference vectors at inference partially restores temporal reasoning without training.
  • A Shared SE(3) Trajectory Representation Can Cover Objects, People, Cameras, and Robots. World Motion Models use different masks to support prediction, completion, control, and cross-embodiment transfer.
  • Open-Weight Models Still Struggle With Exact Kali Commands. Across 1,642 Kali tools, none exceeded 42% exact-command accuracy without explicit tool hints. KaliBench also provides verifiable training rewards without running commands.

Read the full edition →

Don't miss what's next. Subscribe to AI Research Brief:
← Newer Pivot-SD Wins With 200 Problems; Queen Gains 915 Elo Older → Chunked KV Compression Shows Retrieval Gaps of Up to 40 Percentage Points
Powered by Buttondown, the easiest way to start and grow your newsletter.