What You Need to Know: OpenAI used an internal version of Astra to tackle problems untouched for at least ten years, publishing Lean 4 formalizations and an LLM-generated reconstruction of the proofs. AMD released a fully open 16B MoE model with 2.8B active parameters trained on Instinct GPUs and published every training stage. NVIDIA shipped Molt, a compact PyTorch-native agentic RL framework built around an asynchronous loop with Ray, vLLM, and NeMo. Developers should watch how these releases affect both research tooling and production agent stacks this week.

DEPTH OVER BREADTH (news items)

Top Story

OpenAI directed an internal version of Astra at ten mathematical problems that had seen no progress for at least a decade. The effort cost less than $2,000 per problem at GPT-5.6 Sol token prices and produced Lean 4 formalizations now available in the openai/ten-proofs repository along with a paper and an LLM-generated PDF reconstructing the reasoning. The results add to a pattern of frontier labs using large token budgets for genuine research rather than benchmark tuning. Mathematicians are reacting with a mix of excitement and unease, echoing earlier responses to Deep Blue. Builders working on formal verification or automated theorem proving now have concrete artifacts to inspect. Watch for whether other labs publish similar internal runs and whether the prompts or reasoning traces are released. Source: simonwillison.net

Model Updates

AMD Releases Instella-MoE-16B-A3B: A Fully Open Mixture-of-Experts LLM With 2.8B Active Parameters Trained On Instinct GPUs: MarkTechPost AMD released Instella-MoE-16B-A3B, a 16B-parameter Mixture-of-Experts model that activates 2.8B parameters per token using Gated MLA and FarSkip-Collective. The model was trained from scratch on Instinct MI300X and MI325X GPUs. AMD published weights from every training stage, data mixtures, configs, and inference code under a fully open license. This gives builders direct access to intermediate checkpoints for studying MoE training dynamics. Teams evaluating open-weight alternatives to closed frontier models should test Instella-MoE on their specific workloads this week. Source: marktechpost.com

Accelerating Transformer Training with NVIDIA Transformer Engine, Fused Kernels, BF16, FP8, and GPU Benchmarking: MarkTechPost The tutorial walks through configuring the NVIDIA Transformer Engine to use fused GPU kernels, FP8 delayed scaling, and BF16 for GPT-style causal language models in PyTorch. It includes practical code examples and performance analysis on current GPUs. Developers can apply the same patterns to reduce training time and memory footprint without changing model architecture. The post emphasizes measurable throughput gains from the fused kernels and scaling techniques. Anyone running custom transformer training should try the FP8 path on supported hardware. Source: marktechpost.com

Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks: MarkTechPost Supabase open-sourced supabase/evals, an Apache-2.0 benchmark that runs coding agents against real Supabase tasks such as building schemas, debugging Edge Functions, and fixing RLS policies inside containerized stacks. Scoring uses deterministic checks plus LLM-as-a-judge. The framework currently evaluates Claude Code, Codex, and OpenCode. Teams building or choosing coding agents for database work now have a reproducible way to compare them on production-like tasks. Source: marktechpost.com

Agent & Tool Developments

NVIDIA AI Releases Molt: A PyTorch-Native Agentic Reinforcement Learning Framework: MarkTechPost NVIDIA released Molt, an 8.6K-line PyTorch-native framework for agentic RL research that composes Ray, vLLM, and NeMo AutoModel around a single asynchronous loop. The agent remains ordinary Python, trajectories stay token-exact, and throughput matches a Megatron-based stack. This removes the need to thread every algorithm change through trainer, distributed backend, and rollout glue. Researchers modifying RL algorithms can now iterate faster without touching low-level infrastructure. Source: marktechpost.com

datasette-apps 0.2a0: Simon Willison The 0.2a0 release adds app_debug() and app_list() tools that let Datasette Agent open apps invisibly, run JavaScript tests inside a sandboxed iframe, and list apps the user can edit. The iframe uses opacity:0 and pointer-events:none so the agent can measure elements and smoke-test functionality without visual interaction. This builds on the context.browser_task() mechanism from datasette-agent 0.4a0. Developers using Datasette for agent-driven app creation now have a practical way to verify edits automatically. Source: simonwillison.net

Coding Agents Don’t Need Bigger Context Windows — They Need a Context Compiler: Towards Data Science The article argues that coding agents suffer when context grows because irrelevant code competes for attention and agents begin compressing their own memory mid-task. It proposes treating prompt construction like a compiler that decides what to keep, reduce, or discard. The approach aims to prevent the “forgetting” that appears when windows fill. Builders hitting context limits on long codebases should experiment with explicit compilation steps before simply requesting larger windows. Source: towardsdatascience.com

Put the Agent Inside the Workflow: Towards Data Science The post describes a hybrid pattern that embeds adaptive agent behavior inside a predefined workflow rather than giving the agent full control. This keeps the reliability of structured flows while allowing the agent to handle exceptions and variations. The pattern is positioned as a practical middle ground between rigid pipelines and fully autonomous agents. Teams building production agent systems should test whether wrapping agents in workflows reduces failure modes on their tasks. Source: towardsdatascience.com

Practical & Community

How Symmetric Are the Insides of a Go Network? [R]: r/MachineLearning A new interpretability study on KataGo examines whether superhuman Go networks learn rotation- and reflection-invariant internal representations or memorize orientations separately despite 8-fold data augmentation. The work was largely AI-driven with human oversight and includes linked code plus an accessible write-up. Researchers studying symmetry in vision or game models can use the same methodology on other domains. Source: reddit.com

CausalVLBench: Benchmarking Visual Causal Reasoning in Large VLMs: r/MachineLearning The new benchmark evaluates visual causal reasoning capabilities in large vision-language models. It was posted to r/MachineLearning with links and discussion for further details. Teams working on multimodal reasoning now have a targeted test for causal understanding beyond standard VQA or captioning tasks. Source: reddit.com

Under the Hood: Context Compilation for Coding Agents

Everyone talks about giving coding agents more context as if bigger windows are the only lever. In practice, the real engineering problem is deciding what belongs in the prompt at all. A context compiler treats the prompt like a build artifact: it runs retrieval, then applies reduction rules that drop low-relevance files, summarize others, and keep only the symbols the model is likely to edit. This adds a small preprocessing step but prevents the model from diluting attention across thousands of irrelevant tokens. The quality gain is largest on repositories above roughly 50k lines; below that threshold the overhead rarely pays off. When the compiler also tracks which files changed in the last edit cycle, it can further prune the window by 30-40% with almost no drop in task success. The gotcha that bites most teams is treating the compiler as a one-time filter instead of re-running it after every agent action; stale context is what produces the “it forgot what it just did” failures. Use a compiler when your codebase exceeds a few hundred files or when agents start mid-task compression; otherwise simple retrieval plus a hard token cap is still simpler and sufficient.

Things to Try This Week

• Try AMD’s Instella-MoE-16B-A3B on a coding or reasoning task you currently run on a dense 7B or 13B model to see whether the sparse activation pattern changes latency or quality for your workload.
• Run Supabase’s new evals harness against Claude Code and any other coding agent you are evaluating on real schema and Edge Function tasks.
• Experiment with the context-compiler pattern from the Towards Data Science post on a repository larger than 50k lines before simply increasing the model’s context window.
• Test NVIDIA Molt’s asynchronous loop on a small agentic RL experiment if you have been modifying RL algorithms and want to avoid threading changes through Megatron-style infrastructure.

On the Horizon

• More labs are expected to publish internal research runs similar to OpenAI’s ten-proofs effort as the cost of long-horizon model use continues to drop.
• Additional open-weight MoE releases from AMD and others will likely include full training artifacts, increasing the data available for studying sparse training dynamics.
• Agent evaluation frameworks like Supabase Evals are likely to expand to more real-world task domains beyond database work.

💬 Reply to this email — Patrick reads every one.

Share: X · LinkedIn · WhatsApp

Forwarded this email? Subscribe here — it's free.