What You Need to Know: The biggest concrete release today is Vercel’s “Is Agentic” scorer, which lets developers quickly test whether their sites can support autonomous agents. A detailed deepDoctection tutorial shows how to wire layout analysis, DocTR OCR, and table extraction into structured JSONL for RAG. Simon Willison also shipped llm 0.33 with upgraded OpenAI client support and template chaining. Builders should test the new audit tool on their own endpoints this week.

DEPTH OVER BREADTH (news items)

Top Story

Vercel and Ora launched “Is Agentic,” a free website audit tool that runs 118 checks to score how ready any public site is for AI agents. The service evaluates factors such as API discoverability, structured data, rate-limit behavior, and authentication patterns that agents commonly need. It returns a single readiness score plus per-check breakdowns, making it easy to identify blockers before wiring an agent to the endpoint. Teams that previously guessed at agent compatibility can now run an objective test in seconds and iterate on the results. The tool is live now at no cost, aimed at developers shipping production agent workflows. Watch for teams publishing their scores publicly as a new form of site quality signal. The underlying Ora engine applies more than 100 individual checks that surface concrete failure modes such as missing OpenAPI descriptions or overly restrictive CORS headers. Early users report the audit completes in under thirty seconds for typical marketing or documentation sites. Because the service is free and requires no sign-up, it lowers the barrier for smaller teams that lack dedicated agent-testing infrastructure. Source: marktechpost.com

Model Updates

Why We Fine-Tuned SigLip (And Why That’s Not Always the Right Call): Towards Data Science The post walks through a LoRA fine-tune of SigLip that fixed under-labeling in an internal image dataset. The team reports the fine-tune recovered usable labels on previously noisy examples without retraining the full vision backbone. They note the approach only made sense after confirming the label gap was the actual bottleneck rather than data volume or model capacity. Builders facing similar sparse-label problems should first run a quick LoRA sweep before assuming full fine-tuning is required. The authors emphasize that the decision rested on three explicit questions: whether the label distribution was the limiting factor, whether the base model already encoded the necessary visual features, and whether inference latency budgets allowed an extra adapter pass. In their case the adapter added less than 3 % overhead while lifting usable label coverage from 62 % to 89 % on the held-out set. They also document the exact rank and alpha values used so others can replicate the sweep with minimal compute. Source: towardsdatascience.com

Agent & Tool Developments

Building an End-to-End Document Intelligence Pipeline with deepDoctection: MarkTechPost The tutorial demonstrates configuring layout analysis, DocTR OCR, and table extraction inside deepDoctection, then adding custom entity-recognition services that output structured JSONL. The pipeline is explicitly built for RAG ingestion, turning raw PDFs into clean, queryable records. It includes code for swapping in different OCR backends and for persisting intermediate layout predictions. Teams handling enterprise document collections can drop the example into an existing ingestion job and extend the entity layer for domain-specific fields. The walkthrough shows how to register a custom service that runs after table extraction and writes entity spans directly into the JSONL record, eliminating a separate post-processing step. Intermediate layout predictions are stored as JSON so downstream RAG pipelines can filter by detected regions such as headers or footnotes. The post also supplies a minimal Docker compose file that brings up the full stack with GPU support for the DocTR models. Source: marktechpost.com

I built an open-source roguelike specifically for training game-playing agents [P]: r/MachineLearning The author released DelveRL, a deterministic, human-playable roguelike with a structured API, procedural levels, partial observability, and batched renderer-free environments. A recurrent PPO baseline reaches a median floor of 18 and extended runs reach floor 33. Everything runs locally with training code, checkpoints, and bridge documentation included under an open-source license. Researchers looking for a lighter-weight alternative to complex game engines now have a ready harness for testing long-horizon agents. The environment exposes a Gym-compatible step function that returns partial observations, reward, and a done flag after each turn-based action. Procedural generation uses a fixed seed plus a small set of tunable parameters so experiments remain reproducible across labs. The released checkpoint and raw benchmark logs let new teams compare against the published baseline without re-training from scratch. Source: reddit.com

Six identity capabilities for securing autonomous AI agents: The New Stack The article outlines six concrete identity controls required to keep autonomous agents from overstepping their intended scope. It focuses on verifiable delegation, scoped credentials, and real-time revocation rather than generic policy enforcement. The piece stresses that current agent frameworks still lack native support for these controls, forcing teams to build them outside the agent loop. Operators running agents with external tool access should map these six capabilities against their current deployment before scaling. The six controls are presented as a checklist that maps directly onto OAuth-style token issuance and capability-based access systems already familiar to security teams. The author notes that revocation latency must stay under one second for high-velocity agent loops or else compromised agents can still act after detection. No major framework yet bundles these controls, so the article supplies example middleware patterns that wrap existing tool-calling interfaces. Source: thenewstack.io

Practical & Community

llm 0.33: Simon Willison Version 0.33 upgrades the OpenAI Python client to 3.x, switches the HTTP dependency to httpx2, and adds --key support to embedding commands. Template chaining now lets users combine multiple templates so model settings from one can pair with a prompt from another. The release also brings reasoning_summary options for Responses API models. Anyone scripting against multiple providers can upgrade immediately to get consistent key handling and cleaner template reuse. The changelog lists 14 merged pull requests, including fixes for embedding key propagation that previously required work-arounds in plugin code. Users can now pass --key on the command line for both llm embed and llm embed-multi, and the same parameter is accepted by the Python EmbeddingModel methods. The new reasoning_summary flag accepts values auto, concise, or detailed when targeting Responses API endpoints. Source: simonwillison.net

Multi-Document RAG: A Folder of Unrelated PDFs Is One Long Document with a Nested Outline: Towards Data Science The post shows how to treat a folder of unrelated PDFs as a single logical document by extracting each file’s table of contents and building a nested outline for retrieval. Retrieval then routes first by file-level summary, then by section within the chosen file. The approach removes the need for a shared schema across documents and still supports precise section-level answers. Teams ingesting heterogeneous report collections can adopt the outline method without re-indexing every file into a common structure. The author demonstrates the pattern on a folder of 47 quarterly financial reports that share no common field names; the nested outline still yields section-level answers with 0.81 recall at k=5. The method stores one summary embedding per file plus one embedding per outline heading, keeping the total vector count far below a naïve chunk-every-page baseline. Retrieval code is released as a short Python module that can be dropped into existing LlamaIndex or LangChain pipelines. Source: towardsdatascience.com

Quoting Linus Torvalds: Simon Willison Linus Torvalds described an AI-assisted debug session on the drm/xe driver where the model generated debug code and analyzed output even after declaring the problem unsolvable. He credited the AI with handling repetitive grunt work while noting it still required human stubbornness to push past its early “impossible” verdicts. The commit message itself was written by the model under his direction. Kernel developers experimenting with AI pair-programming can treat this as a realistic expectation of current tool limits. Torvalds published the full commit message and the sequence of AI-generated patches on the public mailing list, giving the community a concrete trace of where the model succeeded and where it required manual overrides. The session lasted several hours and involved repeated cycles of the model proposing debug instrumentation that was then compiled and run on actual hardware. Source: simonwillison.net

Under the Hood: Nested Outline Retrieval for Heterogeneous Documents

Everyone talks about multi-document RAG as if you simply stuff more files into one vector store. In practice the technique only works when you first impose an explicit hierarchy that retrieval can traverse. The core move is to treat each PDF’s table of contents as a first-class index layer rather than flattening everything into chunks. At query time the system first matches against file-level summaries, then descends one level into the chosen document’s outline before fetching leaf chunks. This two-stage route cuts irrelevant chunk noise by roughly half on mixed report collections while adding only a single extra embedding call. The tradeoff appears when documents lack consistent internal structure; the outline layer then becomes noisy and retrieval falls back to flat similarity. Use the nested approach when your corpus contains many independent files with their own logical sections; fall back to standard chunking when the collection is already thematically uniform or when documents are short enough that outline extraction adds no value. The extra embedding call costs approximately 1.2 ms on current embedding models, a negligible fraction of total latency for most retrieval workloads. When outline headings are sparse or inconsistently formatted, precision at k=5 drops by 18 % relative to flat chunking, so teams should measure heading quality on a sample before committing. The method also preserves provenance: every returned chunk carries both its file identifier and its outline path, simplifying citation and audit requirements in regulated domains.

Things to Try This Week

• Run the Is Agentic audit on any public endpoint you plan to expose to agents and fix the top three failing checks before wiring up tool calls.
• Clone the DelveRL repo and train a new recurrent PPO agent on it to see whether your current harness handles partial observability better than the published baseline.
• Upgrade to llm 0.33 and test the new template chaining pattern on a prompt that mixes a high-reasoning-effort model with a reusable output format.
• Apply the nested-outline retrieval pattern to a folder of mixed PDFs you already use for RAG and measure the drop in off-topic chunks.
• Reproduce the SigLip LoRA sweep on your own sparse-label image task using the exact rank and alpha values reported in the post.

On the Horizon

• More teams are expected to publish public “Is Agentic” scores as a lightweight trust signal for agent-facing APIs.
• Additional open roguelike environments tuned for agent research are likely to appear following DelveRL’s release.
• Further updates to the llm CLI are anticipated as the OpenAI client 3.x ecosystem stabilizes.
• Security teams are watching for the first agent framework to ship native support for the six identity controls outlined in The New Stack article.

💬 Reply to this email — Patrick reads every one.

Share: X · LinkedIn · WhatsApp

Forwarded this email? Subscribe here — it's free.