Nerra Network

Archives
Log in
Subscribe
October 18, 2026

Local models just matched a frontier coding agent on… · M&A 🤖

View this email in your browser
Models & Agents — Daily AI models, agents, and practical developments.

Models & Agents

Daily AI models, agents, and practical developments.

Ep 207 · Oct 18, 2026

🎧 Today's episode
Episode 207 · Local models just matched a frontier coding agent on five real tasks while running on a single 16 GB card.
2026-10-18
▶ Listen now
Local models just matched a frontier coding agent on five real tasks while running on a single 16 GB card.

What You Need to Know: Nine models tackled five coding problems on one RX 9060 XT; DeepSeek-V4.1-Flash and a Qwen3.8-27B variant both scored 5/5 while the rest fell short on interval scheduling or prefix matching. NVIDIA released Structured Data Models, a PyTorch library that wraps TabICLv2 and KumoTabular for in-context tabular predictions with no training step. Voxel51 3.0 added an agent layer with 45+ MCP tools for multimodal sensor data curation aimed at robotics and physical AI teams. The 1017 DeepSeek-V4.1-Flash checkpoint altered 40 % of routed experts, 30 % of engrams and 50 % of the vision tower while preserving the prior architecture. A 5070 Ti plus 3070 rig reached 82 tok/s on a 35 B MoE at 16 k context with layer splitting.

Top Story

NVIDIA released Structured Data Models, a new library that exposes several pretrained tabular foundation models through a single PyTorch interface for zero-training predictions. The library wraps TabICLv2 and KumoTabular for single tables plus KumoRelational for full relational databases, with every model answering via in-context learning by receiving labeled rows and query rows in one forward pass. Users can reuse an encoded context across many queries, read calibrated prediction intervals from regression outputs, and scale to twenty-class problems. The tutorial walks through typed table tensors, comparison against scikit-learn baselines, a three-table shop database example, and common preprocessing pitfalls such as dropped timestamp columns. The install step requires a specific git commit because the wheel URL construction inside the package mismatches the actual torch minor version wheels published for pyg-lib. Builders working with structured data can now test foundation-model predictions without fitting any weights. The relational sampler path needs the correct pyg-lib wheel built for the exact major.minor torch version rather than the patch version the library hard-codes. Source: marktechpost.com


Model Updates

DeepSeek V4.1 Flash 1017 for Dwarfstar (ds4): r/LocalLLaMA DeepSeek released the 1017 checkpoint of V4.1-Flash, a major revision that changed roughly 40 % of routed expert tensors, 30 % of engrams, and 50 % of vision-tower experts while keeping the same architecture. A mixed Q2-Q8 quant following antirez’s original Dwarfstar recipe now fits the 152 GB wired backbone plus 200 GB engram table on 128 GB systems with SSD streaming. The imatrix calibration was rebuilt from the 1017 checkpoint rather than the original weights. Early tests show the model remains competitive as the leading open-weights release. The wired backbone streams from SSD on 128 GB hardware while the engram table pages on demand. Source: reddit.com

Nine models, five coding tasks, one 16GB GPU: r/LocalLLaMA Nine models were run on five coding problems that each require writing solution.py and test_solution.py, executing tests, and meeting a performance gate. DeepSeek-V4.1-Flash (cloud) and Qwen3.8-27B UD-IQ4_XS + MTP both scored 5/5; the remaining seven models failed at least one of the interval-scheduling or shortest-prefix tasks. The Qwen variant used 185 k tokens over 94 minutes while the cloud run finished in 236 seconds with 43 k tokens. Tasks included CSV quoting rules, an arithmetic evaluator with pinned exceptions, a POSIX-ish shell tokenizer, max-profit interval scheduling with tie-breaks on 200 k intervals, and shortest-prefix pattern matching against 4 000 characters. Both 5/5 runs used a Thompson NFA with epsilon closure on the final task and a bisect DP on the scheduling task. The cloud model completed the entire suite in 236 s; the local Qwen run spent most of its time on a 9 000-case fuzz after reporting success. Source: reddit.com


Agent & Tool Developments

Meet Voxel51 3.0: A Unified Multimodal Data Platform for Physical AI: MarkTechPost Voxel51 3.0 unifies search, curation, annotation and evaluation for multimodal sensor data aimed at robotics, vehicles and factory teams. The release ships Voxel51 Agent with 45+ MCP tools, 80+ operators and 18 open-source skills plus dataset versioning and a free self-service tier limited to three datasets of 10 k samples. Time-series support now aligns multiple sensor streams on one timeline with segment-level embeddings. Supported modalities include MCAP, images, video, 3D point clouds, meshes, DICOM, NIfTI, geospatial data and audio. Voxel51 Search uses a persistent hybrid index that combines vector, scalar and full-text retrieval on LanceDB and avoids full rebuilds on incremental updates. The free tier caps storage at 20 GB and inference at 30 000 requests per month. Source: marktechpost.com

Anthropic turns to lawyers to help AI agents strike deals: Financial Times Anthropic is enlisting legal expertise to enable AI agents to negotiate and execute contracts autonomously. The move targets the practical gap between agent capability and enforceable commercial agreements. Source: ft.com

What Does Okta (OKTA) Gain By Shaping Identity Rules For Autonomous Agents?: Simply Wall Street Okta is integrating early support for the emerging ID-JAG identity standard into its Cross-App Access solution ahead of formal IETF standardization. The work lets autonomous agents authenticate across APIs and enterprise workflows using the same identity layer that already serves human users. Okta’s early implementation gives it influence over default patterns before the standard is finalized. The company is aligning ID-JAG usage with existing Cross-App Access and Auth0 for AI Agents products. Source: simplywall.st


Practical & Community

My 5070 ti/3070 Chimera build.: r/LocalLLaMA A 5070 Ti paired with a 3070 on a PCIe riser runs Qwen 3.6 35B-A3B at 82 tok/s with 16 k context using Ollama or llama.cpp layer splitting. Roughly 23 % of layers spill into system RAM while the 3070 keeps 4 GB unused. The 5070 Ti was purchased for $750 after tax in November 2025. Layer splitting works because the riser runs at x1 and the split passes very little data between cards. The user has not yet tuned the split ratio between the two GPUs. Source: reddit.com

We all joke about our rigs being space heaters, so I had Qwen vibecode it: r/LocalLLaMA A vLLM-backed controller uses motherboard temperature sensors to modulate GPU load as a space heater, pausing inference when heating is active and exposing a token-gated localhost UI. The project is released as a sanitized public repo focused on the control loop rather than production use. The system pauses heating during inference and accepts target temperature commands from a partner via the gated webpage. It is built around the user’s existing vLLM backend and an ESP32-based thermostat is planned next. Source: reddit.com

Fish Audio Drama 3 says it needs a 48 GB NVIDIA card. My 16 GB Radeon disagrees.: r/LocalLLaMA Red Curtain ports Fish Audio Drama 3 (35 B MoE, ~3 B active) to an RX 9070 by keeping FP8 experts in system RAM and caching only active ones on the 16 GB card. It produces 3.3 s of speech in 13 s while using 11–12 GB VRAM. The port keeps top-k off, supplies each speaker with a reference voice clip, and avoids starting lines with sound tags. A small web UI supports voice cloning from a reference clip and multi-speaker scripts with the same tags used in Fish’s own editor. The first line sometimes plays then goes silent, matching a known behavior in the original vLLM recipe. Source: reddit.com

You don’t need that expensive of a build for local llms.: r/LocalLLaMA A single 9060 XT 16 GB card with 24 GB system RAM runs Qwen 3.8 27b at 262 k context using unsloth IQ4_XS, KV cache streaming and n-gram prediction, delivering 42.7 tps generation and 658 tps prefill at 20 k context. The setup uses a fork of llama.cpp that required a Grok-assisted fix for missing PCIe atomics. At 200 k context the same hardware sustains 35 tps generation and 500 tps prefill. The user runs the model with q4 KV cache values and q8 keys. Source: reddit.com


Under the Hood: Pipelined Speculation Across Heterogeneous Devices

Multi-device MoE inference often looks like simple layer sharding, yet the real constraint is that four or five devices spend most of their time waiting for the slowest link. The practical fix is pipelined speculation: a small draft head on the first device guesses the next few tokens, the entire chain validates them in one pass, and the first device immediately starts the next speculation while earlier guesses are still travelling. Each verified step lands roughly 2.2 tokens on average, turning idle time into useful work. On a 48-layer 512-expert model split across two PCs, an M4 Pro MacBook and two phones, this yields 37–39 tok/s at 60 k context with every weight kept in VRAM or unified memory. The first PC holds layers 0–9 plus the output layer and draft head; the MacBook holds layers 10–29; the second PC holds layers 30–47. Phones handle expert selection for layers 10–18 over USB-C while the Mac keeps the hottest experts in its own memory. The draft head must outrun the added PCIe and USB-C hops as context grows; the quality gain disappears once acceptance rate falls below roughly 55 %. The layout serves 69.6 k context natively before KV cache pressure forces a second configuration that moves two layers back to the first PC and keeps 8-bit KV on the Mac and phones. Every change is exact by construction except the Mac’s expert cache, which adds partial sums in a different order; the KL check against a reference has not yet been run on today’s changes. Use this pattern when you already own heterogeneous hardware and need long-context decode; stick to single-GPU expert caching when the model fits in 24 GB and you value simplicity over marginal speed.


Things to Try This Week

  • Run the nine-model coding benchmark yourself on any 16 GB card to see which local model actually finishes real tasks without hanging.
  • Install NVIDIA Structured Data Models from the GitHub commit in the tutorial and test zero-shot tabular predictions on your own relational shop database.
  • Try the Red Curtain port of Fish Audio Drama 3 on an AMD card if you need multi-speaker voice cloning without a 48 GB NVIDIA GPU.
  • Load the 1017 DeepSeek-V4.1-Flash quant in Dwarfstar on 128 GB hardware to compare vision and language quality against the prior checkpoint.
  • Reproduce the 5070 Ti plus 3070 split on your own mismatched pair and measure the effect of different layer ratios on the 35 B MoE.

On the Horizon

  • More community quants of the 1017 DeepSeek checkpoint are expected in the next few days.
  • Additional MCP tool packs for Voxel51 Agent will appear as robotics teams publish their workflows.
  • Further mismatched-GPU split-ratio experiments will surface as 5070 Ti owners add older cards.
  • A second layout for the five-device Qwen3.8 Flash Next chain that reaches full 262 k context is already in testing.

💬 Reply to this email — Patrick reads every one.

Share: X · LinkedIn · WhatsApp

Forwarded this email? Subscribe here — it's free.

▶ Listen to the podcast

📺 Watch on YouTube  ·  📝 Read the blog  ·  🖼 Free image gallery (CC BY-SA)  ·  📊 Data Hub & Story Trackers  ·  🧭 Start Here

Nerra Network · AI-narrated voice (Grok TTS) · Editorial by Patrick

You're receiving this because you subscribed to Models & Agents on nerranetwork.com.

Issue #207 · Models & Agents · Oct 18, 2026
Don't miss what's next. Subscribe to Nerra Network:
← Newer Four Artemis II astronauts accepted the Stephen… · Frontiers 🛰️ Older → SpaceX Dragon experiments on artificial retinas and… · SpaceX Daily 🚀
nerranetwork.com
Twitter
Powered by Buttondown, the easiest way to start and grow your newsletter.