Frozen 16 MB embedding tables let ternary 46 MB models… · M&A 🤖
| View this email in your browser |
![]() Models & AgentsDaily AI models, agents, and practical developments.
|
🎧 Today's episode Episode 208 · Frozen 16 MB embedding tables let ternary 46 MB models hit 615–740 tokens/sec on laptop CPUs while matching float baselines on LAMBADA and PIQA. 2026-10-19 ▶ Listen now |
What You Need to Know: A new arXiv paper shows that replacing learned embeddings with a frozen 512-bit code table cuts parameters dramatically yet preserves or improves downstream performance on several commonsense benchmarks. RealCLI introduces a paired benchmark that measures both task execution and human-agent collaboration quality for CLI agents, exposing large gaps in frontier models. ProPerAgent demonstrates memory-driven proactive assistance that lifts strict accuracy from under 10% to 37.5% on long-horizon personal memory tasks without additional training. Top StoryResearchers replace the full learned embedding table in a Gemma-shaped 270M model with a single frozen 16 MB table of 512-bit codes derived by principal-component projection and sign binarisation from a pretrained embedding. The frozen table is trained from scratch on the same 2.05B tokens; the learned table improves validation loss by 0.20 nats but the frozen version matches or beats it on LAMBADA, HellaSwag and PIQA. When paired with a ternary BitNet b1.58 body the entire model fits in 46 MB, ties the float baseline on LAMBADA and PIQA, and runs at 615–740 tokens per second on a laptop CPU in 62 MB of memory—five to six times faster than the float model’s 8-bit GGUF in llama.cpp at roughly one-seventh the memory. The frozen table also accepts 10,000 new words after training at a 3% cost while the learned table collapses by 43%. All weights, data and evaluation answers are released. Source: arxiv.org Model UpdatesFrom Parallels to Divergence: Language Models' Learning of Action Nominal Constructions — arXiv NLP: Researchers construct 96 synthetic target languages varying clause word order, noun-phrase order, alignment, complement system and ANC strategy, then train GPT-2-small models from scratch on each. Targeted minimal-pair evaluation across 34 grammatical phenomena shows typological parallels in basic clause configurations but clear divergences in complement systems and ANC strategies, with models also acquiring combinations rare or unattested in natural languages. Source: arxiv.org Pearl-Chat: Engineering a native Luganda language model from first principles in JAX and Flax NNX — arXiv NLP: An 8.47-million-parameter causal decoder-only transformer is built for Luganda using a byte-level BPE tokenizer (V=8,192) that achieves 1.60 tokens per word fertility. Two-stage training on 59,501 pre-training sequences and 510 instruction dialogues drops validation cross-entropy from 9.4695 to 0.1818 nats; the model reaches 0.9792 ROUGE-L and 0.9405 BLEU on a held-out diagnostic benchmark while preserving Bantu noun-class concord. Source: arxiv.org HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments — arXiv NLP: A language-agnostic pipeline denoises noisy KB-text alignments with an NLI filter and releases the 1.5-million-instance HeRE silver dataset for Hebrew RE. Gemma-4-31B-it reaches 0.915 F1, matching GPT-5.4 at 0.921, and 0.844 F1 on a fully held-out dual-annotated test set. Source: arxiv.org Cultural (In)Competence: Investigating Culture-Specific Stereotype Detection Capabilities of LLMs — arXiv NLP: Experiments across US, Dutch and Chinese contexts show LLMs over-predict stereotypes and that non-English prompts improve cultural-context consideration. Additional instruction tuning reduces false-positive rates while training-data size moderates detection effectiveness non-monotonically. Source: arxiv.org Inductive Learning of Concept Embeddings from Cross-Linguistic Colexification Patterns with Textual Attributes — arXiv NLP: An inductive framework combines full and partial colexification networks with short concept descriptions to embed previously unseen concepts. The resulting embeddings correlate with cross-linguistic similarity measures and generalise to nodes absent from training data. Source: arxiv.org Causal Latent Workspaces for Looped Transformers — arXiv NLP: A four-round shared-depth architecture adds prefix-specific latent slots that each position writes to and reads from before the next shared block. At 101.71 M parameters the model leads on WikiText-103 raw, enwik8 and TinyStories, improving perplexity 0.73–1.06% over the strongest non-workspace control across nine seeds. Source: arxiv.org Agent & Tool DevelopmentsRealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents — arXiv NLP: The benchmark supplies two modes—REALCLI-EXEC with complete specifications and REALCLI-COLLAB with a scripted user delivering progressive corrections—and scores 22 sub-dimensions across 51 interaction templates. Frontier agents reach at most 49.2% average score and 37.6% joint collaboration score, with consistent shortfalls in steerability, trust calibration and safety boundary management. Source: arxiv.org The Model Knows Better: Defending Medical Multi-Agent Systems Against Prompt Injection — arXiv NLP: A single injected passage succeeds in up to 61% of cases across two multi-agent systems and three benchmarks; collaboration does not reliably protect against attack. A training-free defense that separates patient findings from clinical judgments and verifies each judgment against findings alone lowers attack success on MedQA to 4–7% while preserving accuracy on clean cases. Source: arxiv.org ProPerAgent: Toward Memory-Driven Proactive Assistance — arXiv NLP: ProPerMem pairs egocentric video with year-long distractor-rich memories averaging 618 k tokens and human-verified trigger windows. ProPerAgent surfaces relevant memory via a graph store and gates it to avoid unnecessary interruptions, lifting strict accuracy from below 10% to 37.5% on Qwen3.8-27B and 38% on GPT-5.3-Chat without training; RL post-training reaches 42.4%. Source: arxiv.org Explore Separately, Adopt Selectively: Execution-Guided Tool Use over Large Tool Libraries — arXiv NLP: BRIA isolates candidate tool investigations in separate branches, executes each, then adopts only useful results into the main trajectory. On MCP-Atlas with a shared pool of 1,897 tools it achieves an average claim score of 89.77, a 14.37-point gain over MCP-Zero, while remaining on the Pareto frontier for performance versus input-token cost. Source: arxiv.org Practical & CommunityBroad and Efficient PII Detection with Unified Labels — arXiv NLP: A 141 M-parameter encoder trained on 1.12 million examples from 77 public datasets, relabeled under a consistent 12-category schema, achieves the highest recall against larger baselines in 11 of 14 comparisons on AI4Privacy and Nemotron-PII at a low-FPR operating point. It delivers 2–10× higher throughput and 94–97% less peak GPU memory than the baselines. Source: arxiv.org CoLU: Confidence-Weighted Layerwise Unlearning for Large Language Models — arXiv NLP: CoLU updates a single selected transformer layer with confidence-weighted forgetting and retain regularisation. On TOFU it improves both forgetting and retention over NPO, SatImp and GA-based FOM-UL in all six settings; on MUSE it records the highest retained KnowMem and PrivLeak closest to zero among compared methods while using 77.4–79.5% less GPU memory than full-model unlearning. Source: arxiv.org TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents — arXiv NLP: An external controller learns which micro-skills to inject, which irrelevant tools to prune, and when to break unproductive loops across sequences of realistic coding tasks. It reduces average token cost 18.4–30.9% for Claude Code, Codex and GitHub Copilot CLI paired with frontier models while preserving task success; savings generalise to web development and security auditing. Source: arxiv.org Prune the Leaky Neurons: Mitigating Private Data Leakage in Fine-Tuned LLMs via Joint-Embedding-Guided Pruning — arXiv NLP: JEPA-Prune trains a public reference model on PII-free views of the same records and uses cross-model representation probes plus integrated gradients to identify and prune 1.6% of neurons. PII inference accuracy drops up to 7.4× and extraction recall up to 13.9× while unseen-text perplexity stays within 16% of the unpruned model for most LLMs. Source: arxiv.org Under the Hood: First Passage AttentionFirst Passage Attention replaces the monotonic accumulation of decay along the key-query path with a path-extrema operation that lowers a key’s logit by the deepest dip of a learned scalar field below that key’s level. The mechanism naturally forms a stack when the field and levels follow a shared learned walk, allowing a constant-dimension head to realise any subset of keys left alive by subsequent dips. In synthetic experiments a single-layer FPA head converges to the target alive-key sets on every seed while comparable softmax and Forgetting-Transformer baselines fail on all seeds. At 110 M parameters the model matches Forgetting-Transformer perplexity yet achieves perfect passkey retrieval at eight times the training length even when its key-value cache is compressed to roughly 25%. The added scalar field introduces one extra learned parameter per head and a small constant-time max-tracking operation per step. Use FPA when long-context retrieval stability matters more than the marginal training cost of the extra field; the main gotcha is that the learned walk must be shared between field and levels, otherwise the stack property disappears and performance falls back to softmax levels. Things to Try This Week
On the Horizon
|
💬 Reply to this email — Patrick reads every one. Share: X · LinkedIn · WhatsApp Forwarded this email? Subscribe here — it's free. |
📺 Watch on YouTube · 📝 Read the blog · 🖼 Free image gallery (CC BY-SA) · 📊 Data Hub & Story Trackers · 🧭 Start Here Nerra Network · AI-narrated voice (Grok TTS) · Editorial by Patrick You're receiving this because you subscribed to Models & Agents on nerranetwork.com. |
| Issue #208 · Models & Agents · Oct 19, 2026 |
