What You Need to Know: A 9B model fine-tuned with FlowSpec lifts documented compliance checks from 67-71% to 93-100% on four held-out debt-collection procedures. New model releases include LLM-jp-4 (8B dense and 32B-A3B MoE) and Titan-Llama memory retrofits, while benchmarks such as NEPSEE, LePhilBench, and PersEvalBench target low-resource languages, legal philosophy, and persona handling. Builders should watch how specification-driven supervision and memory modules change agent reliability in constrained domains.

Top Story

A single 9B model trained on 2,506 conversations from 297 randomized procedures now follows four entirely unseen regulated flows, including three debt-collection procedures from a withheld lineage. FlowSpec supplies the machine-readable specification that generates instructions, gates tools, supplies supervision, and defines tests; the agent is restricted to an approved sentence catalogue. Documented checks rose from 67-71% to 93-100% while grounded checks already passed at 97-100% before training; scenario pass rate improved from 15/43 to 40/43. An audit of the shared specification uncovered 11 defects, three of which the metric itself missed. The approach demonstrates that a small self-hosted policy can generalize across creditor-specific revisions of the same legal rules without retraining. The evaluation used a Thai outbound-call agent and included one non-debt procedure alongside the three debt-collection cases. Every creditor turns legal rules into its own procedure and revises it, yet the trained policy transferred without additional fine-tuning on the target procedures. Source: arxiv.org

Model Updates

LLM-jp-4 Technical Report: arXiv NLP LLM-jp-4 comprises an 8B dense model and a 32B-A3B mixture-of-experts model, both trained from scratch on approximately 12 trillion tokens with explicit criteria for data, external models, and software selection. The models show substantial gains over prior LLM-jp releases and competitive results against open-weight models of similar scale. The full development process is released for independent verification and reproduction. The emphasis on transparency includes public criteria for choosing training data and external components. Evaluation results place the new models ahead of earlier LLM-jp versions while remaining competitive with other open-weight systems at comparable parameter counts. Source: arxiv.org

Titan-Llama: Retrofitting Pre-trained LLMs with External Neural Memory Modules: arXiv NLP Titan-Llama replaces attention layers in Llama-3.2-1B-Instruct with a hierarchical memory architecture that preserves full-fidelity attention over recent tokens while compressing history into recurrent fixed-size neural memory modules. On QuALITY accuracy rises from 23.87% to 40.51% and on LongHealth from 14.50% to 40.75%; inference shows more than 20% faster prefill and 37% faster decode at medium context lengths with 24% lower steady-state memory usage. The method applies post-training without training a new model from scratch. The blockwise attention mask keeps recent tokens under full attention while historical context moves into the neural memory modules. The formulation allows pre-trained attention layers to be swapped for the memory architecture after initial pre-training. Source: arxiv.org

Copy-Paste Language Models: arXiv NLP Copy-Paste Language Models add a lightweight Copy-Paste head to the standard language-modeling head, creating an alternative gradient path that mitigates the LM-head gradient bottleneck. Models from 135M to 1.7B parameters show improvements across scales and backbones with negligible training and inference overhead. The approach improves training efficiency, downstream performance, and operating points in long-context and retrieval settings. The added head supplies a direct gradient route that bypasses the primary language-modeling head during back-propagation. Validation covered multiple model scales and different backbone architectures. Source: arxiv.org

Qwenfant: Train Big Character-level BabyLMs on Concise Parallel Text: arXiv NLP Qwenfant, a Qwen3-based model submitted to the 2026 BabyLM Multilingual track, uses LLM-summarized and translated data plus a character-level vocabulary and larger model size, realized through LoRA-style fine-tuning. At submission time it ranked first in the Multilingual track. The training data came from concise parallel text produced by summarization and translation steps. Both the character-level vocabulary and the increased model size required LoRA adapters to reach full performance. The submission achieved the top position among entries in the multilingual category of the challenge. Source: arxiv.org

Agent & Tool Developments

HERO: Harnessing Evidence for Test-time Refinement and Prompt Optimization in Program Generation: arXiv NLP HERO learns reusable prompt updates from execution-verified successes and failures, validates revisions before adoption, and at test time treats generated code as an instance-specific harness refined by execution feedback. Across four SQL, Cypher, and Python benchmarks it improves pass@1 by 7.6 points offline; test-time refinement adds another 2.0 points and improves execution efficiency. The offline stage extracts prompt revisions only from cases where execution confirms correctness or exposes clear failures. At inference time the generated program itself becomes the harness that receives further execution-driven edits. The method was tested on text-to-SQL, text-to-Cypher, and text-to-Python tasks. Source: arxiv.org

Learning What to Keep: Transfer-Aware Personal Skill Learning from User Feedback: arXiv NLP SkillGrain decomposes user feedback into atomic rules, predicts transferability, and retains only rules suitable for future requests using rubric-guided rewards and fine-grained credit assignment. Across three base models on five personalized writing tasks it achieves the best overall average performance in both in-domain and out-of-domain settings. The framework evaluates each rule for content quality and for the decision to retain or discard it. Reinforcement learning jointly optimizes skill induction and the skill-conditioned generation step. The benchmark spans five distinct personalized writing tasks with both in-domain and out-of-domain test splits. Source: arxiv.org

PersEvalBench: Can LLM Evaluators Judge Contextually Appropriate Use of Persona Information?: arXiv NLP PersEvalBench contains 403 multi-episode scenarios testing nine controlled error subtypes across under-, over-, and misapplied-state personalization. On the human-evaluated subset the strongest evaluator recognizes 63.9% of under-personalization errors while humans recognize 94.4%; highlighting relevant history raises recognition only slightly. The benchmark covers three families of errors and was constructed so that generated errors match their intended types according to human audits. Five LLM evaluators were tested on the full set of scenarios. The largest performance gap appears on under-personalization cases. Source: arxiv.org

Practical & Community

NEPSEE: A Double-Annotated Benchmark for Nepali Sentiment Analysis: arXiv NLP NEPSEE retains 3,189 sentences where two annotators agree on polarity from 5,614 tweets; on the three-class benchmark NepBERTa reaches 0.877 macro-F1 and three commercial LLMs reach 0.882-0.896 zero-shot. On the five-point scale Claude Opus 5.5 beats the best encoders by 0.10-0.12 macro-F1. Restoring non-spacing Devanagari marks changes macro-F1 by +0.084 or -0.095 depending on the tokenizer. The dataset draws from tweets by Nepali public figures and keeps only items with matching polarity labels from both annotators. On the five-point scale 766 sentences carry two adjacent labels that are both accepted during training. Source: arxiv.org

TopicMerge: A Divide and Conquer Approach to Topic Modeling: arXiv NLP TopicMerge first over-segments documents into fine-grained micro-clusters, then uses an LLM-guided iterative merging algorithm with local neighborhood context to consolidate topics. On Bills and Wiki datasets it reaches state-of-the-art clustering while reducing required LLM calls by over 92% compared with document-level methods. The initial over-segmentation produces semantically pure micro-clusters that receive preliminary semantic labels. The subsequent merging stage arbitrates candidate pairs using neighborhood context before any merge is executed. The pipeline was evaluated on the Bills and Wiki collections. Source: arxiv.org

Whose Data, Whose Benchmark? A User-Centric Framework for Auditing Evaluation Resources: arXiv NLP The framework audits 16 resources across seven dimensions and finds American-centered conventions account for 62.4-94.4% of matched signals in 11 text benchmarks; geographic visibility in five image-text resources correlates with national GDP share (mean Pearson r = 0.780). All 16 resources operationalize source-user context but none operationalize target-user context. Paired scoring experiments on DROP and TruthfulQA illustrate how meaning-preserving answer variants affect exact-match and reference-similarity scores. The audit covers benchmark content, metadata, documentation, and scoring procedures. Source: arxiv.org

Under the Hood: Target-Token Asymmetry in Layer-Ablation Studies

Layer-ablation studies compare early- versus late-layer damage across two corpora and interpret a larger early-layer effect on one corpus as task-specific depth specialization. The comparison is actually performed over different target-token distributions: math solutions contain far more whitespace and numbers than encyclopedia prose. When the pooled interaction is decomposed by target-token type, only whitespace tokens produce a positive early-late interaction; removing whitespace reverses the sign of the pooled effect in all three fully decomposed model instances. The same reversal occurs in two of three instances once the full target mixture is matched across corpora. A simple screen computed from the tokenizer alone places every held-out pair into the risk region its label predicts. The diagnosis was validated under position matching and adversarial reassignment of token types. A smaller content-word effect one token ahead follows the share of content targets that the unablated model already predicts. The practical consequence is that any claim of depth specialization must first report the target-token mixture, decompose the effect by token type, and confirm the sign survives mixture matching; otherwise the result is an artifact of the evaluation corpora rather than model architecture.

Things to Try This Week

• Test the FlowSpec training recipe on your own regulated workflow by generating synthetic procedures and measuring compliance lift on held-out flows.
• Run the NEPSEE three-class and five-point splits against your preferred Nepali-capable model to check tokenizer sensitivity on Devanagari marks.
• Apply Titan-Llama memory retrofitting to a Llama-3.2-1B-Instruct checkpoint and measure QuALITY and LongHealth accuracy gains at 128K context.
• Evaluate PersEvalBench scenarios against your current LLM judge to quantify under- versus over-personalization error rates on multi-episode histories.
• Compare the Copy-Paste head addition against a standard language-modeling head on a 1B-scale model for long-context retrieval tasks.

On the Horizon

• Additional sovereign LLM releases following the LLM-jp-4 transparency model are expected from other national efforts.
• Further agent frameworks will incorporate execution-verified harness updates similar to HERO.
• Expanded low-resource language benchmarks will continue to test transfer from high-resource pre-training.
• More post-training memory retrofit methods are likely to appear for additional base model families.

💬 Reply to this email — Patrick reads every one.

Share: X · LinkedIn · WhatsApp

Forwarded this email? Subscribe here — it's free.