A new weekly strand starts here. The Primer explains how these systems actually work, from zero, for lawyers who will have to advise on them — and lands each part on a duty under a real data-protection law.

TL;DR

• A language model does one thing: given the text so far, it produces a probability for every possible next fragment, picks one, and repeats. That is the entire mechanism.
• It is not a database and it does not look anything up. When it produces a citation, it is producing something citation-shaped, not retrieving a record.
• OpenAI's own researchers published in September 2025 that hallucination is a natural statistical consequence of how these systems are trained and graded — not a defect awaiting a patch.
• India's data statute never says "artificial intelligence". It does not need to.

Also as a PDF. This issue is available as an 8-page illustrated document with five diagrams — the loop, what survives training, why a real and an invented citation are indistinguishable, a word-by-word audit of the DPDP Act, and the running clock. Free, no signup: Open The Primer, Part 1 (PDF, 8 pages)

THE PRIMER — Part 1: What a language model actually is

Most of what a lawyer needs to know about artificial intelligence follows from one sentence. Before the sentence, though, the words in it have to mean something — so this issue defines every term it uses, as it uses it.

Start with the name. A model, here, is simply a piece of software that has been tuned until it is good at one narrow task. A language model is one tuned to do this:

LANGUAGE MODEL A program that takes a stretch of text and produces, for every possible continuation, a number saying how likely that continuation is to come next. It then picks one and adds it on. That is the entire specification. Nothing about truth, meaning, sources or intention is built into it.

Why would anyone build a next-word guesser?

This is the question that never gets answered, and everything else depends on the answer.

Nobody set out to make a machine that guesses words. They set out to make a machine that could use language. The obstacle was never ambition — it was teaching. Language cannot be written down as a set of rules. Decades were spent trying: grammar rules, dictionaries, hand-coded exceptions. Those systems broke on the first sentence a real person actually said.

Next-word prediction solved a teaching problem, not a language problem. Take any text ever written. Hide the last word. Ask the machine to guess it. You instantly know whether it was right, because the true answer was sitting there in the text all along.

No human has to mark the paper. That single property is the whole reason this approach won. The machine could be set an exam, and corrected, billions of times a day, against writing that already existed — with nobody supervising.

And then came the part that surprised the people doing it. To guess the next word well, you cannot avoid absorbing almost everything else:

WHY GUESSING FORCES LEARNING "The capital of France is ___" — unguessable without the fact. "The accused was granted ___" — unguessable without the grammar and the legal register. "She dropped the glass and it ___" — unguessable without knowing something about how the world behaves.

Facts, grammar, tone, register, the ordinary physics of objects — all of it had to be picked up along the way, because all of it is what determines the next word.

Prediction was never the goal. It was the only exercise anyone found that was cheap enough to repeat a trillion times — and repeating it dragged everything else along behind it. What you talk to is the side effect.

What the machine is actually handling

Two more words, because the rest of this issue leans on them.

TOKEN The machine does not work in words. It works in tokens — small chunks of text, roughly three-quarters of a word on average. "Petition" may be one token; an unusual name may be three or four. Where this issue says "word", it is choosing readability. The machine is handling tokens.

STRING A string is a piece of text treated purely as a sequence of characters — letters, digits, spaces, punctuation — with no meaning attached. To the machine, "AIR 1973 SC 1461" is a string. So is "AIR 1999 SC 8402". It has no way to ask whether either one corresponds to a judgment, because a string is all it ever had.

With those in hand, the sentence at the top can be stated properly, and it is now a description rather than a slogan.

A language model predicts the next token. Give it some text. It produces a probability for every possible next token. It picks one. It appends it. Then it does the same thing again with the slightly longer text, and again, until it stops. What you read as an answer is that loop running a few hundred times.

the text so far  →  a probability for every possible next token
                 →  pick one  →  append it  →  and again, a few hundred times

There is no step in that loop where the system decides what is true. There is no step where it consults anything. There is only the next token, and then the next one.

Where the numbers came from

The system was shown an enormous quantity of text, and billions of internal numbers were adjusted, over and over, until its guesses about what came next got good.

PARAMETERS (also: weights) The adjustable numbers inside the model — billions of them. Each one is a dial. Together they decide, for any input, which token comes out. Training is the process of nudging those dials: show it text, let it guess the next token, measure how wrong it was, adjust every dial a fraction in the direction that would have made it less wrong. Repeat, at enormous scale.

When the training finished, the text was not kept. The parameters are all that remains. The model is those numbers and nothing else.

This is why the question "where did it get that from?" usually has no answer. The honest reply is not that the source is hidden or hard to find. It is that the system never held a source in the first place.

Why it invents things — and why that is not a bug report

Every lawyer who has used one of these has watched it produce a case that does not exist, with a citation formatted perfectly.

Here is what is happening. A real citation and an invented one are identical to the machine — both are simply strings of a familiar shape:

Nothing in the mechanism distinguishes a string that corresponds to a judgment from one that merely follows the pattern of citations. There is no tell — which is why verification cannot be delegated to the tool that produced it.

In September 2025, OpenAI published a paper by Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala and Edwin Zhang titled "Why Language Models Hallucinate." Its argument is that these errors are neither mysterious nor incidental.

"Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty… We argue that language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty… Hallucinations need not be mysterious — they originate simply as errors in binary classification." — arXiv:2509.04664, 4 September 2025

First, the error is structural, not accidental. The paper's position is that "if incorrect statements cannot be distinguished from facts, then hallucinations in pretrained language models will arise through natural statistical pressures." Distinguishing a real citation from a citation-shaped string is exactly such a case.

Second, the way these systems are scored rewards guessing. Standard evaluations give no credit for "I do not know." In the authors' phrase, "language models are optimized to be good test-takers, and guessing when uncertain improves test performance" — precisely as a student sitting a paper with no negative marking will outscore one who leaves blanks.

That is the part worth being blunt about: the confidence of the answer tells you nothing about its accuracy.

The three things that follow, for practice

1. "The AI said so" is not a source. It cannot be, because the system has no concept of a source. Anything it produces that matters must be verified against the actual instrument — the bare Act, the reported judgment, the notified rule.

2. The verification burden has not moved. It sits with the professional who signs, exactly as it did when the first draft came from a junior.

3. Fluency is not competence. Wrong output arrives in the same confident register as right output. There is no tremor in the voice. The paper above explains precisely why you should not expect one.

And the legal turn — where this series is going

The Digital Personal Data Protection Act, 2023 does not use the words "artificial intelligence." Neither do the Rules made under it. India's AI Governance Guidelines, released on 5 November 2025, are expressly a reference framework rather than a compliance requirement.

The conclusion most people draw is that nothing binds them yet.

The conclusion the drafting supports is different. The Act does not regulate AI as a technology. It regulates processing. And every stage of what you have just read is processing: the text the system was trained on, the text you type into it, the text it keeps in order to be useful to you next time.

Which is why this series starts at the machine rather than at the statute. You cannot apply a processing statute to a system until you know, concretely, what the system does with the material it is given.

INDIA DESK

There is no automated-decision provision, and the absence is precise. Read the Act itself:

The entire Article 22 vocabulary is absent from the Act. There is no Indian equivalent of the GDPR's right "not to be subject to a decision based solely on automated processing," and no right to be told the logic of a decision. If a client's system refuses a loan, declines a claim or filters a candidate, the individual's handles under the Indian statute are the ordinary ones — notice, consent, purpose limitation, correction, erasure, grievance. The output being machine-made adds nothing to their position.

The binding AI rules are not in the data statute. The Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026 were notified on 10 February 2026 and came into force on 20 February 2026. They introduce a regime for "synthetically generated information": labelling obligations for AI-generated visual and audio content, and a takedown window cut from thirty-six hours to three. They bind intermediaries — and reach platforms that enable the creation of synthetic content. These are in force now, while the substantive DPDP obligations are not.

GLOBAL — DATA PROTECTION & AI GOVERNANCE

A European regulator has already held that an AI model is not automatically anonymous. In Opinion 28/2024, adopted 17 December 2024, the European Data Protection Board considered when an AI model trained on personal data can be treated as anonymous, and concluded that it cannot be assumed — it must be assessed case by case. The Board accepted that although these models are not designed to produce personal data, such data may nonetheless be embedded in the model's parameters as mathematical objects.

Read that against the section above. The parameters are all the model keeps. If personal data can survive inside them, then "we deleted the training data" is not the end of the analysis.

We come back to this properly in a later part of this series.

RUNNING CLOCK — computed 7 September 2026

The distances below are as at the date of this line and nowhere else in this issue. Everything above is stated in absolute dates so that it reads correctly whenever you open it.

PRACTITIONER'S VERDICT

What to tell a client this week. If they are relying on one of these systems for anything that reaches a client, a court or a regulator, the question to ask is not "is it accurate?" — it is "who checks it, and what do they check it against?" There is no configuration, no prompt and no subscription tier that makes the output self-verifying, and the people who build these systems have published the reason why. If the answer is that the tool is trusted because it sounds right, that is not a workflow, it is an exposure. And if they tell you no Indian law applies yet because there is no AI statute — they are right about the statute and wrong about the exposure. The data law does not need to name the technology to reach it.

NEXT IN THIS SERIES

Part 2 — training versus using. Your client's data can end up in the weights, or in the prompt. Those are different events with different consequences, and most privacy policies do not distinguish them.

ABOUT THE AUTHOR

Rushikesh R. Mahajan — Advocate · Data protection, technology and AI governance LL.M. in Law and Technology, Queen's University Belfast Practises before the Bombay High Court and courts across India in civil and criminal matters Enrolment No. MAH/11261/2021 · Bar Council of Maharashtra & Goa

Writes on the Digital Personal Data Protection Act, cross-border data transfer, AI governance and technology contracts — and publishes open reference tools for Indian data and AI law as wolfgang_rush.

This issue as a PDF (8 pages, 5 diagrams): The Primer, Part 1 Every free document: wolfgangrush.github.io/resources Writing and open tools: wolfgangrush.github.io LinkedIn: linkedin.com/in/rushirmahajan

This publication is informational and educational only. It is not legal advice and creates no advocate–client relationship. Authored from India; readers in other jurisdictions should map terms to local law.

Every statutory reference and every date in this issue was checked against at least two independent sources before it went out. Where a claim could not be verified to that standard it does not appear, or it is stated as an absence of record rather than as a fact. The archive, and everything else I have been building, is at wolfgangrush.github.io