FlowingData Daily

FlowingData Daily

Archives
Log in
Subscribe
October 19, 2026

Calculation volumes for AI training

More big numbers coming in. For NYT’s the Upshot, Aatish Bhatia demonstrates the millions of trillions (octillions!) of calculations required for training AI.

During its training, the A.I. is fed a portion of a sentence, and the knobs are adjusted using a series of math calculations to improve its guesses for the next word. This process is then repeated over trillions of words of training data.

It takes only a few additions and multiplications to adjust a single knob. But when trillions of knobs are adjusted trillions of times, the math quickly adds up.

And this is just the first step in training a language model, known as pre-training. It’s followed by additional steps of calculations that hone the A.I.’s behavior and improve its ability to code and solve math problems.

All those calculations require energy, data, and time. The big companies are taking what they can get with the first two things in efforts to reduce the third.

Tags: Aatish Bhatia, calculations, scale, training, Upshot

+


FlowingData is 100% made possible by members. Join today and get access to courses, guides, and more resources from this real person (me, Nathan).

Become a Member

Don't miss what's next. Subscribe to FlowingData Daily:
← Newer Increasing number of shots in movies Older → Warming Stripes as a screensaver