Your agent can't add. Stop asking it to.
Hey there,
You ask your agent:
“How much did I spend on food this month?”
The number is off.
Not wildly. Just enough that you can't trust it.
So you tighten the prompt: “Be precise. Double-check your math.”
Then you try a bigger model.
It's better on Tuesday and wrong again on Thursday.
You start to think agents just aren't ready for anything that has to be exact.
But the model isn't the problem.
You gave a stochastic system a deterministic job.
Summing, counting, converting and comparing dates are deterministic operations. They have one right answer.
Put them in a function.
The model decides which function to call and with what inputs. Your code gets the answer right.
Asking the model to add up a list of expenses is a coin flip.
Giving it a spending_by_category tool turns the same question into a deterministic lookup.
And this leads to a useful debugging rule:
A surprising number of “the agent hallucinated” bugs are actually tool-design bugs.
The tool the agent needed was missing.
Or it couldn't reach it.
We shipped that second kind of bug in our own course. 😮
In week 2, students build and test a convert_currency tool that calls a live FX API.
But every interface was creating the assistant with structured_output=....
And one line quietly determined whether the tool was available:
if self.tools and not self.structured_output:
create_params["tools"] = self.tools
create_params["tool_choice"] = "auto"Ask:
“How much is 20 USD in EUR right now?”
And you get an empty answer or a guessed rate.
Remove structured_output, and the same model and prompt can call the live API.
The model didn't suddenly get smarter.
We had accidentally taken its tool away.
And even when you do put the deterministic work in code, the code still has to be correct.
In a cohort review this summer, an expenses endpoint returned a total — except total was the number of expenses, not the amount they cost.
No LLM was involved.
A one-line test catches that bug.
A prompt telling the model to “be precise” doesn't.
That's the boundary we care about in the Agentic AI cohort:
Let the model reason, choose tools and handle ambiguity.
Let deterministic code handle things that must be correct.
You build an expense agent end-to-end, test the important pieces, and learn how to design the boundary between stochastic LLM behavior and deterministic software.
And, importantly, how to build agents that know when to stop.
We're also going live on October 14 to talk about building deterministic systems on top of stochastic LLMs.
Cheers,
Bob & Juanjo