Last week, a client sent me a message about an AI assistant we'd been testing inside his business.

"Still making lots of errors."

He had been checking its payroll answers and correcting it as it went. He assumed the assistant would learn from the feedback and get better.

It didn't.

My first thought was that it needed a better model. A brain upgrade.

We did give it a stronger model, but once I read the chat log properly, I realised I had blamed the wrong thing. I had built the workflow badly.

I had given an AI assistant access to a large appointment schedule and asked it to reconstruct each staff member's working hours through conversation.

That schedule included ordinary appointments, lunch, administration, meetings, cleaning, cancellations, no-shows and split shifts. Some staff were full-time, others were paid from appointments, and a few had their own exceptions.

The assistant had to find the right records, keep each staff member separate, apply the correct rules and compare the result with the submitted hours.

It could do that... sometimes.

Employees tend to get a little cranky when you only pay them "sometimes".

The assistant mixed records, confused appointment durations with start and finish times, and occasionally changed its answer after being challenged. Worse, the answers sounded confident enough that the client had to keep checking them.

I was supposed to be removing work from his week. I had added another thing he couldn't trust without checking.

We stopped the payroll calculations through normal chat.

I replaced that part with a fixed payroll checker. It reads the exact source fields, filters one staff member at a time and applies the agreed calculation rules. It only runs against completed payroll periods.

The assistant can retrieve the result and explain anything that needs attention. The calculation itself belongs in ordinary software with tests. It does not need an LLM.

I also made it stop when the source data is ambiguous.

We saw this in a fresh test this week. Two days contained multiple earnings lines in the accounting system. The old calculation added them together and produced a ridiculous 66-hour weekly total.

The new version refused to give me a total. It told me those lines needed to be classified before it could continue.

The assistant cannot approve a timesheet, payslip or pay run. It gives the client a result he can check against the source screens. We won't call it reliable until a completed week matches his manual check and we can explain every discrepancy.

I went into this thinking the model was the problem. It was part of it, but the bigger problem was me asking a language model to calculate something that had a clear right answer.

The client's corrections inside the chat weren't becoming permanent payroll rules either. The assistant could acknowledge the correction, sound like it understood and then make the same mistake later. I should have separated the calculation from the conversation much earlier.

I still want the assistant involved. The client should be able to ask it to check a completed week, see which days match and get a clear explanation of anything that needs attention. He should not have to compare every schedule entry with every submitted line himself.

We're still testing this payroll checker. Confidence is key for the client and his staff! :)

If you have a recurring admin or accounts job that always ends with someone comparing the same two systems, reply and tell me what it is.

I've probably seen a version of it already, and I'll tell you honestly how I'd handle it.