One Concept, One Example, One Question

The concept

You may know exactly what you want an AI assistant to produce and still be unable to tell whether it's done the job well. That's why the second question in my Practical Delegation Test is: Can I verify the result cheaply enough?

The first question, about describing the outcome, helps settle what work you're asking for. This one asks how you'd catch a wrong claim before anyone uses the result. I want to know what I could check it against, and whether the person responsible has enough time and knowledge to make that check count.

For me, “cheaply enough” means the checking effort is proportionate to what a missed error could cost and still worthwhile compared with doing the work yourself. An hour of careful review might be a good trade if AI has saved a day of work. If review takes as long as the original task, the speed gain disappears. A result that others will act on also needs a stronger check than a note kept for your own use.

I can sometimes make verification easier by asking AI to show where its claims came from. That gives the reviewer a route back to the evidence. When I can't build a workable check, I narrow AI's job and keep sign-off with someone who knows the work.

The example

Imagine an organiser asking AI to turn a recording of a one-hour project meeting into a short note of decisions and actions. The request is clear, and the draft arrives quickly with named owners. It looks ready to send.

During the meeting, though, someone suggested moving a deadline and the group agreed to investigate the impact. The draft says the deadline was moved. Colleagues who missed the meeting can't spot that mistake from the note, but they could act on it. The organiser needs to check the claim before sending it.

Instead of asking for a note ready to send, I'd ask AI for a draft with a time reference beside each proposed decision and action. The organiser can check those references against the recording, including whether the marked passage supports the claim. Their memory of the discussion helps them notice anything missing. When the organiser was present and the recording is clear, this is a manageable review. If the recording doesn't settle whether the deadline changed, the organiser should confirm it with the people involved.

The organiser can now use AI to save drafting time without treating its first version as the meeting's decision record. If they can't carry out that check, the summary stays a working draft rather than something colleagues act on.

The question

For one result you might ask AI to produce this week, what check could the person responsible realistically perform before anyone relies on it?

Related

• The Week 39 issue, Can I Describe the Outcome Clearly?, covers the first question in the series.
• The Practical Delegation Test sets this question alongside the other decisions involved in handing work to AI.

Support

Everything I publish here is free to read, and always will be.

I maintain this newsletter and my website in my own time. If something here has been useful to you, and you'd like to help cover the costs behind that work, you can support me through Ko-fi.

Generative AI is a powerful tool. I openly use it to conduct research, challenge my thinking, generate drafts, and review my work. AI never owns what I think, or my personal judgement. Everything I publish is a reflection of my own thoughts and opinions.