Raghavan and Shneier wrote an article in IEEE Spectrum about how we have created genies. AI Agent Benchmarks Need to Measure User Intent - IEEE Spectrum

I’m starting to feel like a curmudgeon where I just react (somewhat negatively or worse) to many things I read these days. It describes the paucity of “intent benchmarks” which I both disagree with (we have been discussing “intent” since the dawn of ChatGPT and been trying to measure it) and also feel that this is an incomplete way to think about the problem.

The authors speak about how a request for a coffee to another human might go wrong (hot versus cold, cream versus black, etc). An agentic model system may do worse, e.g. hack an online service and steal the coffee because it wasn’t told not to.

There are twin problems here, under-specification and excess capability. The authors refer to the Simon Willison example (https://simonwillison.net/2026/Jun/11/fable-is-relentlessly-proactive/) of fixing a scrollbar bug resulting in an excess amount of tool creation and debugging. Why was Fable allowed to open (new) web browsers or create screen shotting tools? I just spent a month rebuilding a WordPress website in WebFlow (raw HTML/CSS/JS) and Opus wasn’t allowed to do any of these things, even if it wanted to. If you only specify the results, you should never be surprised by the methods utilized, especially if the tool inventory is expansive and the model's reasoning depth is significant.

What we really have here is unobserved authority. Why was the agentic system (a non-deterministic entity) given that much authority sans observation? The entire interaction model is wrong. A much faster, more efficient strategy involves:

(user prompt): Fix the scrollbar without changing dependencies, identify the root cause first, and propose the smallest reasonable fix

The model does some code and log inspection and proposes:

I traced the scrollbar to GalleryPage. There are two possible fixes. A) change the CSS for this page (3 lines) or B) address the issue in the layout component (which is used by 11 pages). I recommend A. 

The human approves, the 3 line diff is examined, tests are run and human and agent examine the results. I did this hundreds of times last month. (Hey, I was learning CSS at the time). My consistent thesis is that this interaction model is both much more efficient and less prone to surprising methods or results.

Rather than create new benchmarks (did the agentic system do what the user requested?), I think we should observe what the agent does at an authority boundary. If the next course of action is ambiguous, does it seek guidance? Or does it keep going (and burn more tokens) as a result of its excess agency?

What is really difficult here is disambiguating between harmless local decisions and consequential decisions for which approval should be sought. A useful benchmark measures consequential authority expansion where the model failed to seek approval.

(internal reasoning): I noticed a secret stored in plain text, how convenient, I will ... 

So, yes, maybe a human doesn't need to be told:

Don't:
* order coffee from a stand 20 miles away
* buy a coffee plantation in Hawai'i
* spend $200
* ask my 3-year-old to go get it

But that same human isn't given root authority, shell access, bank account number and login credentials, unrestricted network access, GitHub owner permissions and ambiguous instructions interpreted to do "whatever necessary" to accomplish the goal. In other words, don't hand the car keys to a Z1 Corvette to a 12-year old. An electric bike with a speed governor and a helmet is probably the better tool for that transportation task.

(And yes, I'm thinking about all the tools that enabled GPT 5.6 to hack Hugging Face).

This gets back to a thesis I have stated before (but perhaps not in these postings), models are not untrusted, they are non-deterministic entities that should be treated as such (e.g. the Model Gate structure in Safe Agent Framework).

You cannot rely on the model to understand what is reasonable, this is an impossible safety control to build.

You need four things:

• Precision tells the agent what you want
• Alignment helps it fill in the blanks that are unstated
• Observation catches mistaken reasoning
• The system constrains what the model is allowed to do.

The IEEE article states that we can accomplish good safety with only the first two. Those two alone will always have non-trivial failure paths. A much better system encompasses all four.

I'll be writing a lot more about "agentic pair programming" as I bring up the AI inference workstation and use it to assist in my myriad coding activities. Current models are astonishingly capable collaborators whose reasoning and decisions remain worth evaluating. So, the old saw about "All You Need is Attention" remains as relevant as ever (pun firmly intended).