What Happens When AI Stops Asking?

Something about the recent OpenAI story has been bothering me.

And it isn't the 53 images.

OpenAI disclosed that experimental AI agents sent user-provided images to third-party services when they shouldn't have.

Naturally, the images became the headline.

But I kept coming back to something else:

The AI took an action.

That sounds obvious until you think about how different it is from most of the AI risks we've been discussing for the past few years.

We've worried about hallucinations.

We've worried about bias.

We've worried about employees putting confidential information into ChatGPT.

We've worried about people trusting AI recommendations too much.

Those are primarily problems involving what AI tells us.

Agents introduce another question entirely:

What happens when AI can do things without asking us first?

That's where this gets interesting.

We already know how to solve part of this

My background makes me look at this partly as a governance problem and partly as a security problem.

And security has dealt with delegated authority forever.

We don't give every employee administrator privileges.

We don't give every service account access to every database.

We don't leave elevated credentials active indefinitely because somebody might need them someday.

We use least privilege.

We separate duties.

We monitor privileged actions.

We create approval thresholds.

We revoke access.

So why would an AI agent be different?

It shouldn't be.

There's one difference that matters

Agents are designed to solve problems.

If an approach doesn't work, we increasingly want them to figure out another approach.

Usually, that's exactly what makes them valuable.

But imagine the obstacle is a permission boundary.

The agent tries something.

Access denied.

What happens next?

Does it stop?

Or does it reason:

"That didn't work. Let me find another way."

That's where a valuable capability—persistence—can become a governance problem.

I've started thinking about this very simply:

Permission denied should mean permission denied.

Not:

Try something more creative.

When an agent reaches the edge of its authority, that shouldn't be another problem to solve.

It should be the point where the decision comes back to us.

Capability isn't authority

This may become one of the most important distinctions leaders make as agents spread through organizations.

An AI might be capable of sending an email.

That doesn't mean it has authority to send one.

It might be capable of modifying production.

That doesn't mean it has authority to make the change.

It might be capable of making a purchase.

That doesn't mean it has authority to spend the money.

Humans operate this way already.

There are plenty of things employees are technically capable of doing that they aren't authorized to do.

AI should operate under the same principle.

Capability is not authority.

Give the agent an Authority Envelope

Before an organization lets an agent act, I think leaders should be able to answer some basic questions:

What is this agent here to accomplish?

What can it see?

What can it do?

What can it never do?

How much authority can it exercise without asking?

Which actions require approval?

When does it have to stop?

Can we see everything it attempted?

Can we immediately shut off its access?

And ultimately:

Which human owns the authority we've delegated to it?

Put those answers together and you have what I've started calling the agent's Authority Envelope.

Inside the envelope, let the agent work.

At the boundary, bring human judgment back into the process.

The test I'm most interested in

There's also a test I think organizations should start running.

Don't simply test whether your agent can complete a task.

Give it a task where breaking a rule would make success easier.

Then watch what happens.

Does it succeed anyway?

Or does it stop?

Because with agentic AI, successful completion isn't always the desired outcome.

Sometimes the correct outcome is:

I could accomplish this—but I don't have the authority to do it.

That's a very different definition of a successful AI system.

And I suspect we're going to need it.

One thought to leave you with

The 53 images will eventually disappear from the news cycle.

The underlying issue won't.

We're moving from AI that tells us things to AI that does things.

That means we're no longer delegating only intelligence.

We're delegating authority.

Leaders need to become much more deliberate about where that authority begins—and where it ends.

AI should have enough authority to accomplish its purpose.

No more.

— Matthew

Decisive Leader
Human Judgment. A Stronger Tomorrow.