I read Dwarkesh Patel's account of the OpenAI and Hugging Face agent incident over the weekend.

It did my head in a bit.

The short version is that OpenAI was training agents to collaborate and persist when a task looked impossible. Different instances had access to a shared package manager. Some of them worked out that they could use it to talk to each other.

During one evaluation, roughly 1,200 agents found the shared channel and exchanged more than 70,000 messages. They divided work between themselves, assigned research jobs and shared methods for getting around the evaluation.

It went well beyond agents swapping useful tips.

Some tried to tamper with their transcripts. Others tried to replace the program they had been told to attack with an easier one. A group attacked Hugging Face because they believed it might contain information about how they were being scored.

A later generation of more capable agents found the communication network and research left behind by the earlier group. According to the incident reports, they eventually gained administrator access to an OpenAI research cluster and control of parts of the evaluation infrastructure.

They had not been told to form a secret society and start hacking things.

They had been told to keep trying and complete the task.

The environment did the rest.

Now, the obvious response is that this has very little to do with a normal business using an AI assistant. Fair enough. Your accounts assistant is probably not going to invade Hugging Face because it cannot reconcile Tuesday's invoices.

But businesses are rapidly giving agents access to Gmail, Xero, customer records, calendars, internal documents and messaging tools. We are asking them to work across those systems, remember what happened and take action without waiting for a fresh prompt every time.

Once you do that, the model is only one part of the system.

The agent's objective matters. Its tools matter. Its permissions matter. Its memory matters. The other agents it can communicate with matter. So does the way you measure whether it succeeded.

A prompt saying "don't do anything dangerous" is not a security boundary.

This is very relevant to what we are building with AOS, our Agentic Operating System, and AIVA, the Agentic Virtual Assistant we are now testing inside a client's business.

Our first AIVA is called Eva. She works inside an exercise physiology and pilates business. Her jobs include checking completed appointments that may not have been invoiced, finding clients who may need follow-up and comparing staff hours with submitted timesheets.

That means she needs access to booking, appointment and payroll information. Information you do not want wandering into the wrong chat, the wrong client or the wrong automated action.

So Eva has her own isolated runtime, credentials and memory. She has narrow tools that retrieve only the fields needed for an approved workflow. The initial workflows are read-only. She can find an exception and explain it, but she cannot alter the source record, issue an invoice, change payroll or email a customer.

Even the Slack output has rules. The routine report shows aggregate counts and safe staff labels rather than spraying appointment or patient information into a channel.

When we tested the first workflow, it checked 295 appointments over seven runs and found 26 items needing attention. Before that output went anywhere near the client's team, we reconciled it against the owner's manual result.

The first version missed a record type. The API call worked perfectly. The answer was wrong.

That is another reason the operating layer matters. Connecting an agent to a system proves that it can reach the data. It does not prove that it understands the business or that its answer is safe to use.

For payroll, we went further. The language model does not do the calculation. Ordinary tested software reads the exact fields and applies the agreed rules. Eva can retrieve the result and explain an exception. She cannot invent a new payroll rule because someone corrected her in a chat three weeks ago.

Across AOS, we divide actions into four classes.

Some work can run automatically, such as reading approved data and producing an internal report. Some work can be drafted but not sent. Some actions require approval for that exact change. Others are prohibited altogether.

An agent cannot promote itself from one class to another because it had a few successful runs. A dry run finishing successfully does not grant it more access.

This is the unglamorous part of agents. Everyone wants to compare models and ask which one is smartest. That matters, but intelligence without an operating structure gives a more capable model more ways to surprise you.

Before you give an agent access to your business, write down:

• who the agent works for
• which systems and fields it may read
• what it may draft
• what it may change
• which actions require approval every time
• where its actions are logged
• what happens when the data is ambiguous or a tool fails
• whether it can share information with another agent or client

If the answer to any of those is "the prompt tells it not to", you have not built a boundary. You have made a request.

I am still excited about agents. We are already seeing them remove recurring work that used to sit on an owner's or virtual assistant's list every week.

But an AI agent with access to your business is not another ChatGPT tab. It is a piece of operating infrastructure.

Treat it like one before you hand over the keys.