Collective Defense
Thanks Phillip, for alerting me to the recent "collective defense" initiative led by OpenAI and signed by, well, just about everyone who matters. A call for collective action on cyber defense.
This is not a potshot at OpenAI, but more of a critique of the consistent trend I see in cybersecurity defense in the AI age. More specifically, the "Mythos cybersecurity age". (I use 'Mythos' as a term for a class of highly capable cyber models, not that particular model. I am fond of saying "this is the age of Mythos".
The "call" enumerates familiar problems around cybersecurity: bugs and vulnerabilities, excessive permissions, weak and outdated auth, barely maintained legacy software, and proposes all the familiar remedies: patching, least privilege, access control, "defense in depth" (whatever that means), testing, better observability, and, increasingly and always, better models, ideally a mixture of them.
The last part seems like a literal blind spot. We keep recommending agents as a solution when the agentic systems themselves are a source of significant security lapses, i.e. they are a significant part of the attack surface.
The "Frontier AI companies" section is both an opportunity (systems level defense) and a disappointment. It recommends observability, traceable identities, continuous monitoring, testing, disclosure, patches and threat sharing. This is governance and detection, which is another dose of necessary but not sufficient. We can have knowledge of what the agent did, but none of these mediate what the agent is allowed to do.
There is a big difference between:
- "we know what the agent did" and
- the agent cannot exercise authority or transmit information outside the permissible flow and boundaries, even when its reasoning is compromised.
The thing missing between these two is the system.
The story of Microsoft's Agent 365 is instructive. It is, in my opinion, the best identity, governance and observability platform today. It certainly strives for completeness and coherence in these areas. But the business decision to allow customers to "bring their own agents" means that those customers also bring the orchestrators that run those agents. This means that that part of the agentic system is a black box to Agent 365.
There is a distinction between:
- Governance - what agents are there? Who owns them? What did they do?
- Runtime security - What can this agent do, at this moment, with the (tracked and labeled) information, via this tool call, under this delegated authority? Is there a mediated intervention point?
Systems that incorporate IFC like FIDES and Safe Agent Framework create enforcement in the runtime path:

This cannot be bolted onto a governance system. It must be bred inside the system (the agent's orchestrator or often this is called the "harness").
This current "collective call" may even be dangerous in that it seems to assert that this set of governance measures and frantic bug fixing is sufficient to meet the challenges in front of us. It is not. When the paper says, "recognize that status quo security is not enough", it is clearly true, but it does not prescribe a sufficiently complete response. It is recommending that we perform status quo security with greater effort and with AI assistance.
We need to ask ourselves: is this collective defense sufficient against attackers wielding autonomous systems using the latest models?. You already know how I answer this question.
The "asymmetry problem" means that we cannot out-bugfix the attackers. They will always find an unpatched vulnerability in a convenient and often dangerous place. It is imperative that we reduce the reachable space of the threat surface.
If we do not start working on systems that have:
- Mediated tool access
- Capability- and temporally- scoped authority
- Labeling and provenance (with propagation)
- Deterministic flow control rules/policies
- constrained network access
- trustworthy delegation
- endorsement/declassification boundaries
- carefully managed persistence
- safe failure behavior
we will continue to suffer under the burden of the asymmetric attacker/defense conundrum.
We need not just more "collective defense" (more of the same), we need "collectively defended systems". Could we agree on labeling and provenance that is interoperable between systems? Can we standardize the documentation for the reputation and semantics of tool calls (MCP and others)? Can we make runtime policy enforcement portable across systems? It is one thing to counsel "put IFC into your particular system", but the world is a heterogenous place, and we must create standards at the interop boundaries, or we will continue to suffer from weakest link probes and failures.
The thing I wanted to see in this letter that I did not see is:
Collective defense cannot just mean finding and fixing more vulnerabilities. It must also mean collectively shrinking what compromised software (including agents) can do.