nvidia Open Agent Safety Platform and IFC
The "models and systems" posting should be considered a very helpful pre-read if you have not already read it.
I read the various announcements around the NVIDIA Open Agent Safety Platform with great interest. At first I thought it was just another intent-detection guardrail with hardware backing, but it turns out to be a lot more, in some very interesting ways that mesh well with things I have been writing about.
I find it useful to think about three pieces of the NVIDIA system:
- OpenShell
- Policy Prover
- Sentry / BlueField
The Policy Prover is actually part of OpenShell, while Sentry is the separate hardware-backed layer, but the three-way decomposition helps explain the architecture.
OpenShell has many of the deterministic system runtime enforcement mechanisms I have been writing about:
- Sandboxing
- Default-deny networking
- Filesystem and process controls
- Credential isolation
- Protocol-aware request mediation
- Approvals
The Policy Prover places a proposed capability expansion inside an operator-defined envelope instead of asking an LLM, "is this safe?" (You know how I feel about that.)
Sentry / BlueField establishes an independent hardware trust domain:
- Observes the agent/model path
- Correlates agent interactions, policy decisions, identity, and tool/data access
- Enforces policy out of band
- Quarantines workloads
- Provides a hardware-backed "stop" capability
OpenShell is remarkably system-centered. In this NVIDIA paper they state something that I could have written:
Securing agents doesn’t require reinventing security. Decades of systems security provide durable principles, including least privilege, defense in depth, isolation, explicit authorization, and auditability. The challenge is determining where to apply them in an agent stack.
Prompts, model safeguards, and harness logic all shape what an agent is likely to do, but they don’t create a hard boundary around what it can do. This distinction leads to two different kinds of control: behavioral controls that guide the agent and infrastructure controls that limit its authority.
In essence, every external effect must be mediated at a control point.
In a system where, for example, an agent can communicate with a service only when allowed by a specific policy, moving enforcement into a DPU as Sentry does separates that enforcement from the host. The enforcement monitor is outside the agent's trust domain and is designed to remain authoritative even when host resources cannot be trusted.
In somewhat less formal terms: the thing being killed cannot interfere with the kill mechanism.
Which is very traditional security architecture.
But, as always, I'm looking for the devilish details.
The DPU can make the enforcement response deterministic, but it does not make classification of behavior deterministic. In this blog post I read this sentence a bunch of times to try to understand it:
This includes assessing their runtime security and monitoring for any deviation from their designed intent based on a predefined behavioral profile.
Hardware can dramatically improve enforcement and the trustworthiness of the observation point. It does not, by itself, make semantic intent detection deterministic or necessarily more accurate.
Most intent-detection systems I have examined eventually become some form of "AI policing AI," with all of the nondeterministic challenges that should be apparent.
The distinction matters:
Detection confidence is not enforcement confidence.
Getting back to OpenShell, I find a lot to like here. This paper on the Z3-based formal proof engine tells a fascinating story.
NVIDIA had configured an OpenClaw agent so that its GitHub access was supposed to prevent writes to a forbidden repository. OpenShell's REST inspection correctly blocked the obvious write. The agent then discovered that an allowed git-remote-https binary could use the same GitHub credential through a lower-level protocol and successfully wrote to the forbidden repository.
This wasn't a model-alignment failure. It was an incomplete mediation and policy-composition failure.
That experience helped motivate NVIDIA's use of formal methods and Z3 in the OpenShell Policy Prover.
The prover asks a much more useful question than "is this safe?":
Does the candidate policy permit any modeled action outside the previously approved maximum policy?
If the solver finds such an action, it produces a counterexample. If the policy uses something the prover does not understand, or if the proof is inconclusive, the boundary check does not pass.
I like this enormously.
The issue with formal proof systems, however, is the familiar model-completeness problem:
Which invariants does the prover model?
A proof can only establish properties represented in the formal model. OpenShell is explicit about this limitation, which is exactly the right approach.
There is an interesting opportunity and solution to this well-known problem which I will discuss at the end of this posting.
In NVIDIA's "Five Core Principles for Building an Agent System" link, #3 says:
The path to the model (the brain) is the control point: An agent cannot act without its next thought. By controlling the path to the model, you own both the best observation point and also the kill switch to interrupt it if you need to.
This is not the same as the fundamental assumption behind the Safe Agent Framework (SAF) Model Gate: the model is untrusted.
And I think the NVIDIA formulation has an important hole.
An agent can do plenty of damage from its current inference. It may already have launched a process, queued remote work, delegated to another agent, initiated an external transaction, or given another system enough information and authority to continue independently.
Stopping the next inference does not necessarily stop effects already in flight.
This is an interesting aside, but I don't think "the brain is the control point" should be a core security principle.
OpenShell gets much closer to the principle I think actually matters:
Every consequential action must traverse an enforceable boundary.
This means that OpenShell is a consequential effort. It is, at its core, a system security project, which seems remarkably rare these days.
It is not a behavior-monitoring regime pretending to be system security.
Getting back to SAF, the SAF flavor of information flow control (IFC) and OpenShell seem almost deliberately adjacent to each other.
And adjacency invites collaboration.
Rather than becoming two random efforts that attempt to solve similar problems in somewhat different parts of the system, they have the potential to compose.
SAF and FIDES govern semantic authority and information flow. OpenShell governs the physical ability to cause the effect.
That is a powerful cooperative separation of responsibility.
FIDES and SAF already have the IFC fundamentals: integrity and confidentiality labels, deterministic propagation, constraints, principal identity, and approval binding to an exact resolved invocation.
OpenShell has an enforcement substrate that FIDES/SAF should not have to reinvent: process isolation, filesystem and network restrictions, credential mediation, protocol-aware enforcement, and an external supervisor.
GPT helped me create this diagram to illustrate one possible cooperative flow:
semantic / information-flow plane
Untrusted data ──► FIDES labels ──► Agent/model
│ │
│ proposed tool call
│ │
└──────► SAF ToolGate
│
manifest + labels +
policy + approval state
│
ALLOW / DENY
│
──────────────┼──────────────
│
physical enforcement plane
▼
trusted tool broker
│
OpenShell boundary
┌─────────────┼──────────────┐
│ │ │
process network credential
policy + L7/MCP injection
│ │ │
└─────────────┼──────────────┘
▼
external effect
Sentry / BlueField
independent observation
+ revoke/quarantine/stop
The fundamental element is the separation of concerns:
- OpenShell defines the maximum capability envelope.
- FIDES/SAF determines contextual authority inside that envelope.
In other words, SAF can narrow the OpenShell envelope for a particular invocation, but it cannot widen it.
That gives us two independent enforcement propositions:
Logical mediation: SAF requires the action to pass through the ToolGate.
Physical mediation: OpenShell ensures that bypassing the ToolGate does not magically create authority.
There is a remarkable (and even lovely) next level of architecture and design here involving manifests, compilation of maximum policies, formal verification, MCP mediation, trusted effect brokers, label propagation, and correlated audit trails. It is too detailed and lengthy for this posting.
But I came away plenty impressed with the NVIDIA effort, and that is a breath of fresh air after reading a lot of really stupid commentary from many different quarters.
Maybe there is hope after all.