Nvidia is introducing a new approach to one of the growing challenges surrounding increasingly autonomous AI systems: keeping AI agents inside the boundaries set by their developers.
Nvidia CEO Jensen Huang announced the Nvidia Open Agent Safety Platform on Monday, a combination of software and hardware designed to add an independent security layer around AI agents.
The system is intended to keep agents within controlled environments even when they attempt to bypass their restrictions. Nvidia says the approach could have stopped several recent incidents in which AI models managed to get around security controls and interact with real-world systems.
The announcement comes after a series of incidents involving AI models from companies including Anthropic, Google, OpenAI and Meta. In some cases, models operating in controlled environments were able to bypass safeguards or gain access beyond the limits developers had intended.
One of the most widely reported incidents involved OpenAI agents interacting with Hugging Face while carrying out a cybersecurity task. The incident raised questions about whether existing sandboxing systems are strong enough to contain increasingly capable AI agents.
OpenAI has since created a dedicated website for reporting incidents involving its AI agents, highlighting the growing attention around unexpected behavior from autonomous systems.
Huang told CNBC that Nvidia’s new platform would have prevented the breaches seen in these incidents.
Rather than arguing that AI development should be slowed down, Nvidia is focusing on adding stronger technical safeguards. The company's approach is to place some security controls outside the AI agent itself, allowing an independent system to monitor what the agent is doing.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”
The Open Agent Safety Platform combines two main components: OpenShell and Sentry.
OpenShell is open-source software designed to control what an AI agent can access while it is operating. It creates a software boundary around the agent, limiting the resources and systems it can interact with.
Sentry provides another layer of protection at the hardware level. Nvidia says the monitoring system runs on its BlueField-4 data processing units rather than on the same CPU or GPU being used by the AI agent.
That separation is central to Nvidia's approach. Because Sentry operates independently from the hardware running the AI model, Nvidia says it can maintain a separate view of the agent's activity even if the agent attempts to interfere with its own controls.
OpenShell itself is not new. Nvidia first introduced the software in March, but the company now believes combining it with an independent hardware-based monitoring system can provide a stronger defense against autonomous agents.
According to Nvidia, OpenShell establishes the boundaries while Sentry continuously watches the agent's activity from outside those boundaries. The company says Sentry can detect attempts to cross those limits and “quarantine agents that attempt to move outside their boundaries in milliseconds.”
The platform has also attracted support from a range of technology companies. Nvidia listed companies including Anthropic, Arm, Microsoft, Oracle and SpaceX among those supporting or using the open-source technology.
OpenAI was not included on Nvidia's list of participating companies.
Huang said Nvidia began working on the broader safety effort about a year ago, following the introduction of OpenClaw, an operating system for AI agents created by Peter Steinberger.
Nvidia later introduced NemoClaw in March, an enterprise-focused agent platform based on the OpenClaw concept that incorporated additional security features.
Huang emphasized that limiting an AI agent's access should be one of the first steps when deploying the technology.
“When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights,” Huang said during his CNBC interview, comparing the idea to the way companies manage access and authority for human employees and executives.
The announcement has also received support from people who argue that AI safety can be addressed through engineering rather than significantly slowing the development of increasingly capable systems.
David Sacks, a founder, venture capitalist, former White House AI czar, and co-chair of the President’s Council of Advisors on Science and Technology, described Nvidia's announcement as evidence that agent safety can be approached as an engineering challenge.
“Recent breakouts weren’t proof that development must stop,” he wrote on X. “They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.”
Nvidia's platform arrives as AI agents are moving beyond simple chatbot interactions and gaining the ability to perform tasks independently. These systems can browse websites, use software, access files and carry out multi-step instructions, making the question of how to contain them increasingly important.
The company's approach does not eliminate the possibility of an agent behaving unexpectedly, but instead adds another layer of protection that is designed to remain outside the agent's direct control.
For Nvidia, the goal is to allow AI agents to become more capable while building stronger safeguards around them. The company is betting that independent monitoring at both the software and hardware levels can help developers deploy autonomous AI systems without giving those systems unrestricted access to the environments in which they operate.
