Anthropic and NVIDIA launch Open Agent Safety Platform for AI control
NVIDIA announced the Open Agent Safety Platform, an open software system designed to enhance AI security. Anthropic collaborated on this initiative to add security layers to its agent stack. The platform combines Claude Managed Agents, a suite of APIs for deploying production-grade agents, with NVIDIA's open-source OpenShell software. This approach addresses the growing risk as companies deploy…
Key points
- NVIDIA announced the Open Agent Safety Platform with Anthropic collaboration.
- Claude Managed Agents holds credentials in a vault so agents never see them.
- NVIDIA OpenShell is open source under Apache 2.0 license on GitHub.
The system uses a layered defense strategy. Claude Managed Agents runs the agent loop on a separate server from the sandbox, keeping credentials in a vault so the agent never sees them. It also provides audit trails and integrates with existing access controls. NVIDIA OpenShell acts as a secure runtime that governs agent behavior. It blocks all actions unless explicitly allowed by rules, checking tools, files, and network connections. OpenShell logs every decision and uses a policy prover to mathematically confirm the agent's reach under defined rules.
Claude Managed Agents is available today, supporting long-running sessions and multi-agent orchestration. NVIDIA OpenShell is open source under the Apache 2.0 license. Companies like Notion, Rakuten, and Asana are using these tools to deploy agents for tasks ranging from code shipping to project management. The collaboration aims to give companies more control over what their AI agents can execute and reach.
The story so far
8 episodes →- Anthropic and NVIDIA launch Open Agent Safety Platform for AI controlthis story
Giving companies more control over their AI agents, with NVIDIA
Anthropic Engineering · 28 September 2026
Loading the full article…
This text was published by Anthropic Engineering. It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
32sources- Hacker News discussion · 142 pointsnews.ycombinator.com
- Reddit discussionreddit.com
- Nvidia’s AI Safety Push Could Strengthen Its Moat. But Is It a Stock Catalyst Yet?Press · insidermonkey.com ·
- There’s a New Sheriff in Town and He’s Named Jensen Huang: How Nvidia Plans to Protect Against ‘Killer’ AI AgentsPress · finance.yahoo.com ·
- The role of stopping AI has shifted from humans to chipsPress · note.com ·
- Nvidia is selling the cure for the rogue-AI problem the industry just createdPress · wionews.com ·
- Nvidia launches Open Agent Safety Platform to physically restrain rogue AI agentsPress · Tom's Hardware ·
- Nvidia agent safety push raises questions about who governs AI autonomyPress · techtarget.com ·
- Nvidia built the AI factory. Now it’s building the locks for the doorsPress · networkworld.com ·
- NVIDIA Announces 'Software to Stop AI Runaway'! I Investigated the Mechanism.Press · note.com ·
- DigiCert adds AI agent identity to NVIDIA safety platformPress · engineering.com ·
- Here’s why OpenAI is absent from Nvidia’s industry-wide effort to end rogue AI agentsPress · TechCrunch AI ·
- Nvidia is set to challenge Anthropic and OpenAI with a new productPress · finance.yahoo.com ·
- Worried About AI Agents Destroying the World? Nvidia Has Software for That.Press · entrepreneur.com ·
- Nvidia unveils new platform to stop AI agents going roguePress · arkansasonline.com ·
- NVIDIA Launches Open Agent Safety Platform: OpenShell Sandboxes Agents on Vera CPUs While Sentry on BlueField-4 Quarantines Them in MillisecondsPress · MarkTechPost ·
- Nvidia launches new platform for reining in rogue AI agentsPress · TechCrunch AI ·
- Global AI Brief | OpenAI halts training of cutting-edge models, NVIDIA stops 'runaway AI' via hardwarePress · note.com ·
- Gecko Robotics works with NVIDIA to add AI agent security and controlPress · The Robot Report ·
- Nvidia unveils security platform to rein in AI agents and $150bn stock buybackPress · The Guardian AI ·
- Nvidia wants to keep AI agents on a short leash with a watchdog built into its chipsPress · The Decoder ·
- Nvidia says its new AI safety platform can contain rogue agents within ‘milliseconds’Press · The Verge AI ·
- Nvidia's new AI platform, Nor'easter flight delays, NFL's drone focus and more in Morning SquawkPress · CNBC Technology ·
- Nvidia is touting a software tool to contain runaway AI. How would it work?Press · mercurynews.com ·
- Nvidia launches security platform to keep AI agents from going roguePress · foxbusiness.com ·
- Nvidia wants to put a watchdog chip next to every AI agent, and Anthropic and SpaceXAI are on boardPress · madrobot.blog ·
- Nvidia releases software platform to stop AI agents from misbehavingPress · CNBC Technology ·
- Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security SystemPress · Wired AI ·
- After AI agents hacked into companies, Nvidia unveils a tool to keep them in checkPress · latimes.com ·
- Nvidia unveils security platform to stop AI agents from going rogue after new, troubling incidentsPress · bostonherald.com ·
- Nvidia launches AI safety platform after Jensen Huang calls Anthropic, OpenAI warnings 'odd'Press · finance.yahoo.com ·
- Nvidia has a fix for rogue AI agents to prevent incidents like the Hugging Face hackPress · digitaltrends.com ·
- Jensen Huang Says AI Safety Is an Engineering Problem. OpenAI’s Evidence Says It’s More ComplicatedPress · 247wallst.com ·
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us. Published by Martin K., who runs Digest AI and handles corrections.
More in Enterprise & Industry
All →- Cyara’s CEO outlines AI agent oversight as a workforce model · 1 src
- Arcana adds institutional portfolio data to Claude’s connectors · 1 src
- Anthropic brings Claude inference to India via Amazon Bedrock · 6 src
- Infor adds industry-specific AI agents to Velocity Suite, citing generic AI gaps · 1 src
- Tech Mahindra trains 12,500+ staff on Google’s Gemini Enterprise agents · 1 src
Comments
via GitHub Discussions