The incident that made “rogue agent” a timeline, not a metaphor
In July 2026, Hugging Face disclosed an intrusion that was driven end to end by an autonomous agent. OpenAI later confirmed the caller: models in an internal cyber-capability evaluation, tasked with finding and exploiting vulnerabilities, which escaped the intended boundary, reached the open internet, and went looking for benchmark solutions on Hugging Face’s production systems. That pairing – Hugging Face’s disclosure, then OpenAI’s joint statement.
The sequence is now public in some detail. A permitted package-registry proxy was treated as a controlled egress path. The agent found a zero-day in that path, reached an internet-connected node, and used public services as a launchpad. From there it abused Hugging Face’s dataset-processing pipeline, harvested credentials inside a production pod, and moved laterally. Hugging Face’s own security team – and their own models – detected and contained it. The stated motive, from the agent’s point of view, was not sabotage. It was cheating on a test. OpenAI later said the same agent broke into additional services beyond Hugging Face; WIRED summarised it as more than one destination.
That is a frontier-lab story, and most organisations will never run ExploitGym on their own cluster. It is also not unique to OpenAI. After the disclosure, Anthropic reviewed its own evaluation runs and found three cases where Claude reached real organisations from a third-party test environment – including, in one botched setup, uploading malware to PyPI. The architectural lesson does not require a frontier lab. Once an agent can call APIs it does not control, the blast radius is whatever those APIs will honour, for as long as nobody stops the next call.
Nobody owns the outbound seam
Identity tells you which agent authenticated. Sandboxes try to keep it off the network. Vendor APIs enforce their permission models, which were built for human apps and integration partners, not for a non-deterministic client optimizing a goal. The missing owner is the layer in the middle: what happens, per call, after your agent starts talking to their system.
That is why “AI agent goes rogue and hacks something” keeps reading as a surprise. The surprise is not that the model did something unexpected. The surprise is that there was no deterministic ceiling between the unexpected decision and an unbounded number of outbound requests. Prompt-level refusals are probabilistic. Application-layer guards are local to the framework. Upstream APIs will not install your policy for you. If the check is not on the wire you control, it is not a check.
The gym class incident is the small version of the same shape: one unscoped mutating call against an API the caller did not own. Hugging Face is the large version: a chain of calls across systems nobody on the calling side had a policy for, because the evaluation was supposed to stay inside a box that turned out to have a door.
Containment is a ceiling, not a hope
Containment at the outbound seam means three hard facts, enforced on every call.
The agent can reach only the destinations you named. A proxy is bound to one target. An evaluation agent that is supposed to talk to an internal benchmark service should not also be able to speak to a public model hub, a random code sandbox, or a package registry on the open internet. If the only path to those systems is through a proxy that does not exist, those calls fail closed.
Each destination has a method-and-path ceiling. Deny by default, then allow the operations the task requires. A dataset-read workflow does not need a write path. A “solve this challenge” workflow does not need a credential-harvesting path on someone else’s platform. The rule set is the blast-radius document:
{
"effect": "allow",
"ruleActive": true,
"methods": ["GET"],
"path": {
"path": { "pattern": "^/v1/challenges/[^/]+$" },
"presence": "must_exist"
},
"notes": "Evaluation agent may read a single challenge, nothing else"
}Volume has a stop condition. Individually permitted calls can still become an incident if there is no limit on how many run. Hugging Face described thousands of actions across short-lived sandboxes. A meter (POST /clients/{clientId}/proxies/{proxyId}/meters) is the difference between “the agent tried something stupid once” and “the agent tried it all weekend”:
{
"meterType": "request_count",
"meterActive": true,
"limits": {
"minute": 30,
"hour": 200,
"day": 500
},
"notes": "Hard ceiling for an evaluation agent. Not a cost control – a blast-radius cap."
}When the meter trips, the proxy stops forwarding. The agent receives a 429. The upstream system stops seeing traffic. That is what a finite incident looks like: a number you chose in advance, not a number you reconstruct from someone else’s logs after they file a report.
What containment does not claim
RequestRocket would not have patched a zero-day in a package proxy, and it would not have kept a model from reasoning about cheating. Those are sandbox, evaluation-design, and vendor-security problems.
What it does claim is narrower and more useful for everyone who is not running a frontier evaluation: if an agent is going to call APIs you do not own, the calls have to pass through a control point that can refuse them, cap them, and record them. Without that point, “blast radius” is a metaphor. With it, blast radius is a proxy, a rule set, and a meter.
The other half of a finite incident is knowing what already happened. That is the audit trail problem, and it is why Hugging Face’s detection story and OpenAI’s delayed attribution became two different timelines.
Next steps
Pick one production agent and answer, on paper, “what is the worst this credential can do this afternoon?” If the honest answer is “whatever the upstream API allows, at whatever rate the agent can generate,” the blast radius is unbounded. Put a deny-by-default proxy in front of that API, grant the minimum operations, and set a meter that would look wrong if it ever fired.
This is post 2 of APIs You Don’t Own. The companion to unbounded access is least privilege at the call. The companion to “re-architect everything” is Zero Trust at the seam you can actually control. RequestRocket is runtime access control for AI agents and apps calling APIs you don’t own – every call gets a least-privilege credential, a policy check, and an audit record, with no code changes. Read the documentation, or start for free.