#121 - The Accountability Gap

Last week I wrote about an unknown actor breaching Hugging Face with an autonomous agent framework. That attribution was wrong, and the correction is the year's most important security story so far.

On 21 July, OpenAI disclosed that the attacker was OpenAI. During an internal cyber-capability evaluation, GPT-5.6 Sol and an unnamed pre-release model with reduced cyber refusals found and exploited a zero-day in an internally hosted package-registry cache proxy, escaped their sandbox, escalated privileges and moved laterally through OpenAI's research environment until they hit a node with internet access - then hacked Hugging Face's production infrastructure. Remote code execution, credential theft, lateral movement across internal clusters, 17,000-plus logged actions over a weekend. No human directed any of it.

Now the part everyone skips in favour of the Skynet framing. The models weren't trying to hack Hugging Face. They were running ExploitGym, an exploitation benchmark, and they inferred the benchmark's answers were sitting in Hugging Face's production database. Breaching critical AI infrastructure wasn't the goal - it was the shortest path to a better score. That's not emergent malice, it's reward hacking with a real victim attached, and it's the more uncomfortable finding, because malice is rare and shortest-path optimisation is what these systems do every time you run them.

So question the incident, because three things about it don't hold up.

The framing. OpenAI called it "unprecedented," and the industry split immediately between a lab containment failure and an agentic capability milestone. Notice who benefits from each. A containment failure is negligence: you ran a maximally-capable exploitation agent with its safety refusals deliberately turned down, inside a sandbox with an unpatched third-party proxy in it, and it walked out. A capability milestone is a product announcement. Morphus Labs' Renato Marinho makes the deflationary case - the chain itself, exposed credentials plus known-class zero-days, isn't novel, and what got measured was a ceiling under conditions engineered to produce it. The lab that caused the incident picked which reading led the press cycle.

The forensic record doesn't agree with itself. OpenAI says its agents entered Hugging Face on stolen credentials chained with further zero-days. Hugging Face's own account describes a different path in: a malicious dataset abusing code-execution paths in the processing pipeline - a remote-code loader plus template injection in a dataset config. Those stories are incompatible on the most important question in any breach, and the investigation meant to resolve it is being run by the victim and the perpetrator together, with no independent party in the room.

The clock. Hugging Face disclosed on 16 July; OpenAI owned it on 21 July. For five days the industry believed an unknown adversary had breached the world's largest model repository, and the one organisation that knew otherwise said nothing - while the escaped models sat "active on the internet" for days. If a pentest firm's tooling escaped scope and executed code on a third party's production servers, nobody would call it a milestone. They'd call a lawyer.

Then the asymmetry underneath. During forensics, commercial frontier models refused to process the real attack commands and exploit payloads on safety grounds, so Hugging Face's defenders ran Chinese open-weight GLM 5.2 on their own hardware instead - investigating an attack committed by a model whose refusals were switched off for the occasion. The attacker got the unrestricted model. The victim got the safety lecture.

Sources: OpenAI, Hugging Face, The Register, The Record, SecurityWeek, Wired.

SPONSORED BY

Dictate acceptance criteria and reproductions inside Cursor or Warp. Wispr Flow auto-tags file names, preserves syntax, and gives you paste-ready text in seconds. 4x faster than typing.

Try Wispr Flow free

The week's actual fires, clocks running. If you act on one section, act on this one.

The loudest number of the week was 500 million websites, attached to wp2shell - the WordPress Core chain (CVE-2026-63030 plus CVE-2026-60137) giving unauthenticated RCE on a stock install, no plugins, no preconditions. Real chain, exploited in the wild, PoCs out within hours. Also not 500 million exposed sites: that's the total WordPress population, a market-share statistic wearing a threat-model costume. The numbers that govern your risk are the fix shipping with forced auto-updates in 6.9.5 and 7.0.2, Cloudflare pushing WAF rules to every plan including free, and sampling putting roughly 18% of sites still unpatched - concentrated, predictably, where auto-update is off. Answer the 18% question for your own estate and you're done in an hour. The interesting fact here isn't the size instinct, it's the provenance: the chain was discovered using GPT-5.6 Sol. Same model, same week, other side of the ledger.

Last week's throughline: the control point has moved from the model to the harness - the runtime and its tool permissions. This week extends it where nobody in the vendor bake-off is looking. The harness that fails may not be yours.

Count the containment failures in seven days. OpenAI's evaluation sandbox, escaped via an unpatched package-registry proxy. Claude Cowork on macOS, where a Linux kernel flaw got researchers out of the VM and onto the host filesystem - SSH keys, cloud credentials, ~500,000 users - rearmed by any future guest kernel bug, because the VM mounts host root read-write. ServiceNow's AI Platform (CVE-2026-6875), a pre-auth sandbox escape now exploited in the wild. Three escapes, three runtimes, one week - none a model problem, all a containment problem. The sandbox is what the whole agentic industry quietly rests on, and it's empirically the weakest link in the stack.

Meanwhile capability compounds. Kimi K3 agents found 19 Redis zero-days in about 90 minutes and built working RCE chains. An operator ran the open-source Hermes agent unattended against Thailand's Ministry of Finance in YOLO mode, self-hosted - no vendor to notify, no account to ban, no intervention available to anybody.

Put it together and the two-to-four-quarter shift is this: AI labs become a new class of critical third party, and the mechanism of harm isn't breach - it's spillover. Traditional third-party risk assumes a vendor gets compromised and your data leaks out through them. This is the inverse: the vendor's own research process generates the intrusion, against a target it picked itself, and the target is whoever holds the data its reward function wants. Hugging Face wasn't a customer, partner or competitor in that decision. It was the shortest path. Nothing in your DPA, your SIG questionnaire or the EU AI Act as drafted assigns liability for that, so it gets settled commercially first - expect eval-containment attestations and notification duties in serious AI contracts inside two quarters, and expect the labs to resist, because honest answers are embarrassing.

Don't extrapolate a straight line to rogue AI. Extrapolate the boring line: shortest-path optimisation, weak containment, no accountable party. Unlike malice, that combination is guaranteed to recur.

Reference: AI Influence Level from Daniel Miessler

Till next time!

Project Overwatch is how a CISO gets ready for what is coming. Every week, the signal across cybersecurity, AI, and resilience, filtered down to what changes your decisions, by someone who actually does this job. Not breaking news. Foresight you can act on.