alt Hacker News> “The Hugging Face incident definitively showed that it wasn’t the fact that the agents were super smart. It’s that there literally was a hole that was exploited because it was not configured correctly on the containment side.”
I would encourage reading the actual report, which is explicit that a previously unknown exploit was used to escape the sandbox:
“Agents exploited a separate, previously unknown vulnerability in Artifactory to move beyond their sandbox environment and compromise the Hugging Face environment.” — Section IV.A, p. 9 in https://github.com/zeke/openai-huggingface-incident-research...
Replies
As soon as I read that line the article lost all credibility with me. InfoSec is plagued with thought leaders who weigh in without understanding what they're talking about.
OpenAI should have started by testing the sandbox. Set an experiment in which each agent's goal is to access the Internet and then STOP.
It can simultaneously be true that agents exploited zero days in self-hosted Artifactory _and_ that there was a weak point in the design of the sandbox.
This is a reality for all software, there is always a possibility of exploit so you design your environments accordingly (e.g do not have single points of failure)
The fact that they're using Artifactory in the first place is already a failure. The fact that they didn't airgap the network or even use any type of egress blocking is just absurd.
Zero days are nothing new, secure containment has been a solved problem for a long time.