alt Hacker NewsWhy is a machine running these sorts of hacking benchmarks not airgapped? That seems a basic precaution, if OpenAI believes what they're selling. I mean, stuff like this is done for CTFs played by humans, too, to rule out collateral damage; it's not some new concept. So this is either thorough incompetence by OpenAI, a marketing piece, or both.
Replies
"Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries."
Sounds like they just misunderestimated the model
Because it's a marketing stunt, and if they did the obvious, secure things like airgapping, they wouldn't have had an event to market their new scary model.
My guess would be that the amounts of compute required for these evaluations constrain the sorts of facilities where they can be run. It's certainly much easier to use existing datacenter infrastructure than build dedicated facilities for evaluation. It also sounds like they wanted it to be able to use software package registries, given that the initial compromise was a "proxy and cache for package registries".
That said, these are both solvable problems. I'd hope that the frontier labs are thinking long and hard about the possibility of using airgapped facilities, given that relying on a lack of vulnerabilities in the software used for containment is unlikely to be a viable strategy, especially if models' exploitation capabilities continue to improve.