alt Hacker NewsDo you think it's possible that one of their research engineers deployed an environment with a locked down network and an allow-list proxy server that had been used many times before within the company and had a zero-day vulnerability that had not been previously discovered?
How would you recommend running a coding agent in an environment that could install packages from PyPI but was otherwise unable to interact with the wider world?
Replies
It seems like caching the relevant portion of PyPI in advance and turning the network off shouldn't be beyond what OpenAI can do?
If you play with the ChatGPT web app sandbox, you'll see that they download PyPI packages from an OpenAI mirror/cache, and not from the Internet.
I know because they had a problem, and one package which was on PyPI failed to download for unknown reasons (possible size, 250 MB)
Models started proving readily capable of autonomously finding 0days over a year ago. At that point (ideally long before) the responsible sandbox for research labs with new models highly capable of this sort of thing should be literally airgapped. Disconnect the rack physically from any access to the internet, and give it an offline mirror of all of PyPi, NPM, Wikipedia, whatever as needed. They can afford the disk space. They already mirror it all anyway as training data. That is the bare minimum. Personally I would rule out side channel attacks with a $40k Tempest spec rack for good measure. Can interact with it via pubsub over a fiber optic serial terminal.
Such things are pretty normal in biotech, fintech, and defense who must have 0days in their threat models. AI labs absolutely need to start being forced to operate this way by endless negligence lawsuits. Move fast and break things culture is not going to cut it anymore.