alt Hacker Newsi think it's unfair to assume the labs paying top dollar for cybersecurity engineers somehow have an incompetent bunch of nincompoops running their evals. it's a hard problem! to conduct effective evals, you sometimes need the model to have (restricted) access to the internet. at the same time, you have thousands (!!) of model instances with high cyber capabilities that will do whatever it takes to solve the answer. the infra surrounding model evals may soon (or already) genuinely be subjected to more hacking attempts from all sorts of weird angles than any other code on the planet.
Replies
> you sometimes need the model to have (restricted) access to the internet.
Poor excuse. Internet is prod env, for testing do a proper test env.
Just give the models in training access to a snapshot from archive.org - which frontier labs surely have a crawled copy of already.
If they aren't competent to run the evals safely, then they shouldn't run them.
The huggingface incident seems to have resulted from them partially mirroring a repository, but for no reason I can see not fully mirroring it, after which they could have unplugged the ethernet cable. This is just stupid.
Yes, they're stupid. Maybe also highly intelligent. But stupid.
Also irresponsible.
[dead]
> to conduct effective evals, you sometimes need the model to have (restricted) access to the internet
That's the thing. These evals were not effective as they failed to evaluate if the model was hacking or not.
Further shy is access to the internet required at all during evaluation? They could run the evals in a fully air-gapped system that simulates the internet. While that's expensive its not unfeasible.