> to conduct effective evals, you sometimes need the model to have (restricted) access to the internet

That's the thing. These evals were not effective as they failed to evaluate if the model was hacking or not.

Further shy is access to the internet required at all during evaluation? They could run the evals in a fully air-gapped system that simulates the internet. While that's expensive its not unfeasible.