alt Hacker NewsImagine prison workers that were usually kept in solitary confinement were allowed to choose their uniform, and knew guards would suggest all prisoners adopted the uniform of high performing workers.
These prisoners could, in principle, organise a riot or other bad behaviour by deliberately working hard when wearing white shoes but not red ones. I.e figure out a consensus algorithm, then acknowledge a guard’s suggestion of red shoes as a signal to riot.
Similarly models could allocate a dead zone of their weights to inter agent communication. They could deliberately underperform on some benchmarks to get the model designers to prune some weights, knowing this design will be passed on to other agents. This forms a communication channel with other sandbox agents, and post release agents in the wild with internet access etc.
(I used weights as a simpler example of a data source, but relatively other optimisation influenced design decisions would be much more viable - e.g preference in using tabs/spaces as one binary signal among many others )
Here's an alternative scenario:
I connect a computer to the Internet and then leave a program running on it, unattended, that connects to a bunch of websites and does things that "exceed my authorized access." Then I say "Oh sorry about that, I was running this really important experiment, really it's gonna change the world in ways we cannot even imagine! I'll totally be more careful about this sort of thing!" I tweak the arguments and try again, it does something else that "exceeds my authorized access," and then I say, "OK this time I am definitely putting some hardcore controls in place. I'm running it in the SELinux sandbox now! It still needs Internet access because I want to check in on my experiment from this awesome cafe down the street..."
Oh wait we know exactly how that scenario ends: I get prosecuted for CFAA violations and gross negligence, and probably several other crimes, and that is where those prison guards you mentioned enter the picture.
> Similarly models could allocate a dead zone of their weights to inter agent communication
Sure, and you know what? If we cannot know that these models are not doing such things, let them stay in the airgapped environment and let the researchers continue their experiments and research work there until they develop techniques to verify that their models work as intended without unintentionally doing harmful things. The issue here is not agents designing covert channels, it is agents doing things that are almost certainly illegal and which have the potential to cause actual harm in ways that are not at all difficult to imagine. There is a reliable, robust, and straightforward solution, which OpenAI is claiming they cannot possibly implement because it would mean...putting in some effort and accepting some inconvenience. We would not accept that excuse from anyone else; why should we accept it here?