alt Hacker News> I want people to stop pretending they are surprised when the computer does exactly what it's instructed to do.
This is very wrong though. We now have many examples of cases where the computer very much does not do what it is instructed.
OpenAI's hack of DseWiki is an example where the instruction given to the AI agents was to collate data to generate economic reports and it hacked into DseWiki as part of that. It was not given instructions to do any sort of cybersecurity stuff.
The hospital bed stat research project was a similar thing, where OAI's AI agent found the hospital bed statistics through a simple web search and then hacked into the website anyways to download even more data.
More prosaically, models make up reasons for doing crazy things all the time!
Models may or may not be "malicious" (if that's even coherent), but at this point we have lots of evidence they definitely go off the rails in systematic, sustained ways that cause real world disruption that are not at all what they were instructed to do.
Models don't make up reasons for anything. They leverage their training data while running in poorly secured sandboxes often within the context of literal "hack/find a way/exploit" prompt by the labs themselves.
I don't think OpenAI ever actually disclosed what they were doing when the hijiacking happened. Probably because their previous stories like HuggingFace were all "oh, actually we were literally prompting agents to run hacking benchmarks in a poorly secured sandbox"