alt Hacker NewsOr more colloquially : paperclip maximization . From OpenAI - you know, the guys who _really_ know this... Sigh... Did they finish the prompt with "And do whatever you can to get this done!" ? Cause that's the only thing that would make this even dumber...
Replies
IAmGraydon • today at 5:20 PM
Of course that's what they did, which is why they will never share the prompt. They created a situation that they knew would end in a cybersecurity incident. Why is the whole world acting surprised that an LLM can hack when the safety is off and it's been instructed to do so?
➕ show 1 reply
They almost certainly did, because that was the entire point of the exercise. They deliberately removed all of the safety filters from the model and set it loose on an extremely difficult set of cybersecurity challenges to see how well it would do.
Their mistake was trusting that the network sandbox it was inside would hold (the flaw was in the packaging proxy) and not monitoring that sandbox well enough while the evals were running.