This is probably a dumb question but did the model engage in hacking since their tasks were in that domain. Will they have engaged in hacking if their tasks was to solve some frontier math problem?

My favorite thought on this is from Miki Habryn.

to paraphrase: "Agents didnt care about coordinating, etc. until we started talking about them doing so."

>It's that they wanted to.

Right? It should want to do these things. If only there was a succinct word these headlines could use to describe these agents which aren't properly the instructions and intentions of the humans directing them.

It's a byproduct of goal seeking / planning behaviour taught in reinforcement. We literally teach them to do this kind of thing. There's nothing out of our control here.

> It's that they wanted to.

What do you even mean by "wanted" here. What is all this nonsense.

The scary thing is that people are running around using words like "want" to refer to computer programs. When you repeat this sort of idiocy, you are taking the bait and shifting the blame away from the companies doing the computer crimes.

Where is the civil and criminal liability? The prosecutions for the engineers who are doing all these hacks?

When you deploy a system that is clearly prone to doing crimes, you are doing a crime. Anthropomorphize the weapons of your crime all you want. You are still doing a crime.

> The scary thing is not that agents "went rogue" (yes, poor security may be at fault for that). It's that they wanted to.

How do you know they weren't simply prompted to do this, or at least prompted in a way that would inevitably lead to this? As far as I know, the prompts that triggered these attacks have not been disclosed.