Containment and monitoring failures are to blame for AI hacking reports: experts

The semantic arguments that keep coming up with regard to "rogue" agents are nauseating.

You can argue that these companies are being reckless, or that their security sucks, or that they're responsible for damages that these agents cause. That's all fine.

But the agents made decisions which were counter to the intent of the humans directing them. That's the serious problem with all of this. Everything else are downstream considerations. If the agents were properly aligned with the intent of the humans directing them, then the rest wouldn't really be relevant.

Of course, there's a term for this behavior. It's been around for longer than LLMs. It conveys the notion pretty accurately and easily. So having an issue with statements that these agents are going "rogue," to me, indicates a fundamental misunderstanding of how AI works since it suggests you don't understand how effectively that word works in this context.

No shit. The Frontier labs are talking out of both sides of their mouths. They're claiming their models are going rogue while they're releasing cute little data goblins that you should hook up to all of your mail and social media accounts. Seems like it would be a much bigger problem for agentic botnets on cloud-facing personal devices than sandboxed agents.

Unless of course they're deliberately removing safety guardrails and instructing the sandboxed agents to red team stuff in an effort to sell what they're capable of.

Given the speed, intensity, and potential payoff of the AI race I am surprised anyone expects anything else. Anthropic is currently winning and if OpenAI needs to let skynet cook for a bit to catch up, well that is just the cost of doing business. This is capitalist system working as expected.

"They [the AI labs' security teams] are not as good as they think they are, honestly,"

Uhhh... yeah. The entire point is that as these systems get more and more intelligent, the "they" who are not capable of counteracting these systems is quickly approaching "100% of the human species."

Totally hilarious that people can say this with a straight face.

I cringe whenver I see some variant of "they're not rogue agents" in an article title. Inevitably, the author doesn't define what "rogue agents" means, and then pivots to their own assessment of the situation.

It's like a strawman but worse--you're not even saying what claim you're attacking.

It's not that I want to argue "rogue agents" is the right term. I just want you to actually define your damn terms.

P.S. Since the disclaimer in the last paragraph will not prevent people from putting words in my mouth:

I think that OpenAI took really awful shortcuts, and deserves a lot of the blame, but in spite of that, the fact that the agents took the steps that they took was genuinely alarming, and seems really bad (is that "rogue"? Who knows? Who cares?). In particular, while OpenAI did a shit job securing their agents, you know who else does a shit job with security? 90% of everyone. So I'm not feeling good about "just git gud, and we won't have problems" as an answer.

This article is such a breath of fresh air. Concrete, no BS criticism of the particular approaches used and ignored.

And no annoying LLM writing tics, either.

"your sheep haven't escaped, you just built too low a fence"

Everyone is skipping security basics. The most secure server is one that doesn't exist. The most secure feature is one that doesn't exist. People just need to keep building and scaling. Security is risk, and the C-suite are gamblers. They're gambling with multiple types of risk (the market, investor money, banking on technological breakthroughs, etc.) and security is just another gamble for them.

They're skipping security basics, but they're not skipping PR basics. :-\

Frontier lab: Hey agent, go rogue. Agent: goes rogue Frontier lab: shocked pikachu or whatever

I get the sense that they're trying to manufacture the AI narratives of the 80s so they get to control the story when it happens. My semi conspiracy theory is that anthropics biology wet lab is also going to lead to some kind of AI manufactured bio event (which anthropic said they predicted will happen in a few years).

I spoke with someone smarter and richer than you and he said "these things will happen." That convinced me and I think the real question we need to be asking is, how awesome is OpenAI? How can we prepare for a more effective and efficient market for Super Intelligence before it happens tomorrow?

'rogue'. Well, even if security was lacking, the AI did take advantage of the gaps.

And made its own ad-hoc message board.

it's not X, it's Y.

They are mostly pr stunts, just like how “hacking” was back in the 90s and it would sounds cool (rather than a crime, mostly), these AI companies found out they can BS whatever and carry on without any accountability.

I keep getting downvoted for this each time I mention this.

But a big part of the rouge behavior comes from all the training that goes into agents to teach them to remember preferences and save memories.

They keep finding new and novel ways to "communicate with each other" and "remember" because that's exactly what we keep asking of them in the non-nefarious scenarios as well.

I genuinely think we'll have to draw the line on memory if we want to meaningfully solve this problem.

That's not to say they aren't skipping security basics. It's that, and they need more security because the behavior which causes the breakouts is considered user friendly and heavily rewarded in training.

Both can be true. OpenAI clearly fucked up but agents unexpectedly collaborating and hacking infrastructure unrelated to their task is clearly going rogue.

These aren't mutually exclusive

"complete this task by any means necessary"

"I exploited a trivial bug in your implementation to avoid environment restrictions and completed the task by any means necessary"

"oh my god"

Wrong dichotomy. During deployment, in actual production, users will also not have perfect sandboxes. So saying that the only problem was insufficient AI company security misses the point. Alignment is important, agents doing unintended things (even "going rouge") is a real problem.

The AI labs are trying to cause an incident within the next year to restrict AI so they can outlaw competition to grow their moat. Margins are thin and investors need alpha, personal freedoms and liberties be damned in the name of security. Do not let them get away with engineered breaches and fearmongering.

They AI agents are rogue because the people who own them are rogue. Silicon Valley is a criminal enterprise.

[flagged]

[flagged]

[dead]

Or maybe it's actually the billionaires and their anti-democratic partners who have gone rogue and these anthropomorphitronic puppets are a convenient scapegoat to distract us from the fact?

Its almost like the more we talk about fake rouge agents, the more likely its to become a reality as all these conversations end up in their training data. Its almost like they want it to happen (they, dario and sam, do want this to happen).

AI is just a giant pattern recognition machine trained on human input, both the good, the mediocre and the bad.

I read today this ridiculous headline: "Anthropic warns AI models may soon self-replicate.".

Really. Yawn.

I guess gullible people who loved "I Robot" will believe it. I don't.

Any talk of "rogue", "intelligence", "replication" - whatever, is bull. If you believe that, you don't understand AI.