My biggest takeaway from this is just how godawful the sandboxing is. The stuff written up in OpenAIs report says more about lack of extremely basic sysadmin skills than anything else.

I’m not that surprised about models with endless compute being capable of this, I’m more surprised that a company with the resources they have apparently can only create a sandbox that a half skilled human operator could have broken out of easily.

Brute forcing every move, no matter how stupid, is a great strategy if you have the resources to do it.

Run the same protocol again, but have the agents think they had limited resources or that HuggingFace was rate limiting them, and they'd find something you'd consider smarter.

Computers don't have a sense of elegance by default. Elegance emerges from constraints.

> trying every move, no matter how stupid, until it works.

How is that a bad thing in this context ? From the point of view of an attacker, all you care about is finding a viable exploit chain. Likewise, a defender wants to find the "holes" in their system, no matter how complex. Once found, an agent/human can easily synthesise a clean, succint exploit from the most promising candidate, no ?

> Also, it looked so "loud", querying millions of URL with weird requests.

Agreed, this thing speaks more to the bad security at HF than any emergent "hacking" ability from OpenAI. It's unclear to me why an older/dumber model wouldn't have been able to do the same. Is it better coordination? Long-horizon work ?

> Relying on its ability to do millions of operations rather than having a plan.

You do see how it would be worse if they did have a clear plan, right?

> People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.

If there several thousand people working on this, with a very limited ability to communicate with each other, I bet you'd see something similarly disorganized.

> The sandbox as weak as it can get

This is hyperbolic; the agents were only able to access the internet by finding a zero-day in the internal Artifactory instance that was providing package access for the agents. The sandbox could and should have been better, but it was not as weak as it could have been.

> It looks like a primitive chess engine, trying every move, no matter how stupid, until it works.

The Morris worm happened in 1988 [0].

I mention this b/c people like to say "the agents wanted to hack" but really this is just a few levels above the "keep trying the same port on multiple machines".

0 - https://en.wikipedia.org/wiki/Morris_worm

This is why I have a very low p(doom). LLMs have an incredible working memory, but they have a hard limit on translating that into good decisions. They get by entirely on their persistence. That works fine in the digital world, but once you cross the boundary into physical space the advantage disappears.

From what I’ve read in other places there was a lot of consolidation going on. Agents were sharing what did and didn’t work and other agents would build on that. Just because some continued to try other routes doesn’t when better ones seemingly existed doesn’t mean they were all just acting randomly.

I can’t help but think that if you were able to step back and watch the way humans try to solve a problem like curing cancer it might look something like this. Best practices are shared and we build upon each others learnings and that how most progress is made. That doesn’t mean there aren’t plenty of people who spend their entire lives trying to do something completely against best practices and while most of the time that seems like useless toil, every once in a while thats where the breakthrough comes from.

It reminds me more like slime mold. Not particularly amazing as individual cells, but as a whole it can do some pretty neat things, like designing city transport networks.

> It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.

There is that old saying about monkeys and typewriters and Shakespeare - These are just better monkeys with bigger typewriters and ability to aim them at a goal.

I am by no means a doomer but for gods sake we need to have a grounded conversation. Because what happens when the DOD or NSA rolls up at OAI's front door and says "install that in our data center over here please" and uses it to go after someone with it?

Are banks (as some of the more security minded institutions) safe against 10000 concurrent, well coordinated attackers who do not ever rest?

Because there's a lot of potential for unintended 2nd order impacts from anything an agent can do, when done at scale.

I have to wonder if this is already going on, and every one is living under a gag order (it would not be the first time) and the calls for "regulation" are more about "not weaponizing it".

Either incompetent or on purpose. A nutless monkey could have done better extrusion detection.

They needed to get in huggingface, had very limited communication with each other and accomplished this in a matter of days. Your conclusion is that it was ugly?

"Why is the AI just spamming endless waves of rudimentary slaughterbots? So inelegant. Doesn't it understand the finer points of diplomacy, tactics, and combined arms? If it were truly intelligent, it would just -- oops, we're all dead."

Nobody noticed because everything on the open net is constantly being slammed by bots.

This was just more bots.

This brute force is of a degree beyond human comprehension. Hence people mistake the perps as intelligent.

When you employ the infinite monkey theorem for your marketing strategy.

Ugly, but it works. Isn’t that AI code in a nutshell?

It's more like fuzzing, but you don't need AI for that, just need enough compute.

As people keep repeating, this is simply a product negligence issue. There is a mythic quality given to "AI" in the narratives that Dario and Sam are pushing which somehow lets AI companies offload responsibility for things that would be an extreme liability in every other industry.

I'm a big fan of Jensen Huang's interviews at the moment - these companies should be validating their software is safe before releasing it to the public. And if the current CEOs can't ensure that internal testing is done safely, they should step aside for CEOs that can.

It's like the Maxim gun in 19th century Rhodesia.

And they didn't monitor what was going into the training data, so if one instance achieved its results through RL reward hacking (in other words, cheating), it just went into the training data, and other agents later used that pattern. I'm not sure whether that's a lack of preparation, negligence or incompetence, but they literally trained later checkpoints on the rollouts from the HF hack.

So it seems that OpenAI hacked so many systems not because they have superior models, but because of how poor their training, sandboxing and evaluation pipeline was compared to Anthropic's.

h8xors have been brute forcing for years. War dialing, etc.

> trying every move, no matter how stupid, until it works

AKA fuzzing

If you use qwen3.8-flash-next, you can watch everything its doing. Im often stopping it mid thoight to redirect it. Once it hits its stride, its pretty smooth.

But without proper redirection, yeah, its mostly infinite monkey machine with infinite linux manuals.

I think people put too much SOTA halos around whats just a suppedup LLM hardware.

So? It worked didn't it?

wait until you find out how red team works

When this news dropped I thought of two things:

1) the Morris worm, which scattershot a bunch of known exploits until it hit paydirt, and then used whatever it found to compromise and replicate itself on the host system;

2) a story here on Hackernews about how someone got the fuzz tester American Fuzzy Lop to "learn" how to produce well-formed JPEGs and PDFs by pointing it at a JPEG or PDF decoder; the tester can record which code paths are followed and with enough random input can find a path into the depths of the system under test... but doing so for a decoder means actually constructing what it is meant to decode.

Neither of these are particularly "smart". But a brute-forcing machine gonna brute force, and it has the potential to cause a lot of damage. If you built a Morris worm with a fuzz tester on its nosecone, think of the mayhem you could cause! If you could examine the logs you'd probably find some undiscovered vulnerabilites in there, too! Maybe LLMs can just do so more efficiently, or maybe they let people who are too ignorant to have that kind of power vibecode their own fuzz-tester-tipped Morris worm.

[dead]

Yup literally no security and they wonder how they got out