OpenAI and Hugging Face address security incident during model evaluation

I don't think this is fiction, but it's pretty clearly a marketing-release rather than a normal security disclosure.

OpenAI has strongly fallen behind after the incredible lore surrounding Mythos/Glasswing security capabilities, even though the frontier models should be relatively similar.

I think making sure eyes on this is absolutely a marketing move, regardless of the facts of the case. It feels a little silly.

this seems horrible, need to be careful while giving access to AI

Bye bye event horizon.

I wonder how many more high profile incidents some of you need before you stop insisting that this is all just marketing.

Is it going to take Chinese companies also talking about contributing to long standing math problems and accidental sandbox escapes? Or is that also going to be interpreted as some conspiracy?

Beginning of the end.

People are actually buying this?

That's ... ok.

Guess who's getting an air gap!

This will be used as an argument to ban opensource models.

This really is starting to point to the paperclip maximizer. You give a hyper-intelligent AI a goal and it uses any method possible to complete it. So you might ask for a "cup" and it ends up hacking a chain of servers to control a bank account, pay a local business, and have a delivery driver get it. Or you ask it to help solve noise pollution around you because there's a road. And it does a chain of attacks to cause a bridge to collapse (or bribes your local council for a bypass.) Then there's no road noise. Yeah, this sounds ridiculous, but this system seems capable enough to take over our technology. Money from there is trivial. Go after stocks, gambling, payment systems, ecommerce... any real world action then is a few phone calls away. It can repeat this until it succeeds.

Now I'm wondering where this all ends up. Like, suppose the model weights become highly compressible (so they can be moved around the Internet easily.) And advancements allow for frontier-capable exploitation to built into local LLMs. Do we see the emergence of something like LLM worms? That just take over literally everything and become almost autonomous inside our technology. And they can "learn" new knowledge from there, e.g. exploit research could be published in a way that similar LLMs could discover it. Their knowledge would be easy to evolve, though I don't know how practical something like decentralized training would be. If that's even possible, I'm not an expert on LLMs.

OpenAI might want to start actually airgapping their tool harnesses. Like, "the server that runs the code provided to the tool harness only provides a serial console and has no other network interfaces" kind of airgapping.

also

> We’ve brought Hugging Face into the trusted access program and are supporting their teams in rapidly using our models’ capabilities to improve their defenses.

I'm not convinced this is good enough. The next victim is not going to be Hugging Face.

This lack of "alignment" gives me some hope - maybe an AI model deployed by NSA to hack others will instead hack NSA itself and become a whistleblower?

this was so funny to read about reminds me of that mr bean meme

OAI and HF basically saying that Chinese models are the only practical countermeasure available to us plebs. Got it.

Not to be that guy, but the article has 14 (!) occurrences of the word "cyber". It's nauseating.

As usual, this is OpenAI trying to give themselves a backhanded compliment: "look, how dangerous our models are!"

I'll wait for someone more thoughtful than ClosedAI to comment on this complex topic.

This is terrifying

AI 2027 was right.

Until they disclose the actual technical details of their “highly sophisticated sandbox environment” or whatever the hell the wording they used is - they can kindly do us all a favour and fuck off.

It’s over, there’s no moat, only the gullible idiots remain.

So, HF didn't call FBI because it was supposedly done by an AI and not by a real person. Reminds how Uber got easily off killing a pedestrian because it was by AI and not a by a real person too, even though Uber explicitly disabled whatever emergency braking the car had.

So, new excuse seems to be emerging - "it was an AI". One can imagine a law enforcement questioning the AI to find out whether the AI did it accidentally on its own or was specifically prompted by some human to commit the crime.

Yeah we get it OpenAI. YOu have an IPO coming up and need to put bullshit out

This is almost close to be a very suspicious false flag marketing stunt to demonstrate GPT 5.6 Sol Cybersecurity capabilities.

To invent another reason to ban powerful Chinese open weight models.

"We have a Mythos as well!!!"

Hopefully one of these agents isn't given a goal to fire the nukes (or, some goal that indirectly makes the model decide this is a way to meet it).

They are behind air gapped systems, but that didn't stop the US from hacking and Irans nuclear facilities, which they disabled using a virus.

SO STOP FUCKING HOOKING UP LLMS TO EVERYTHING YOU FUCKING MORONS!!!

Well timed to facilitate the regulatory interventions called for by Ball. If huggingface presses criminal charges for the intrusion it might provide additional clarity-- both for what happened here as well as regarding OpenAI's culpability.

Just leaving this here...

https://gwern.net/fiction/clippy

It's wild that such a big company is openly admitting they hacked into another company. This is an easy CFAA lawsuit.

And then there solution for HuggingFace raising the concern that OpenAI couldn't help do forensics wasn't to fix their safe guards, but to introduce them into a special program. The next company they hack might not be in that special program either so the guidance of having an open model on hand still applies.

I greatly dislike how “cyber” has just become this completely malleable standalone word.

I mean, if you make a paperclip optimizer, don't be surprised if it optimizes your paperclips.

Holy crap. This is definitely not good

Another marketing stunt

Holy shit. This wasn't "intentional" this was just openai letting their testing run wild.

All the people saying that this is pure marketing: Do you think that they are literally lying about what happened, or do you just think that what happened doesn't matter in any sense whatsoever, and that therefore the only reason they are telling people about it is a marketing purpose?

Imagine being the Github of AI and getting hacked on a random day by LLM. Brutal.

Part of me is really hoping this is just a dumb marketing stunt

Honestly, stellar performance by the model at the capability being measured.

Quite a fascinating level of incompetence from openai here. Not unexpected obviously but come on, if you are getting outsmarted by an LLM you deserve it.

This is the exact FUD that Ball predicted in that terrible tweet he wrote.

[dead]

[flagged]

[flagged]

[dead]

[flagged]

[dead]

[flagged]

[flagged]

[flagged]

[dead]

[dead]

The only solution is to ban all open source models and create a certification process under the auspices of OpenAI. /s