Timeline of the OpenAI accidental attack against Hugging Face

Had a high opinion on Simon Willison, this broke it.

It’s absolutely wild that agents used a write access oversight in their package manager to communicate amongst themselves. It essentially created an agent ad-hoc chat interface using their own package manager file system.

I think we are in need of Europe's leadership in safety legislation. It is foolish to say 'China will get ahead' when they will harm themselves too. Being unsafe is not something to gloat about.

Stiff fines for such incidents to pressure companies to get their acts together is a good start.

Is it normal for these training/eval runs to go on for over a month?

guys, we should meme the > "ai model leaks from openai and attacks huggingface" to be somehow framed as > "and therefore openai cannot be trusted with ai safety, and we need open weights models". anybody have an idea how to make this easily digestable?

Oh please, these "attacks" are marketing exercises: Look how intelligent and devious our models are, theyre so powerful, fear them!

[flagged]

[flagged]

Ha ha ha ha. Cooperating agents turn out to be smarter than the individual agents, who would've thunk it. It's not like cooperating humans are smarter than individual humans. /s

Not sure this is any different than state-level (-sponsored, cough cough) or the larger collective hacking groups that work in this exact way (internal message boards, exploit-sharing, etc. etc.), with similar outcomes which we hear about in the news frequently.

Heck, this is pretty much how human organizations are organized, just with different goals than hacking.

A layered approach to cybersecurity is the fix to humans exploiting systems, and is likely the best victim-side fix to ai exploiting systems. From this incident itself, where huggingface used a chinese open-weights model to respond quickly, it is very clear that ai will be needed to find, mitigate and resolve cyber issues.

Additionally, on the ai-labs side, perhaps what is needed is initial model training on following the law and the rules of society, just like we do with kids. And hey, it takes much longer to train kids than models, which latter is to our advantage as a society on containing these kind of issues.

Any other approach with "neural-network" based entities (artificial or biological) is likely to fail.

Training/Education, Enforcement/Justice-System, Rehabilitation: the 3 pillars of an advanced, rules-based society.

You know this was "a work" in pro wrestling parlance, right?