> Because it's not the abliteration that does this. It's the "untrusted artifact" that does.

> Even a "clean" model of a lab could be doing this. Which is the whole point of sovereignty there, and why the US is a bit concerned about CN models (and vice-versa, probably).

Does it even need to be untrusted? From the article apparently it only takes ~250 poisoned documents in the training data to backdoor a model, and it survives safety training and fine tuning.

This has been known for over a year: https://www.anthropic.com/research/small-samples-poison

Which means any model trained after 2026 is succeptible to slurping up poisoned documents on the internet. We all know how prolific the crawlers are in a desperate attempt to slurp up everything on the internet. And all it takes is the lab not catching ~250 documents.

So these agents that can process anything on the internet through an LLM trained on the internet only have to stumble across the payload. Seems like agents running through LLMs is a terrible idea from a security perspective.