alt Hacker NewsYou can put lipstick on a pig, it'll still be a pig
"Anthropic has never advocated for a ban on open-weights models."
---
"We should crack down on industrial-scale distillation operations"
"All sufficiently capable models, open and closed, should go through mandatory safety testing"
These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What is the testing criterion? How do you pass it? Is it a government body that approves a pass fail or a global body? If it is government, and boy does it seem to be, how do you disambiguate MASSIVE corporate lobbying to set up the safety testing in such a way that the boys in blue are let through and all others are barred out of safety concerns?
My concerns aside, much of the soft-points being made are non-historic
"But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."
It doesn't mater what his opinion is. The fact is that an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. We aren't in a vacuum, we have real world examples now and these statements are counter-factual.
Replies
What do they even really mean by "safety"? I mean, I can have an Anthropic model do something incredibly unsafe if, for example, I put it in charge of a hydroelectric dam and don't explain properly how the controls work. On some level, everything is simultaneously "safe" and "unsafe". I've never found Amodei's reasoning here to be particularly well thought-through. I think he, like a lot of folks in the area, are starting to realize that they may never be able to build a moat around their businesses.
It's political doublespeak. They want to have their cake and eat it too.
> an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China.
HuggingFace did not seek access to Claude Mythos or OpenAI's equivalent program. They probably could have had access to these models for defensive purposes if they'd done it properly.
> these statements are counter-factual.
The OpenAI incident is a single example. You're massively overgeneralizing. You can't refute an entire class of possible outcomes based on a single event where it went the other way.
I tend to agree that model capabilities will favor defense over attack, but I think there will be a lot of disruption before that equilibrium is reached. If cybercriminals or state-sponsored actors are able to scale up attacks quickly, many orgs with less sophisticated defenses will be caught by surprise.
Edit: just to clarify my position, I don't love Anthropic so much. I think they're marginally better, but I'd still like to see regulation strangle everyone so we get another 20 years to figure this shit out.
I don't really understand how they can argue the security angle with a straight face. It's not like GLM 5.2 is a slouch. I've seen it do things like exploit an IDOR issue when I was experimenting with a quick-and-dirty web automation task. I simply fixed it, as one does. Open models make the world better to a far greater degree than they set it aflame.
Their position is analogous to trying to, say, ensure digital privacy for everyone not by making encryption freely available (because that would let the bad guys use it!), but by making it so you can't use general purpose communications devices that can listen to transmissions not intended for you. Do they hear how moronic that sounds?
Each passing frontier-level open model release makes Anthropic's patronizing rhetoric a little more insufferable, because it becomes clearer how unmoored from reality they've become in pursuit of profit.
I have no idea what to do about the government interference. There's probably not a lot anyone can do.
However, your last point is quite a strong one. Corpos aren't just going to stand there with their collective pants down, and there's not a lot anyone can do to stop them from protecting themselves. There are ways they can get what they want without getting caught.
Remember when the US tried to ban strong cryptography in the 1990s, and how well that went? They may have more leverage with AI because it's a bit harder to hide large scale computing usage, but I don't think it's impossible at all.