alt Hacker NewsOur position on open-weights models
Comments
I welcome any and all competition. But they first need to be able to run the obstacle course I’m setting up with Uncle Sam.
Clearly anthropic is misaligned, how can they claim to be able to align AI when they themselves are misaligned from humanity.
When they use ill gotten data for training their models the world's smallest violin plays when they complain about distillation attacks.
> In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression.
Welcome to bizarro world!
Fist off: "the most dangerous model may be one that is trained in secret" <-- Says the guy that not only restricts commercial use for some of their models but develops them in utter secrecy. With the pretext of guardrails. Then show us the guardrails you really use by opening the weights.
Second: "use in drones [...] for surveillance and repression" <-- writes the King of FUD, as the US is an an active campaign with the help of their models. And/or OpenAI's.
I am very appreciative of the freedoms of the west but this type of hypocrisy and lack of self-awareness is bonkers and it should be called out.
Banning exports to China is merely a narrative device; the real intention is to sideline intelligent technologies.
https://x.com/biggest/status/2081891756769952075?s=46 Official stance of an anthropic-employee being against Dario amodei opinion.
I think this letters tells us everything we need to know about the man. I've rarely seen so much flattery towards the current administration, contradiction, deflection, half-truths and hypocrisy on a single page. This man is scared of everyone: the current admin, the public, and the other companies promoting open-weights.
> Open-weights models that don’t have dangerous capabilities are a public good
Knives should only cut during the day, knives which cut at night are bad.
I found lots of comments here suffer from proximity bias.
Anthropic anti-open-model stance does not mean China is not a threat.
> Open-weights models that don’t have dangerous capabilities are a public good
Note the hedging against 'dangerous capabilities'. Undoubtedly, all the useful ones trigger this condition in Anthropic's eyes. The rest of the post is filled with similar weasel-wording. Make no mistake, this absolutely confirms that Anthropic is against open models in the sense that any reasonable person understands them.
The way the rest of the post unabashedly appeals to the current US administration's China hysteria is hilarious, and not at all subtle.
I guess we'll see about all the doomsaying here, won't we? Kimi K3 is frontier-level, and there's no stopping it now. As far as the world is concerned, anyway. If the US wants to kneecap itself that's another matter.
At least try to release one open weight before you say anything, or all of that only sound hypocrite at best.
> My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
There's not much stopping US from becoming an authoritarian regime, and the US is already using AI against its own citizens.
he's asking for the most powerful models to be in the hands of the few while the majority will be at their mercy.
This letter is so full of contradictions.
> Open-weights models that don’t have dangerous capabilities are a public good
There are no models that don't have dangerous capabilities. There is not a single human in the world that doesn't have a dangerous capability.
> authoritarian government [] build AI models that are more powerful than those built by the US
Yeah, they will do that no matter whether you support the existence of open weight models or don't. In fact, they have probably already done that. I would say "deal with it" but there is nothing you can do actually.
> authoritarian regimes have stolen and used AI
Monkey looking away meme.jpg
> powerful AI models may be misused
Small AI models may be misused just as well. I would say it's the small models that are the problem: cheaply deployable, easily fine-tunable, fast.
> once weights are released they cannot be withdrawn
Exactly. It has been already done. So?
> We should not sell powerful chips or chipmaking equipment to China
I agree. We should have competition and I'm rooting for China to design their own chips. They wouldn't if we let them buy our's.
> We should crack down on industrial-scale distillation operations
Well, at this time it's already irrelevant. We have GLM-2, DeepSeek, K3, which are already SOTA and can be used for distillation.
> All sufficiently capable models [] should go through mandatory safety testing
Problem is, it's unenforceable legally. Models are just math and all previous attempts to ban maths have failed and simply pushed US back. I will not be asking US government permission to calculate matrixes of my choice.
TLDR: "we're not in favour of banning our competitor's products, but we do advocate banning hardware exports to foreign countries where competitors are located".
There's also a very strong implicit double-standard in the discourse, along the lines of "it's dangerous if non-US uses this in military fields, but it's fine if the US does so".
A sober look at actual rogue nations and their use of AI shouldn’t have us fretting over that particular hemisphere
Generational good will loss from these guys.
The Roman empire fell for the same reason, didn't it!? They wanted to control it all ... But it was too big
As an act of good faith and proof, they could stop spending the record-breaking lobbying money for couple years.
This USian death spiral season sure is picking up speed
Do distilled models have to inherit guardrails? Can a model distilled from a “safe” model be made unsafe?
"Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn."
Translation: "we won't be able to monetise it"
"We should not sell powerful chips or chipmaking equipment to China"
Translation: "we can buy them cheaper when there's less purchasing competition".
"Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable."
Translation: "we've been outsmarted and lost our advantage because their way is faster and cheaper"
I think it's a bit rich to complain about distillation, when they been plundering the internet for years for copyrighted works to train their models on without permission.
For smart Chinese hired by anthropic, are you still proud working there?
The Anthropic CEO has an entitlement problem. I read his opinion twice and somehow in his worldview, if any other country gets better at AI than the US, the apocalypse will come and the US will be no more. Or something along those lines.
Secondly, he's like the Trump of tech: China, China, China.
Thirdly, he needs to cut it out with the guardrails and other safety BS he's peddling. If we're going to get Skynet, we will get it no matter what we do.
> For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)5. Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance.
One of two outcomes are true in this scenario: 1) this is unrealistic fear-mongering or 2) we're all fucked.
I just don't see how our solution to this can be export controls or bans. If the fear of a bioweapon engineered by an open weight AI is a legitimate threat, our only option is to develop effective countermeasures (and perhaps the same AI will assist with protecting). Open weights are not going away and pretending like some sort of "ban" will prevent bad actors from accessing them is a fairy tale.
Perhaps, the general public needs to formulate its position on Anthropic models...
It's funny how as an outsider who is neither from the US or China, both look the same.
Edit: To add some more context. What I mean is neither look like the good guys or the bad guys, but one of them is spending an awful amount of energy trying to paint the other as the bad guy and themselves as a good guy, which I hope a lot of people aren't buying anymore. Because at the end of the day, I think the honest truth is that everybody is just trying to serve their own interests.
Suggested title: Dario Amodei's crusade on reality.
If we really cared about AI we would work together
I'm no longer supporting this vendor, for personal or business
Americans don’t get to lecture the world on authoritarianism anymore
Jesus Dario we get it man, you want clout for the IPO.
This constant whining from anthropic about distillation attacks continues to be rich given the amount of stolen data that went into any Claude variant.
Hmmm, how could this post be possibly a good thing for Anthropic?
> "We should crack down on industrial-scale distillation operations" I'd prefer if there was a crack down on industrial-scale scraping. Maybe even reimbursement for the problems it has caused (some nasty AWS bills, tons of man-hours spent on preventing new nasty AWS bills, etc).
> We should crack down on industrial-scale distillation operations.
IP for me, not for thee.
yourself distilling humanity too bitch
With respect to whomever this dude thinks he is, it's egregiously "main character" to frame a policy recommendation for the US in terms of what's important to his personal beliefs.
China will likely gain the capability to produce leading edge chips within a decade domestically. From then, occupying Taiwan will be about controlling the wests the access to that. Same with AI models, though I would say we are already about there. Dario is either lying to us or to himself that distillation plays a major role.
I wonder if publishing these documents is not just a public stunt, but heavily integrated with Anthropic's business storategy to maximize operational efficiency. Companies often have several internal documents for a single policy like "position on open-weight models", one for public (like this), others for the legal team, the lobbyist, the developers, the investors, etc. The differences and nuances of those manuals can be very huge and are necessary to maximize the goal from each branch, but often a cause of headaches like bureaucracy, communication friction, outdated information, etc.
A single canonical official document can make it very simple. Even though each department cannot achieve the maximum gain from nuanced documents, keeping operational context as simple as possible may really improve LLM driven operations to move faster and cut cost.
If "publishing pleasant positions and actually following them in general" becomes a good business storategy in LLM driven society, it can be one of very few good outcomes from this dystopian AI craze.
“By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”
> Anthropic has never advocated for a ban on open-weights models.
What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical.
> China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips.
This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days.
> All sufficiently capable models, open and closed, should go through mandatory safety testing
Applying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive.
It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."
> industrial scale distillation attacks
You mean the industrial scale distillation attack you perform on the entire corpus of human knowledge… idk, call me cynical but you reap what you sow.
Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.
"Open-weights models that don’t have dangerous capabilities are a public good"
Read between the lines folks. Anthropic deems every model that has frontier capabilities as "dangerous", and thus they are against them. We all know that "dangerous" simply means "whatever model hurts our bottom line."
More dishonest framing from the company that constantly lies to everyone. No surprises here.
> 1. My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people.
What a nice implicit way to say that it would be okay if the authoritarian government is the current US government trying to perpetrate incredibly deep repression of their own people.
Does Flock use open weights or Frontier models?
The consent factory has never been more productive.
So many US-haters that live in the US with no intention to leave...That right there is why this is the greatest country on Earth.
Anthropic starting to sound like Microsoft from the 1990s.
I like the idea that Mr. Amodei holds these beliefs because he wants to safeguard democracy as much as the next guy.
I just don’t find it believable.