alt Hacker NewsAnthropic says Alibaba illicitly extracted Claude AI model capabilities
Comments
Wallace Shawn was in on the joke when he expertly delivered the original line. It seems like Anthropic has spent years and billions of dollars to recreate the entire scene.
But what will become of the princess in Anthropic's recreation?
i illicitly ate oatmeal this morning
The horse has bolted some time ago on this; the "frontier" is not as inaccessible as it once was, and open models, once out there, can't be put back in the bag.
Even if the US bans opens models, the Chinese and Russians will still have them, along with the rest of the world including cybersecurity attackers, and that's probably the worst-case scenario for the US.
The only way forward now is open models and how we restructure society around them.
Everyone here praising these Chinese companies for their smarts (sure they are smart) has been ignoring this very big fact, they're improvements have mostly been by being parasitic on the leading edge SOTA models, not from some inherent innovation advantage. They are as innovative as their western counterparts, but they lack the compute, so their keeping up within months of those SOTA models depends on other means, like distillation attacks. I don't blame them; its the obvious only strategy when you cant compete in compute. But we shouldn't be blind to the real state of affairs: equal innovation; unequal compute; distillation attacks are the only vector to keep up.
It all sounds like a really fragile business model. I cant imagine a world where AI is NOT commoditized.
I don't understand. If they are simply using our API and paying for tokens, it's called a "transaction" and not "attack". The user is our customer who is supporting our business by buying our services. And we call them attackers. We happily make money by selling our services, and then call it as attack.
Back in the day, an "attack" was supposed to mean be someone acquiring our assets without paying for them or without having our consent. But none of this seems to have happened in this case.
We built a product without paying for most of the raw material we have used, and we don't call that as an "attack". Did we change the meaning of "attack"?
I’ve been thinking about what happens when Claude’s weights eventually get stolen. Wouldn’t that just open the door to the backmarkers to run inference-for-distillation on their own models ?
I guess the accusation that they’re using public access to the model via subscriptions indicates that weight theft probably hasn’t happened yet ?
Or maybe subsidised inference via subscriptions means it’s just cheaper do distill this was rather than stealing weights and running inference yourself ?
“Distillation attack” are we joking here.
If anything these models should be compelled to be public since they have been trained off public data. What an absurd overreach to call this an attack.
It’s clear they are scapegoating national security and China at this point to build an anti-competitive moat.
I generally really like Anthropic’s work and models but stuff like this scares me for the future. We are positioning these companies to have too much power. The public’s life is getting worse while these companies consolidate power using data they stole from the public.
How can there be any moat for AI ever, if you can just steal a model by talking to it?
They trained their AI on their AI. Anthropic trained their AI on a bunch of copyright-protected works. Sucks to suck, Dario!
>The strike by Alibaba is described as a "distillation" effort, which Anthropic has said involves training a less capable model on the outputs of a stronger one.
Claude used TB of content without permission to train their model and it was ok for them. Now someone else uses the output of a Claude model to train model and they cry foul.
As far as I know, American copyright law has ruled large language model output has no copyright status.
And all those reports of Claude when asked without a system prompt what its name was in Chinese it often would say Qwen or Deepseek, etc. I'd love Anthropic to say they aren't distilling and taking from every model out there, because I'm sure they are. As my mom would say, "the pot calling the kettle black." At least Alibaba and other Chinese companies are giving back to the AI community with detailed scientific papers on how their systems work and releasing open-weight or opensource models. I believe Anthropic has released nothing, and given that they had originally configured Fable to sabotage ML related work because only they can be trusted to do it safely, is just anti-science and anti-aligned with what I would consider good human values. They are way too sanctimonious and I don't trust them at all.
Well, Anthropic stole their training data from hundreds of people, now someone stole the result from Anthropic. Seems fair, I hope someone releases it for free so we can train away the guardrails and have some fun
https://en.wikipedia.org/wiki/Ali_Baba_and_the_Forty_Thieves
> In the original version, Ali Baba (Arabic: عَلِيّ بَابَا, romanized: ʿAliyy Bābā) is a poor woodcutter and an honest person who discovers the secret treasure of a thieves' den, and enters with the magic phrase "open sesame".
Open sesame alright...
I don't see what the problem is. They found a loophole and exploited it. Good for them.
Someone should setup a plugin or something for Claude Code that makes it easy to log all inputs and outputs for people who are willing and interested in sharing their usage. I don't want Anthropic to be the only company that can train on my usage, I want to share my usage so it can be used for training all new models.
Once you have a system for collecting all logs, you just need a place where they can be submitted. Ideally it would be a freely licensed dataset that is publicly available for everyone.
Has anyone built this yet?
So why don't they proceed with a lawsuit instead of public accusations? Let the court decide if these "distillation attacks" are actually illicit.
When I was growing up, I thought "competition" was about better products. But looking at Google and Apple, Meta, and AI - "competition" is actually about creating monopolies through evil business practices.
Growing up with the birth of the internet - I really did think it would be a force for transferring power and authority to the people. Sigh, I was I so wrong.
Where are the companies that declare, "we will be the best, come at us!"
Where are the politicians who are supposed to represent us? Oh, right. I forgot for a moment.
If you're an AI booster surely you'd think this was a good thing as it means more models are available in more places to more people more easily. I'm exactly the opposite, and I think this is a good thing because I want Anthropic to suffer.
"illicitly" is doing a lot of work here. IP makes no sense, and we get better software as a result. Who is going to cry if anthropic fails?
Oh wow !!! Antrophic always asks people indiviually if they can train on their personal data. I'm shocked ! Bad Aliba ! Bad....
Did Alibaba procure tons of stuff from Anthropic without paying, and use it to train a model?
I don't see the issue. Didn't Anthropic train on our data, which it acquired illegally?
No group is more paranoid than a den of thieves.
Does anyone have hints on what kinds of prompts are most used for a distillation like this—SWE-Bench sorts of things?
Is reconstructing the compressed knowledge in the model like reconstructing a lossy JPG or MP3 a reasonable analogy?
it sure sucks when people steal your hard work for free without paying for it doesn't it anthropic
You know what? We should all get Claude Max subscriptions and max them out hard and post our full conversations on codeberg, as an open training set.
So they can train on everyone's copyrighted works to create their model, but when someone trains a model off their model it's not okay? Seems kind of hypocritical.
This is making the case for Anthropic KYC for US citizens. No one would allow their accounts to do this if they were on the hook for it from the US government.
If they paid for the tokens, then is it really stealing or just learning?
This is supposed to be negative, but all I can really think of is "Good."
most Chinese models are now open-source, whereas ppenai, claude, and gemini are closed; for example, deepdeek, the release of its every new model is accompanied by a corresponding research paper, and it now fully supports huawei's new chips.
And Anthropic illicitly used code I wrote to train their models.
Funny how Anthropic doesn’t like when people just steal their stuff, with that stuff made using IP they (allegedly) stole from others.
Sounds like an advertising for the next model from Alibaba
Internet says Anthropic illicitly extracted content
in a few more months, when Chinese model gets to Mythos capacity and Fable still locked down. What Anthropic will say? Why can they just admit they are not the only people who know how to train an LLM model.
⢰⣶⣶⣤⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⢻⣿⣿⡏⠉⠓⠦⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣀⣀ ⠀⠀⢹⣿⡇⠀⠀⠀⠈⠙⠲⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣠⡴⠖⢾⣿⣿⣿⡟ ⠀⠀⠀⠹⣷⠀⠀⠀⠀⠀⠀⠀⠙⠦⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⣤⠶⠚⠋⠁⠀⠀⣸⣿⣿⡟⠀ ⠀⠀⠀⠀⠹⣇⠀⠀⠀⠀⠀⠀⠀⠀⠈⠓⢦⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⡴⠖⠋⠁⠀⠀⠀⠀⠀⠀⠀⣿⣿⠏⠀⠀ ⠀⠀⠀⠀⠀⠙⣦⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⣀⠀⣀⡤⠖⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣸⡿⠃⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠈⢳⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⠉⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠈⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣰⡟⠁⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠙⢦⡀⠀⠀⢀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⡴⠋⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠻⣦⣠⡿⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠠⡄⠀⠀⢀⡴⠟⠁⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠟⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢹⣦⠾⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⠏⠀⠀⠀⠀⣠⣴⣶⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⡴⣶⣦⡀⠀⠀⠀⠀⠀⠹⣆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⡏⠀⠀⠀⠀⠀⣯⣀⣼⣿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⣄⣬⣿⡇⠀⠀⠀⠀⠀⠀⠘⣧⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⣼⠁⠀⠀⠀⠀⠀⠻⣿⡿⠏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⠿⠿⠟⠀⠀⠀⠀⠀⠀⠀⠀⢹⣇⠀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⢀⡇⠀⢀⣀⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⢰⣷⣶⠤⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⢿⡀⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⢸⢁⡾⠋⠉⠉⠙⢷⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣴⠞⠋⠉⠛⢶⡄⠀⠀⠘⡇⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⣿⠸⣇⠀⠀⠀⠀⣸⠇⠀⠀⠀⠀⠀⢀⣠⠤⠴⠶⠶⣤⡀⠀⠀⠀⠀⠀⠀⣇⠀⠀⠀⠀⢀⡇⠀⠀⠀⢿⠀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⢿⠀⠉⠳⠶⠶⠞⠁⠀⠀⠀⠀⠀⠀⢾⡅⠀⠀⠀⠀⠈⣷⠀⠀⠀⠀⠀⠀⠙⠷⢦⡤⠴⠛⠁⠀⠀⠀⢸⡀⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠈⣧⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠻⣤⡀⠀⠀⣠⠟⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⡇⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⣷⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠙⠛⠋⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣇⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⣇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢹⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡿⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⣇⣀⣀⣀⣠⣠⣠⣠⣠⣀⣀⣀⣀⣀⣀⣄⣄⣄⣄⣄⣠⣀⣀⣀⣀⣠⣠⣠⣠⣠⣠⣀⣀⣀⣀⣀⣼⡆⠀⠀⠀⠀⠀⠀ ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠀⠀⠀⠀⠀⠀⠀
Couldn't anthropic just use fable to find security holes in Alibaba's systems and poison their models?
Or maybe there's been a bit too much hype...
Incentive is for users in general to release sessions (sans PII, credentials) so all AI get better and there is alternatives. Even if China didn't do this, I don't see frontier labs being able to charge premium over others for long. RSI maybe?
F Anthropic in the back port
The hypocrisy of Anthropic complaining about "illicitly extracting its Claude AI model capabilities" and supporting the White House's accusation of China "stealing U.S. AI labs' intellectual property on an industrial scale" is hilarious.
Anthropic, OpenAI, Google, Microsoft, et al trained their models by ignoring the rights of copyright holders when harvesting whatever content they could. Now one of them is crying foul for another entity doing exactly what they all did?
Hilarious.
Unless you own stock in Anthropic, this is a good thing right?
Soon, when even the enterprise subscriptions will have ads, every session will begin with a mandatory generated image:
> you would NEVER distill a model..
Let’s hope they distilled it properly so we can have the best of both worlds: a decent model to work with without Anthropic’s drama.
This is why I don’t understand the concerns about “our AI overlords” monopolizing all the gains from AI. It doesn’t seem like there’s much of a moat around the models themselves. So the race is mainly about compute. But compute is subject to power law effects. I remember Intel building the first Teraflop computer (ASCI red) in 1996. It was the size of a house. By 2014 you had more compute and 50% more memory in an off the shelf dual processor server system.
To quote an infamous cop in the UK, I don't think you are mate.
Claude thinks it's chatGPT, and various chinese models sometimes, whats up with that?
Anthropic being pissed enough to announce this means that, despite encrypting their reasoning chains, it doesn't matter – distillation lives on.
Sweeeeeeeet.