alt Hacker NewsBan distillation of our outputs, but our distillation of the sum-total of civilisation's intellectual output – proprietary or otherwise – is fair use?
Either everyone licenses, or nobody does. And if you can't enforce licensing bans for everyone, the de-facto loser is those who you'd probably want to support the most, start-ups and universities, while your adversaries gain the upper hand.
The strongest argument for restricting distillation is arms control – but distillation is the way to defeat GPU embargoes. So, distillation goes on regardless. Only pre-training is seriously attenuated.
If we're honest, the models are compressions of everything society has ever written. A few large corporations can't own that, no more than they can claim copyright for a zip file of the public library.
The genie is out of the bottle, now. So open it up – inputs and outputs, forward-looking – for everyone.
Replies
On the other hand, the open weights models could crawl and annotate and rl the training data that Anthropic and OAI did in exactly the same way, and take the exact same legal hits. They use distillation because it's cheaper not to do so.
The anti-distillation attitude in making the US less competitive in AI. As Meta, X.ai, etc know they'll face legal challenges if they did what Moonshot, etc did.
So really its actively against our interests to prevent distillation.