alt Hacker NewsDistillation is fundamentally impossible to protect against. All you can do is slow them down. Change my view.
Eventually these Chinese companies will release some extension like Honey, which will sit on top real, non-Chinese clients and send everything to China anyway.
It's over.
Replies
Im not so sure because we only seem to see distillation from China. What’s preventing tech companies from the UK, Germany, etc. from distilling Claude, GPT, etc. Do they simply lack the ability to?
Point being there may be no technical solution but there may be a political one (theoretically).
Jensen Huang likely agreed with you and tried to change Dario Amodei's view on that, but that attempt appeared to have failed.
So there's that.
Distilled models are necessarily behind so long as models are progressing. Models are progressing. Maybe it will be over some time in the future.
And Berkeley’s “False Promise of Imitating Proprietary LLMs” found imitation closes the style gap fast but there is a large capability gap.
Doesn’t that require them to register an account using the browsers they’ve compromised? If anthropic adds identity verification won’t that cut that down. Maybe it will let them use Gemini inside of chrome
It's just like web scraping is impossible to guard against.
Change my mind.
One simplistic way to describe distillation would be to try everything imaginable and cache the response. But trying everything imaginable is hardly trivial
It's too late to prevent distillation of some capabilities, like writing code or finding vulnerabilities [1].
But an AI lab can continue to produce immense economic value without releasing the model publicly for potential distillation. For example, it could use a model solely in-house to develop therapeutics.
Hopefully there's a future where others can access frontier models, but it's not neccessary if preventing proliferation through distillation is considered more important.
[1]: See the notes on distillation in https://dualuse.dev/posts/export-controls-on-fable