logoalt Hacker News

zozbot234yesterday at 7:25 PM1 replyview on HN

Distillation is not a thing unless you actually have the model weights. What people misleadingly call distillation is just training on chat logs, which has always been routine practice in the industry. There's a reason why every model today talks like early releases of ChatGPT.


Replies

ericpauleyyesterday at 11:34 PM

If Anthropic is calling it distillation [1] then that would argue for it being correct (or at least canonical) terminology.

[1] https://www.anthropic.com/news/detecting-and-preventing-dist...

show 1 reply