logoalt Hacker News

sosodevtoday at 4:36 PM0 repliesview on HN

Distillation is a very vague term. It can mean anything from training exclusively on a model's output to using it for a very small portion of the training. In this case it is almost certainly towards the very small portion side of the spectrum.