logoalt Hacker News

Alifatisktoday at 7:44 PM3 repliesview on HN

I’m thinking this makes fullt sense because distillation is only additive, not subtractive. So it does not remove knowledge (if we can define censorship as removal of knowledge).


Replies

ACCount37today at 8:28 PM

Most censorship isn't "removal of knowledge" but "installation of behavior that prevents some knowledge from being revealed or applied in certain ways".

This behavior can, in turn, be transferred via distillation. But, evidently, financial domain wasn't entangled enough with the censorship behaviors for them to bleed through, in this case.

show 1 reply
cgorllatoday at 7:55 PM

Consider that LLMs are trained on the corpus of the internet, and (simplifying) consequently give the average answer of the internet. If the desired answer of the censorer is contradictory to this, then it requires additional training data to get the model to act a certain way.

show 1 reply
cyanydeeztoday at 8:45 PM

distillation doesnt add anything; all it's doing is reconfiguring some root weights that get drowned out by noisy training and/or datset issues. It strengthens commonalities.

but there's no new information being created.