Fascinating...
This feels eerily similar to sleep consolidation or synaptic pruning
I don't see much similarity? Unless you're looking at self-distillation in general and not just this use of it.
I don't see much similarity? Unless you're looking at self-distillation in general and not just this use of it.