There are multiple kinds of distillation

https://arxiv.org/abs/2106.03310

https://arxiv.org/abs/2207.12106