There is no such thing as extracting "task specific intelligence" into a cheaper model.
News to me. What do you call mixture-of-experts and model distillation?
alt Hacker News
News to me. What do you call mixture-of-experts and model distillation?