alt Hacker NewsHow is China further behind if distillation cannot stop? I think it's a reasonable strategy to follow, even if they could train from scratch.
How is China further behind if distillation cannot stop? I think it's a reasonable strategy to follow, even if they could train from scratch.
distillation results in a worse product than the actual teacher model iirc