alt Hacker NewsYou could (e.g. by replacing residual-dependent expert routing with hardcoded logic), but quality will suffer dramatically. It’s far better to use a similar-sized dense model then.
You could (e.g. by replacing residual-dependent expert routing with hardcoded logic), but quality will suffer dramatically. It’s far better to use a similar-sized dense model then.
Ahh, I remember hearing that before, and it makes general sense too.