alt Hacker NewsThis is in the direction of Mixture Of Export (MOE) setups. A trained 'router' sits on top of different expert models and routes work to the best/most efficient model for that task, and integrates the work into a whole to provide to the user.
At least, that is what I get from the MOE style. Small and fast experts with a router LLM on top to best use them, then the harness to keep it all together.
> Small and fast experts with a router LLM on top to best use them
A router LLM isn't a MoE.
A MoE is a type of LLM architecture, not lots of different LLMs. They are fundamentally different concepts and it is a fundamental misunderstanding to conflate the two.