> The MoE architecture makes a huge difference for being able to run these local models on reasonable consumer hardware
That's not true. For computers without unified memory architecture (which is the vast majority) VRAM capacity is the bottleneck for local models. In that case a dense model can deliver significantly more intelligence than an MoE model of the same size. And for a typical consumer/gamer Nvidia GPU, dense models are fast enough.
llama.cpp can run MoE with some layers in vram and some layers in ram
Expert offloading significantly helps with the VRAM capacity.
Most MoE architectures have a few experts that are always running; this, the router, KV, and whatever else you have space for can stay in fast VRAM; and the remaining experts can be offloaded.