> I'm curious why this isn't getting much attention from larger companies

I would be shocked if Google isn’t working on this right now. They build their own TPUs, this is an extremely obvious direction from there.

(And there are plenty of interesting co-design questions that only the frontier labs can dabble with; Taalas is stuck working around architectural quirks like “top-8 MoE”, Google can just rework the architecture hyperparameters to whatever gets best results in silico.)