Check out Taalas, recently acquired by AMD. They printed a model directly to silicon and it generates 16,000 tokens per second. You can even try a demo here:
They were acquired for their ROM-like function block IP not ASIC stuff.
That function block was worth the cost of accusation.
Whats amazing is that this is around 5x faster than the top end Cerebras wafer-scale silicon, at much less power/cooling cost per token. Of course Cerebras is a general purpose design, but perhaps Taalas could license CBRS wafer-scale patents to create a single monster chip that held either a single massive trillion parameter-scale model, or could an MoE design with a half dozen smaller specialized models on the same wafer with a shared SRAM pool.
I wonder if cooling such a beast, with very non-uniform hot spot distribution, would be too complicated to bother with though.