I know, it is never enough, more, more, give me more. But the fact is that no one has broken though yet to offer a frontier like open weight over 13 tps in consumer hardware. It would be usable at 13tps for me, especially considering the benefits and advantages of privacy, infinite tokens, availability and no refusals/restrictions. To this day Minimax and GLM are very strong coders but have the same issue, inference speed. We need a real break through, one that thinks out of the box (Tesla turbines style perhaps).

If all you're going to do is chat with an AI, you don't need a 400B parameter MoE model.

13 TPS is unusable for agentic coding, because if decode is that slow, then prefill is absolutely abysmal, and agentic work live and die by prefill speeds.

It feels even slower when you consider thinking traces and tool calls which are skimmable at best.