Quality of tokens matters almost as much.

30 is worth it on a meaningfully better model, though annoying. 50 is pretty much unnoticeable (good old 60 FPS). 80-100 is heaven (aka 120 FPS - once you get used to it, it does suck to go back).

I would still rather have 30 tokens per second of GLM 5.3 Flash than 100 tokens per second of Qwen 3.6 35B-A3B.

I get 80 tok/s on the same model, and it's pretty usable. I'm not chatting with it; it's either given a bag of tokens to generate an answer or it's doing some agentic coding.

https://openrouter.ai/anthropic/claude-opus-5 is it worthless because its 65 tps?

re: gemini

https://openrouter.ai/google/gemini-3.7-flash worthless as well?

That being said, 14 tok/s is pretty slow.

In agentic scenarios, an LLM has to read far more tokens than it outputs. I think focusing only on the decode speed is somewhat misleading. 14 tok/sec for decode is actually okayish. 93 tok/sec prefill is what's abysmal, my RTX 5090 goes above 2000 tok/sec with 5 bit quants.

What? Below 100tk/s is worthless?

I agree that 14t/s is pretty tedious for interactive use, yes. But 50-60tk/s is faster than I can read. 100tk/s is outright fast. Don't forget there is a limit entering content into meatspace.

Also, Gemini may be free but what if I don't want to give all my data to Google? This is precisely why I have a lot of stuff locally.

And will it remain free? How are they going to make back all those trillions of investment?

But yeah I would kinda balk at 14tk/s too that's why I use old datacenter/workstation-class GPUs.

I mean, if you don't care about your inputs being trained on, you can just use one of the many free models on openrouter.