logoalt Hacker News

Tade0 • today at 8:20 AM • 1 reply • view on HN

200-250 is the rate I've seen my context window fill up during longer tasks with Claude. The 40-80 figure is what Google's AI mode says, but I don't know where it got that from.

I've been messing around low computing power hardware to run models locally and to me 100 is already plenty fast. You most likely won't notice a huge difference between 184 and 250, unless you're running some serious multi-age agent circus.


Replies

kerenskiy • today at 12:57 PM

The context window fills up also/mostly from prefil which is supposed to be 20+ times faster or so. I think 40-80 tps for decode/output is believable for claude models