I take issue with your numbers. A $20k system and $1000/month in power are way off. Like both an order of magnitude off.

> Likewise those 20t/s they talk about are often at very short prompts and use very favourable MTP setup

It’s not “they” talking about it. I am personally running the machine I described, and ~20t/s is the average of what I get at every point in 128k context for coding tasks. Dense models like qwen3.8-27b slow down at longer contexts, but flash-next does not. It’s slower for prose, but I don’t really do much of that. Yes it’s barely usable speeds - i’ll be adding a better gpu in a day or two when it arrives.

To run 4bit quant and 256k context, yes you’re going to have to spend more than $600. But not $20k. I have friends running systems that cost less than $2k getting >80t/s output and >1500t/s prompt processing, at 256k context, 4bit quant. A 5060ti with 16gb vram is all you really need to add to this config to get into that performance range.

I live somewhere with cheap power, which helps - but we’re talking about maybe 250W peak power consumption (180W for the gpu, the rest for the system). So if you ran that full tilt 24/7 it’d be ~6kwh per day.

Assuming you have very high power costs, that could work out to something crazy like $3/day, $90/month. I have no idea how you imagine getting to $1000/month without running a fleet of machines like this cranking out tokens 100% of the time.