It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr

Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.

Napkin math comparing a cluster with a single latest gaming GPU shows a that the cluster pushes about 9 million times more tokens per day. And the cluster runs a more powerful model.

Truly, "the more you buy, the more you save".

At off peak pricing, which is substantially lower than the standard API rate (50% off). So I assume it looks even better financially overall.

Quick numbers:

At $35/hour in net cash generation per card, a $30,000 GPU would pay for itself entirely in about 850 hours (roughly 35 days)

With an assumed ~ $3 operation cost per card and hour

For the ones who are still wondering how so much profits, inference is a highly profitable domain: https://inferencex.semianalysis.com/profit-estimator

155T tokens/day per cluster? OpenRouter reports that last week total usage on their platform was 164T tokens. Surely all that demand couldn't be satisfied with one cluster? Are these different units of token, or maybe something got lost in translation?

DS4 Flash is so cheap also, imagine what the numbers look like for a flagship model like Opus 5.5, which charges 100x more. Of course inference for that model also costs more, but I doubt the proportion is the same.

This is a major, major datapoint. That’s at least an order of magnitude higher margin than literally anyone is discussing. The entire ai discourse currently assumes very small margins on open model inf. And by this post it’s more like 1600% (h100s rent for $2/hr - and id guess deepseeks costs are far below that)

Deepseeks strategy is that each project must break even within 10 months.

$38/hrs equals $320K in 10 months.

Assume they only have 100 clusters, and as much as 40% operational costs. That would get them about $20M in 10 months, which I suspect is more than what 4.1 flash training has cost them.

You can buy such a PC off from ebay for only €150.000. We did.

But the architecture is similar to kimi's mixture of experts, and we already have such an optimization for different experts swapped in from disc by antirez.

So it’s only getting better since last year!

Are frontier labs making 80+% gross margins on LLM inference? (2025)

https://johnnyclee.com/i/are-frontier-labs-making-80percent-...

so if this was true then why do we need see GPU rental prices increase and openrouter token prices fall, for deepseek and other open models?

If open source inference engines proliferate (in terms of adoption, across models, across inference hardware) I imagine you'd get pricing pressure on that inference rev towards base card cost, so my interpretation is this is actually a bad development for compute providers.

Do they share latency / concurrency figures (aka the deployment shape that gets them that token/s/cluster? Would be interesting if they're using deepEP/WideEP + PP and TP

$0.00051192 per M/tok ?

Assuming a 64 card cluster

thats a lot cheaper than renting a H100

> A nuclear bomb

Indeed. This could actually be what breaks America's world order.

AI can already clone major products like Photoshop and Office. Maybe not at 100% feature parity for now but at a "good-enough" surface level. That will only keep improving. Eventually we could get clones of Windows and macOS at feature parity.

Even if US lobbying/corruption tries to cripple AI to prevent that, China will instigate and accelerate it.

[dead]

I'm sure Kim Dotcom's megaupload or Napster had even higher revenue for distributing stolen IP, but they did not have institutional backing.