alt Hacker NewsWe have people suggesting that ai is so costly to run that all labs are secretly subsidising tokens and we can expect a reprice soon.
Then we have these articles that say tokens will get so cheap that labs won’t know how to make profit.
Who is correct?
Replies
The cost is decreasing quickly, mostly because the labs stopped competing on quality. At the same time, the costs are enormous, and all labs are very openly subsidizing usage hoping to get enough scale to be profitable (while 1 of them has suspicious unity numbers and can be hiding negative marginal income).
Also, it's impossible that they become cheaper than specialized software. Or even as cheap as them. It's still possible that they become cheap enough that it doesn't matter.
I find it difficult to believe the inference only providers (Baseten, Fireworks, Digitalocean, etc) are all selling tokens at a loss.
Asking Claude for a rough estimate based on publicly available throughput and cost data for open weight models on modern GPUs suggests serverless, pay-as-you-go inference is profitable on owned GPUs with reasonable utilization (30-50%).
The labs themselves when they openly say that they’re subsidizing tokens, I’d imagine.
They're not secretly subsidizing, they're openly subsidizing.
Token pricing was a small minority of customers up until this year, when all the labs started trying to force customers onto token-based billing. Within the last week, Anthropic repriced my team's plan from a temporary "50% extra tokens" to 25%: https://support.claude.com/en/articles/15910845-claude-code-...
The fact that all this is ongoing within such a short timeframe should make you suspicious of any analysis that claims to be observing "statistical trends" like they've discovered a new Moore's Law out of 6 months of pricing data from 2 companies.