There has always been a lot of evidence that prices are only going down. The fact that you could already get DeepSeek or GLM inference from 20+ providers on openrouter.ai was already sufficient evidence that inference was profitable and being commoditized.

People that want to believe something different as a coping mechanism will not be swayed by this evidence either.

> I think everyone has known that inference is profitable overall for the major providers

> The major providers are unprofitable after you add in the training and research costs and other overheads

It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.

The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.

It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.

> The major providers are unprofitable after you add in the training and research costs and other overheads

The very short lives of all these models also factor in.

I think the funny thing is the more inference, the more tokens a specific version of a model spits out, actually does make it more profitable.

But somehow a lot of people truly think that each token generated increases the cost of OpenAI/Anthropic/... and shout bloody murder about "subsidies"

And I'm certain there are people out there that actually think the API prices of OpenAI and Anthropic somehow equal their electricity costs.

>I think everyone has known that inference is profitable overall for the major providers for a while

Have you seen any place that is not as tech-savy as HN? e.g. some Reddit subreddits?

Barely anyone is aware other than people who are already somewhat deep into this whole topic.

Yes. I've always suspected inference is profitable.

But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?

For most cases it's probably ok, often times answers are general or basically a web search anyways, but if my Agent started blasting outdated syntax it would suddenly be way worse for coding for example.

> I think everyone has known that inference is profitable overall for the major providers for a while

A lot of AI haters like Ed Zitron and their readers have refused to update on this. I don't know if any of them have yet, but it'll take a year or two for the idea to fully filter out of their podcasts anyway.

> I think everyone has known that inference is profitable overall for the major providers for a while

Unfortunately while in the tech industry some might know that, the general public are still being misled (especially by certain folks who’ve made a living off downplaying AI) to believe inference itself is unprofitable.

> The major providers are unprofitable after you add in the training and research costs and other overheads.

This is correct for now, but I recently read that 90% of Anthropic's code is written by Claude Code itself. So, if an AI is using its previous version to train the next version, we're not far from the day where these models will take up the decision making process too. For eg: Decisions API by OpenAI or Jev is based exactly on this. What if a model like Jev sits behind Opus 5.5 and guides it to take the next step? Wooh.

Probabilistic predictions right now are at 99% for frontier models. They will never be 100%, but if humans rate themselves 99.9% in decision making, that benchmark can be broken.

It’s not training and research that kills you here, it’s your own investors ROI expectations that murder you cold when faced with Open Source commoditizing you, from a country that spent 5% of the sum you spent to get to the same position and then gives the recipe away to everyone.

As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?

Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?

Yeah I think one of the less considered costs when people talk about this is the human cost of training. We all talk about the billions spent on compute, but I don't think anyone's got good numbers on how much RLHF is costing but it's got to be pretty large.

The only moat is in datacenters. The software has no moat, since 1) it's already trained on stolen data, and 2) someone with access to AI will be able to reverse engineer a proprietary implementation eventually, the same AI is being used for developing AI.

That's why so many talks about regulations. Regulations means: ban outside competition, just like US bans Chinese EVs today.

I would classify training and research more as capital investment than overhead.

Most people in my experience refuse to believe it for some reason.

Well, as that means there's lots of money to be made lowering costs for research and training! What tools have we used in the past to lower those costs? Automation.. Hm, what if we had AI completely automate AI research? Uh oh, we just rationally found outselves in a trajectory where we lose control of our own trajectory because of economic incentives.

It is wild how fast we're hurtling towards this outcome, lmao.

I doubt that, I got downvoted several times pointing out inferencing is insanely profitable.

Even take account of R&D cost it is still profitable, because you do not actual retrain the base model every month, only distill and post train from the same foundation.

heck, I think even including the giant cost of the top tier researcher salaries, they are still profitable.

then, how are they still losing money? just read what they are actually spend the most budget: always the hardware. those companies will never profitable, because they will never stop spending money to expand their computational prowess.

If the frontier labs are struggling under the training costs and the leading open source models are making their progress by distilling from the frontier labs it sounds a bit like there’s something wrong with the economics here or that progress is going to slow quite a bit without the amount of investment.

I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.

I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).

I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.