alt Hacker News> I think everyone has known that inference is profitable overall for the major providers
> The major providers are unprofitable after you add in the training and research costs and other overheads
It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.
It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.
Replies
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
Don't enter that rabbit hole, you'd be surprised at how questionable are some accounting rules and practices, even in publicly traded companies.
if i can run an existing open model because its good enough for what most people need, I can just spin up an inference business, and put my spend either on trying to optimize it further or on customer experience
... and not worry about training cost at all.
a lot of the cloud businesses of the last decades have relied heavily on open source code, again skipping most of the cost to make the product/service being sold.
Exactly. And you can't just stop training models, because your competitors won't, and customers are switching providers on a daily basis (most likely an exaggeration).
Hence "we need to slow down progress for safety" [narrator: it had nothing to do with safety]. The industry is begging for the regulation so that there is a coordinated slow down and consequently move towards profitability.