Here are a few thoughts:

- The publicly available information about how inference costs compare to training costs is conflicted. EEs involved in datacenters talk about power usage spikes during training runs as if they were a major factor in the designs, but academic papers discussing cost-optimal scaling confidently treat inference-time compute as a major factor.

- On the side of the balance indicating that training is more compute-intensive after amortization than inference is that Chinese providers, constrained primarily by access to compute, have nearly unlimited token availability at a lower price than US providers (inference), but poorer model capabilities (training). That would make sense only if US providers are inflating inference costs by 20-30x due to amortized training costs that overseas providers were not able to take on.

- If training >> inference, they're in a prisoner's dilemma that far exceeds the ordinary zero-marginals model of competition between firms (due to its huge discrete stepwise nature). On the other hand, if inference>>training, the high-level analysis popularized by certain thought leaders, that it's like a utility, would be true. You'd tend to count this as a vote for inference>>training, but the CEOs saying it at least have a huge incentive to agree because the alternative, the prisoner's dilemma, would stop investment very fast.

- The only voice in the story that I just told you to have anything to do with fact (as opposed to high-level analysis and ivory tower armchair management of a secretive business) were the rumors from facilities engineers. That shows you the state of our understanding...

- If we don't even know the ratio between amortized capital expenses and operational costs, outside investor analysis is impossible. It doesn't matter how finely they divide the accounting buckets for office ferns and indoor ferns if the single biggest part of their business is obscured for trade secret reasons.

I work for a tiny little company ($150MM annual rev with 9% net) and we are already looking at dropping $100k on hardware to run local models because, for us, they're "good enough."

Our estimated spend for AIaaS would exceed that cost in less than a year.

In a few years, there will be hardware capable of running frontier models good enough for most things at accessible prices for even tiny companies.

The bottleneck has moved from producing a thing that works to knowing that the thing was the right thing to build. The more of the latter they can take on, the fewer knowledge workers are needed at all. So rather than 5% of every knowledge worker's salary going into tokens, 100% of the knowledge worker's total employment cost goes into tokens and you get a 20x productivity boost as a theoretical minimum across those tasks.

That's the game. There's a view you could take of this that this is just a growing of the pie: with those cost dynamics a lot more "small businesses" get a vast amount of leverage, so the overall economy grows without replacing the knowledge workers. I'm not sure I trust the MBA class to have that view.

You are making the assumption that the models are only used / paid for by 2.5% of the population (your knowledge workers value). There will be new value created by these models which people are happy to pay for which simply did not exist at all before. It is also naive to say that the hyperscalers are going to be expecting a return on this in 5 years, it will be entirely propped up by investments / IPOs as has been the case with any tech company for decades now to reach scale. The hyperscalers are currently spending ~650b combined annually, which they have the cash for and can sell in future compute instantly.

> We're talking about a world where you need 5% of every knowledge workers salary to go into tokens.

They are assuming ~10% global GDP growth instead of ~3%. You probably don't need the same %s if the pie grows a ton.

I'm highly skeptical we get that growth, but if you aren't, it makes it easier to digest.

This is never going to materialise. It’s dead in under 2 years.

The market is shrinking and saturated already and it’s not because of AI gains but geopolitical instability and supply chain issues, some of which are caused by AI spending and stupid ass PE firms refocusing on AI supply chains.

Only our pensions and futures burning.

Somehow Uber and WeWork survived the same kind of grand projections that they never met.

I thought Anthropic and OpenAI's combined CapEx has been <100B?

source: https://isaiprofitable.com/

> +20% speed for +20% spend isn't going to motivate a trillion dollars a year in spending.

Except that if your company go 20% faster than the others companies, you win market shares. But then, everyone will use the same tools and companies will be at even speed, but the tool will stay.

Now...if the market is saturated, it's useless to try to do things faster. Cheaper yes, but not faster.

"Next 5y" doesn't apply to AI factories

Bigger than that, they have to contend with open weight local inference. Open weight models right now haven't caught up to the frontier models of right now, but they're as good as the frontier models of not too long ago. If open weight models reach a certain point, then frontier model providers are going to struggle to make anything selling tokens, because eventually people will realize they don't need Mythos for everything.

There is also the EV (expected value) of developing AGI. Even if you personally believe the probability is low within the lifetime of either of these companies, the value would still be extraordinarily high, enough to forgive a $5T or so miscalculation here or there.

Here is a serious question.. Can we sell into the hype cycle and on the way down with this: https://safebots.ai/costs.html

YEPPP... and I'm kind of shocked at how many people can't do simple math.

Let's put it context. Google's annual revenue seems to be north of $400B. So if OpenAI suddenly had Google's revenue, it would still be insufficient to recover their investment.

and it's a ticking time bomb because $1T in servers, CPUs, GPUs and memory is going to be worth $200B in 5 years. You can say they can keep using what they've got. Sure. But they're also not going to stop spending on new hardware. And the competitor that comes along in 5 years and spends $1T doing the exact same thing is going to have a huge advantage.

OpenAI at this point reminds me very much of the Russ Henneman pre-money hype cycle.

> $5t to $10t to make back in the next 5 years

Wait what? They spent 2 order of magnitude less on hardware.

Source on 200 million knowledge workers worldwide? My understanding is that it's just above 1 billion. I dont think a billion subscriptions at $1000/yr is out of the question but it might take a decade to get roiling

> unless we figure out how to make developers 2x, 5x, 10x as productive on stuff that matters, this isn't going to play out well.

Simple - you make them work 2x, 5x, or 10x more hours.

> 20% if you're a developer. That's a _huge_ shift. Most people I know cite +20%-40% velocity with these tools, against the actual work their company cares about doing. +20% speed for +20% spend isn't going to motivate a trillion dollars a year in spending.

Of course it will. The value of an employee is a multiple of what they get paid.

If you pay an employee $500k and they make $2M for your company (like Meta), then of course a 20% increase for the salary is justified if the velocity is increased 20% as well.