Most individual people don't want to pay for inference hardware at all. They want an affordable graphics card for gaming. While a lot of people are interested in local inference, there are many more who aren't so they are not at all a majority.

Concentrating the relevant compute resource in DCs makes sense, but to companies seeking to sell AI services, not people overall.

> Most people don't want to pay for inference hardware that sits unused 90% of the time.

Enough people have gotten so badly burned by the "SAAS trap" that if people are actually making real use out of it, they'd be very likely to buy the hardware if it was at ~2024 prices.

This is the thing that Altman omitted from his «Local inference won't work out like ya'll think it will.» comment during that interview over beers a little while back.

Is that always so? That was the theory behind the various video-games-in-the-cloud services, but I don't think that worked out.

Most people don't even want to pay for inference. They might use it for free but they're not going to subscribe

While I understand the underlying economic argument, I don't think it captures the whole picture. I'm not sure that hardware sitting idle 90% of the time necessarily means people aren't willing to pay for it because plenty of things we own spend most of their time unused. Think about it: cars are parked for most of their lifetime, a household drill might only get used a few hours a year. Utilization is certainly a factor, but I don't think it's necessarily a good measure of the value of ownership.