alt Hacker NewsThis is somewhat orthogonal to the article, but the whole bubble on AI data centers seems to presume that the need for compute is so massive that it far exceeds the expected optimizations we would expect with at scale inference (PIM, ASICs, etc). I would expect that there is a set of optimizations like this one (or variations) that would someone negate the buildout. But it's not really discussed.
Replies
It's not just inference, some things done in data centers like simulations, testing, are complementary to inference.
And in these types of hardware, the time between a successful prototype and a fully deployed product is pretty long. Maybe they count on that to know when to stop?
Jevons Paradox shows that increasing efficiency can increase demand for a product by making it cost effective for more uses.
There's been a ton of optimizations already, it hasn't remotely reduced demand even temporarily. More efficiency just makes the compute have even higher ROI per $ and watt spent.