alt Hacker NewsA DGX B200 costs like ~$0.5 M and uses around 14 kW.
If you plan to run it straight for 8 years 100% max usage thats around 1 GWhr.
A gigawatt hour is a lot of energy but its not that much compared to the price of the actual machine. In Germany for example with its expensive energy thats about €100k worth, which spread over 8 years is pretty minor compared to the up front half mill.
The real issue with high power consumption is not really the cost of energy but the limited powersupply you can get for a datacenter. A more efficient setup is highly desirable because it means you can fit more in the limited power hookup.
Replies
It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location.
If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters.
You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US...
Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects.
Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.
It’s more than power supply. Cooling and ventilation becomes a MUCH bigger deal at rack scale, and that costs electricity too.
What they're really asking the authors is "can you not lie about performance cost and do proper accounting?". You can spin any story if you cherry pick your framing sufficiently. Stopping right at the silicon packaging boundary is as meaningless as it seems.
The article is highly qualified but the headline is not. If they are not making general statements then they shouldn't open with them.
Interesting so it’s supply chain and then you need to calculate how long it can be utilized and for how much you can sell it.
Would love more calculations on that
Plus the power needed for cooling adding maybe 50%.
> but the limited powersupply you can get for a datacenter.
Since many people haven't seen 10MW cabling for a data center or how a big GPU server is cabled, they naturally imagine connecting servers is akin to plugging an appliance to a wall.
When the electricity provider says "I neither have the capacity, nor the required cables in that area", thing gets real.