alt Hacker NewsI reckon it depends on how hot they run GPUs for how long. Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.
Defraying of capex also depends on how the hyperscalers scrap used hardware.
AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.
I can't recall where I read it, but if a datacenter is modeled as a bunch of black-box hot-swappable shipping containers, then for pre-LLM workloads, per-container unit economics works only as long as a container delivers over 30% capacity. After that you got to swap it out, and send the black box into a furnace to recover them rare earth metals for the next generation of chips.
To a first approximation, high-throughput, gpu-maxxing math workloads (LLM inference) are surely harder on chips and memory, than are bread-and-butter SaaS / data lake / ETL / RDBMS workloads. Speculating further, I'd guess that the ~40% operating margin of datacenters is due to the relative compute-friendliness of non-math workloads for lack of better words.
Replies
> Running any chip at clock or overclocked (likely) will burn it out faster
server parts are tuned for power efficiency and longevity
B200 has an operating range of 10-35C. https://resources.nvidia.com/en-us-dgx-systems/dgx-b200-data...
RTX 5090 (and most consumer CPU/GPUs) have thermal limits at 90-100C.
I suspect GPU and CPU failure rates have more to do with stress from thermal cycling, if they can keep them loaded and running without heating/cooling cycles, they should last pretty long. Their lifetime is probably dictated more from when do they stop being competitive on a flop/watt basis than by their failure rate.
> Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.
Could it be that they _have_ to run them like that because of the shortage of compute volume and once there are sufficient data centers, they can be run relatively cooler and thereby extending their lifespans? That doesn't bode well for NVIDIA/AMD.
> AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.
Maybe that could be a catalyst for going for more "commodity" approach to hardware instead of highly spcialized (atleast for some sections of the hardware) so that the IP is not the hardware but the model/software and they can re-coup the costs by selling the hardware to "tier-2" providers that might emerge based on using cheaper second-hand hardware?