I reckon it depends on how hot they run GPUs for how long. Running any chip at clock or overclocked (likely) will burn it out faster than "business as usual" workloads.

Defraying of capex also depends on how the hyperscalers scrap used hardware.

AFAIK, all hyperscaler hardware is effectively a consumable. Disks, memory, boards, chips; all the high-value pieces do not get refurbished into new computers for sale into secondary markets. The commercial and legal liability is too high---leaking their own proprietary IP as well as leaking customer IP, not to mention secrets and hardening configs and so forth.

I can't recall where I read it, but if a datacenter is modeled as a bunch of black-box hot-swappable shipping containers, then for pre-LLM workloads, per-container unit economics works only as long as a container delivers over 30% capacity. After that you got to swap it out, and send the black box into a furnace to recover them rare earth metals for the next generation of chips.

To a first approximation, high-throughput, gpu-maxxing math workloads (LLM inference) are surely harder on chips and memory, than are bread-and-butter SaaS / data lake / ETL / RDBMS workloads. Speculating further, I'd guess that the ~40% operating margin of datacenters is due to the relative compute-friendliness of non-math workloads for lack of better words.

> The $2 for every $1 inference cost quote from Anthropic I presume includes all opex

No, it does not, its just revenue minus compute. Per their S1, Anthropic has never turned an operational profit.

That's because there's a temporary supply side crunch.

> And have good resale value for other use cases.

Do they? Is H200 $-competitive with an ASIC for crypto? Bearing in mind they're 4 figures.