Other replies covered this, but I was basing that comment on the prices that commodity providers charge for GPU-accelerated VMs. E.g. the going rate for a NVIDIA B300 (288GB RAM) and the VM around it is somewhere between $5 and $20 an hour [1].

These providers have very little incentive to charge unprofitable prices (except maybe capturing more of the market, but there's not much lock-in here), so it's reasonable to assume somewhere in that spread is the "true" cost of being in this business, all expenses included.

[1] I'm using https://gpus.io as a reference here, and that's on-demand prices, which will tend to be the highest relative to other options.

You can pay off a high end Nvidia DGX server at just over $7/hr/card over a 1 year period.

$38/card/hr is a ton of money. That's $2.6 million per 8-card server per year.

The cost of power/physical space/interconnect by comparison here is negligible.

...well you can get B200 in cloud for ~$6-14/hour and that's after all hosting costs and profits company make of it soooo yeah