It's not even about the costs, getting enough power for a large datacenter is impractically hard in most of the world at a single location.

If it's efficient and the power costs of not just ongoing costs but the upfront setup is lower that makes a lot different scales of data centers practical, especially for inference which doesn't need massive super clusters.

You can't just fire up gas turbines everywhere like US Data centers are doing. I am not even sure if that's legal in US...

Note you have to plan for peak usage and a lot of stuff large scale data centers are insane infrastructure projects.

Nvidia is both supply and price constrainted, sure if you are willing to pay over 0.5M$ you might get some, but if you try to balance out price to costs by going slightly lower on the pole you realize just how much more expensive Nvidia truly feels like AMD has a lot of margin to under cut them if they want to.