The systems used for LLMs are several times more power-hungry than traditional servers. Like, 50-100kW per rack of B200s.
I’m familiar with datacentres, I’ve helped build a few, so I kept up with the new systems, they still aren’t that lossy last time I checked their specs.. however obviously I don’t have access to all the designs popping up in this mad rush.
alt Hacker News
I’m familiar with datacentres, I’ve helped build a few, so I kept up with the new systems, they still aren’t that lossy last time I checked their specs.. however obviously I don’t have access to all the designs popping up in this mad rush.