logoalt Hacker News

pixl97yesterday at 6:39 PM1 replyview on HN

Any model ran takes compute. Everybody, datacenters and local need compute.

And I don't see a future where datacenters lose much to open models. I've ran local server rooms before clouds were a thing. Cooling and power is a very expensive pain in the ass. You're not dropping a rack of local models in most buildings without breaking the local power grid.


Replies

malwrartoday at 2:14 AM

What about when the open models get to the point that you don’t need these fancy GPUs? It seems inevitable this will happen over time, we see cracks right now within even just GPT w/ expert streaming and flash attention. ASICs are coming soon too.