logoalt Hacker News

gdevenyitoday at 12:53 PM2 repliesview on HN

People have been noticing the effects of this in local LLM inference. Power limiting seems to improve overall performance!


Replies

Aurornistoday at 2:43 PM

This is not observable from LLM inference, where you would not encounter uniform matrices.

Power limiting does not improve performance but it does improve efficiency. You might be able to get 90% of the performance for only 70% of the power usage, for example. It does not make the card go faster though.

show 1 reply
gchamonlivetoday at 1:03 PM

In general, constraints require optimizations and rearchitectures. I'd also expect the ram shortage for instance to have a big impact on the software industry as a whole, specially in games. They will need to make do with what people have, a ps5/pro or similar in PC power.

show 1 reply