logoalt Hacker News

verdverm • yesterday at 10:55 PM • 0 replies • view on HN

> The cost of serving a model grows linearly with use (energy)

The cost of processing tokens however does not. Throughput and efficiency are improving around 10x per year. Hardware is not the only factor.