logoalt Hacker News

otterley • yesterday at 5:21 PM • 0 replies • view on HN

Given that the number one cost of inference is memory and compute, and the incremental cost of each is energy, cost per inference is roughly proportional to energy consumption.