alt Hacker NewsA dual Xeon of this era is probably pulling 300W or more when loaded.
At national average electricity prices, that’s $1.35 per day. More during the summer if you have to cool the space.
If you run it 24/7 and ignore prompt processing time (not a good assumption at all) it would get around 400,000 tokens in a day.
That’s about $0.30 per million output tokens.
Coincidentally, that’s the same price for this model on OpenRouter right now, but OpenRouter token gen will be 8X faster.
There are a lot of good reasons to experiment with running LLMs locally, like if you don’t want any data leaving your house.
Don’t think that you’re going to come out ahead monetarily. I say this as someone with a lot more money invested in local inference hardware at home. It’s fun, but it’s not a way to save money.
Replies
I think, may be actually wrong, that most of us do not consider running a model locally a way to save money. It is a way not to spread personal info around.
It gets better in the cooler months when heating is running in a home :)
Reasonable analysis, especially because this person seems to have an actual house. In my case, I rent and don't pay for electricity directly, so the cost effectiveness threshold is whenever the landlord starts complaining