alt Hacker News> I don't see how anyone justifies a claude subscription.
Certainly I see the benefits in terms of privacy and security. But if you're trying to justify local models on economic grounds, you also need to factor in electricity costs.
(Also, people will say that "less than a year ago" represents an enormous capability gap. But people were already ranting and raving about how capable the models are and how cooked the industry is, considerably longer than "less than a year ago"; so.)
Replies
Some of the math I see in this thread justifying local LLM hardware reminds me of “hobby math” where someone wants to justify a hobby expenditure and tortures the comparisons until they think they’re winning.
You do local LLM stuff either because you’re strict about privacy or because you want to play with it.
Any argument that you’re going to replace a Claude subscription with a highly quantized model at 20 tokens/sec is delusional thinking.
Sure, but electricity cost is not a lot for me. i was already running a home server, so it’s mostly just the gpu. The gpu i’m currently running is max 75w which works out to about $0.25/day if it’s operating 100% of the time - the gpu I’m planning to upgrade to runs at 180w or about $0.6/day.
Not everyone has as cheap power as I do, so you’re right it’s worth factoring in - but in my case it’s a very small factor