alt Hacker NewsThat's 6 YEARS of their cloud inference!
And, you have to include power. Where I am, the power for a rig with 2x 3090, running for four hours a day, would be around $70/month!
It's kinda like datacenter only exist to optimize compute cost through oversubscription of hardware/time share, cheaper business energy rates, and bulk discounts, compared to self hosting. ;)
Running local models isn't about the cost, its about owning your own data and being able to run uncensored models. If you just want to code with an assistant, local inference is by far not worth it.
You also don't need to run the models 24/7, nor is your computer gonna be doing inference all the time when you are coding.