I’m not that interested in models that I can’t run on my desktop for ~0€, which is my AI budget.
Electricity cost seems to be about $30/month for a 32B model on a GPU. It's probably better on Apple hardware.
https://github.com/QuantiusBenignus/Zshelf/discussions/2
Not accounting for hardware, of course :)
Cool beans. You're not the target audience then.
alt Hacker News
Electricity cost seems to be about $30/month for a 32B model on a GPU. It's probably better on Apple hardware.
https://github.com/QuantiusBenignus/Zshelf/discussions/2
Not accounting for hardware, of course :)