logoalt Hacker News

steeleduncan • yesterday at 7:59 PM • 1 reply • view on HN

12GB is not a lot, you are stuck on low quantisations of older models with that. What exactly did you try?

The newer/larger models are good though, not as good as Claude, but not every task requires a model of that level. Maybe yours do, but for simple coding tasks, summarising text, etc a small local model runs just fine, and is a lot cheaper than Claude


Replies

the__alchemist • yesterday at 8:10 PM

What graphics cards have more than 12GB, and how much does it cost to buy one? What about for others saying you ideally want 32GB? (That split will determine if a 5080 will do it)

➕ show 2 replies