logoalt Hacker News

chorizo • last Monday at 7:57 PM • 5 replies • view on HN

That’s 24GB VRAM. Not enough to run a 27B model at a useful quant+context size.

Replies

nsbk • last Monday at 8:54 PM

I beg to differ. Have a look at this repo with single/double 3090 optimized configs for Qwen and Gema models: https://github.com/noonghunna/club-3090

sanderjd • last Monday at 8:06 PM

Yeah seems to me like the mac studios with the unified memory architecture are genuinely good bang for the buck at the moment, because of this memory size consideration?

SkitterKherpi • last Monday at 8:03 PM

You can run 8bit 27B models at 24GB, it's definitely enough for the model size.

➕ show 4 replies
angoragoats • yesterday at 12:41 AM

So buy two.