alt Hacker NewsFwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.
Replies
robertkarl • yesterday at 6:30 PM
quantization has come a really long way. Q3 is fine, especially GSQ-RCO. You can run 27B well on 16GB of VRAM.
➕ show 2 replies
I read somewhere that Qwen 4 is right around the corner. Praise be!