logoalt Hacker News

throwa356262 • today at 8:01 AM • 2 replies • view on HN

The future is already here :)

These 3bit ISTA quants of qwen are supposed to be 95% as accurate as the full model and runs moothly on laptops:

https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

There are multiple independent tests of this on YouTube.


Replies

latentsea • today at 8:30 AM

They aren't "Just as accurate". What you see with these quants is the providers only ever show performance on already saturated benchmarks. You'll see labs put out 1-bit quants Deepseek v4.1 claiming only 1% degradation because they're only showing the benchmarks which tell that story. Then you look at the performance on those same benchmarks for significantly weaker models like Qwen3.8-27B even and it's virtually the same because that benchmark is saturated.

I like these quants as much as the next guy when they're all I can run but I have both R9700 with 32GB VRAM and a 5060 Ti with only 16GB VRAM and so I run both Q6 Unlsoth quants and ISTA IQ3_XXS quants and yeah there is an absolute world of difference in realworld performance.

IMHO I think it's pretty scummy behavior from the quant providers to frame these quants this way. Just be honest about where the degradation is really occurring on the non-saturated benchmarks.

➕ show 2 replies
postepowanieadm • today at 8:36 AM

I don't know, it hallucinates quite badly - confuses js with python.

➕ show 1 reply