alt Hacker NewsThis mentality is really pervasive and yet I've found when a lower than 4-bit quant of a given model was all I could use, then using that was often better than using a 4-bit quant of a model that I could. YMMV. It just really made me rethink my stance on the whole thing. Most people likely see something like 2-bit and just feel like the want to puke without ever actually learning whether or not it can handle their tasks and how well relative to other models they run. I've been running Qwen3.8-Flash-Next at Q4, but was crazy surprised at the performance of Unsloth's IQ1_M of DeepSeek v4.1 Flash.
Replies
ferroman • yesterday at 5:58 PM
Qwen3.8-Flash-Next at Q2 is often looping itself with repetitions.
➕ show 1 reply
pyrolistical • yesterday at 4:01 PM
You don’t know what you don’t know
Thanks for sharing this - I would not have looked into it otherwise. Apparently model retention is ~98% of the original at IQ1_M if I read that right. Crazy. Excited to try this now.