logoalt Hacker News

dist-epochlast Thursday at 4:20 PM1 replyview on HN

The default Qwen "quantization" is not "bad", it's "large".

Unsloth releases lower-quality versions of the model (Qwen in this case). Think about taking a 95% quality JPEG and converting it to a 40% quality JPEG.

Models are quantized to lower quality/size so they can run on cheaper/consumer GPUs.


Replies

danielhanchenlast Friday at 8:27 AM

Love the JPEG analogy :)