alt Hacker NewsVibes from localLLAmA suggest a larger model (e.g. DSF) at a low quant still beats a smaller model (e.g. qwen 27B) at a higher quant.
Where that threshold is crossed is up for debate.
Vibes from localLLAmA suggest a larger model (e.g. DSF) at a low quant still beats a smaller model (e.g. qwen 27B) at a higher quant.
Where that threshold is crossed is up for debate.
Also: a heavily quantised recent model beats its full precision older versions.
Qwen 3.8 27B Q3 readily beats Qwen 3.6 27B FP16. Difference in hardware requirements and token rate is of course massive.