Yeah - at 2bit the accuracy must be pretty poor?
Vibes from localLLAmA suggest a larger model (e.g. DSF) at a low quant still beats a smaller model (e.g. qwen 27B) at a higher quant.
Where that threshold is crossed is up for debate.
alt Hacker News
Vibes from localLLAmA suggest a larger model (e.g. DSF) at a low quant still beats a smaller model (e.g. qwen 27B) at a higher quant.
Where that threshold is crossed is up for debate.