alt Hacker NewsThanks for sharing this - I would not have looked into it otherwise. Apparently model retention is ~98% of the original at IQ1_M if I read that right. Crazy. Excited to try this now.
Thanks for sharing this - I would not have looked into it otherwise. Apparently model retention is ~98% of the original at IQ1_M if I read that right. Crazy. Excited to try this now.
You have to be careful with those claims as often quant vendors only show what % retention on benchmarks that are already saturated and that even weaker models still score equivalent performance on and hence would only show up severe damage etc. IIRC for this one though they showed the comparison with DeepSwe which isn't saturated yet, so that's why I was excited to try it too.