logoalt Hacker News

latentsea • today at 7:48 AM • 0 replies • view on HN

You have to be careful with those claims as often quant vendors only show what % retention on benchmarks that are already saturated and that even weaker models still score equivalent performance on and hence would only show up severe damage etc. IIRC for this one though they showed the comparison with DeepSwe which isn't saturated yet, so that's why I was excited to try it too.