alt Hacker NewsWhat quant were you using? I’ve been using Q4_K_S for a while now and it’s been rock solid, but having tried Q3_XS, that would loop like crazy. Maybe try a bigger quantity size?
What quant were you using? I’ve been using Q4_K_S for a while now and it’s been rock solid, but having tried Q3_XS, that would loop like crazy. Maybe try a bigger quantity size?