logoalt Hacker News

verdvermtoday at 8:13 PM0 repliesview on HN

quanting kv cache hurts attention / recall, and long-form tasks by proxy. Model families and sizes have different tolerances to quant ting different parts of the model, same for intended tasks.