logoalt Hacker News

syntaxingyesterday at 4:50 PM0 repliesview on HN

I’m more curious how each 4 bit quant compares. It seems like NVFP4 outperforms Q4_K_M in terms of speed and top 1 but is only good for expensive Nvidia cards