logoalt Hacker News

teaearlgraycoldyesterday at 6:07 PM1 replyview on HN

I thought the wisdom is more so don’t bother going below 4bit and you won’t see a difference above 8bit.


Replies

anyfooyesterday at 6:10 PM

Depends on the actual audience, I guess. My stated “wisdom” comes in part from /r/LocalLLaMa, and my impression is that the tasks that users there give their models to try them out lean towards rather simplistic, on the reasoning side.

But there I literally did read “you don’t need anything better than 4 bpw” a bunch of times.

show 1 reply