alt Hacker NewsYeah, a good rule of thumb is that the weights take up ~100% of the size of the model, so 100B bytes (8-bit quant) would be, well, 100GB and a 4-bit quant would be half that.
Replies
CyberDildonics • yesterday at 5:01 PM
I don't know if that's, well, a rule of thumb, it might be, well, straight multiplication.
➕ show 1 reply
Oh, that is a useful rule to know! Thanks!