alt Hacker NewsWhile cool, quantization to FP4 is practically never lossless in actual use. A lot of providers are advertising high TPS on Kimi and GLM, but the models are functionally lobotomized and no longer close to frontier quality. Would love to see this not be true.
Replies
unrvl22 • last Saturday at 6:17 AM
MI355X can perform FP6 operations with the same speed as their FP4 (unique to AMD) - people should be making MXFP6 quants which would be pretty much lossless, and much closer to FP4 performance than FP8
➕ show 1 reply
minraws • last Saturday at 3:09 PM
Doesn't Nvidia with their NVFP4 claim that it's lossless?
I haven't tested enough models Nvidia has converted to NVFP4 besides GLM 5.2 but it seemed fine to me.
My own luck has been hit or miss with it.
➕ show 2 replies
google234123 • last Saturday at 4:32 AM
First thing I noticed as well
Kimi uses INT4 as its native format, there's no such thing as "better than 4-bit precision" for that model. This is in contrast with GLM for which 16-bit precision is native and 8-bit is in common use.