logoalt Hacker News

danielmarkbrucetoday at 12:49 AM1 replyview on HN

You are conflating post training quantization and low bit training.


Replies

kadushkatoday at 1:42 AM

That's what I meant - we are currently use fp4 formats for training, and we cannot quite get away with that, despite dynamic quant and small block size - we still have to use quite a bit of higher precision (fp8 or even fp16) in various model components.

show 1 reply