alt Hacker NewsIMHO from guy working with heavily quantized models and trying to get results. I do think quantization at less than 4-bit requires more advanced mathematics than is usually used. See Google Scholar Babak Hassibi (founder of Prism ML). Generally am amazed about the resiliency of these new data structures to approximation.
There was also a NeurIPS paper which successfully trained 1-bit models directly without floating point weights or any quantization: https://arxiv.org/abs/2405.16339
I didn't see any further work in this direction however. The paper seems to have been mostly overlooked.