Eventually, yes. ParoQuant is hopefully the future here, 4-bit weights with no real degradation from FP16:

https://github.com/z-lab/paroquant