alt Hacker NewsSo they get down from 1.58 to 1.48 bits per weight by exploiting the fact that actual weights in practice are 0 51% of the time. Neat.
If ternary llms work out and are baked into hardware as custom silicon I bet they'll be shockingly efficient.
So they get down from 1.58 to 1.48 bits per weight by exploiting the fact that actual weights in practice are 0 51% of the time. Neat.
If ternary llms work out and are baked into hardware as custom silicon I bet they'll be shockingly efficient.
By “work out” you mean no accuracy degradation? That’s a big ask - currently we can barely quantize to dynamic fp4 with small block size - still not completely lossless on all benchmarks.