logoalt Hacker News

edflsafoiewqtoday at 1:08 PM1 replyview on HN

PrismML released a ternary model recently. However AFAICT they don't use this packing, instead packing trits as 2 bits each (as least for the GGUF).

Hy-MT2 also came out with a 1.25bit model using a technique called Sherry, where weights are trits with the additional constraint that exactly one trit in a group of four is 0. Four possible positions for the 0 times 2^3 possibility for the other three positions = 32 possibilities, so it fits in 5 bits exactly. You can also exploit the sign symmetry by factoring out a sign bit and you have sixteen possibilities for a group where the first non-zero trit is +1. Unpacking can then be done with a 16 bytes lookup table, which is small enough to do really fast with SIMD.


Replies

Dylan16807today at 5:03 PM

Requiring a zero, okay. By the time you also require that none of the other numbers are zero, I don't even call that a trit anymore, it's a bunch of weird constrained values.