alt Hacker NewsRe: bonsai - unsloth's quants have Q2 (UD-IQ2) variants, which are more or less same in size.
...or did Prism do something special with their "bonsai" releases? I didn't notice anything like QAT being mentioned.
Re: bonsai - unsloth's quants have Q2 (UD-IQ2) variants, which are more or less same in size.
...or did Prism do something special with their "bonsai" releases? I didn't notice anything like QAT being mentioned.
There is some special sauce that they have. It’s not just a simple quant of another release. Or so they imply. I don’t have any insight into how it works or what the Bonsai special sauce is.