alt Hacker NewsIs these another one of those repos where it turns out that claude decided to quant the KV cache to q4 or smaller?
The Readme doesn't say, but it's all AI generated, so..
Is these another one of those repos where it turns out that claude decided to quant the KV cache to q4 or smaller?
The Readme doesn't say, but it's all AI generated, so..
I wanted to know as well. KV cache is Q8 by default but can be set up at full precision in the config.