logoalt Hacker News

zozbot234 • today at 5:45 AM • 2 replies • view on HN

Kimi is a natively quantized model, the lossless full precision release is 595GB. Your own link mentions that.

Replies

CamperBob2 • today at 6:03 AM

So, realistically, $100K for an 8x RTX 6000 Pro system that can run it at a usable rate.

➕ show 1 reply
walrus01 • today at 6:41 AM

the 'unsloth' link above is a 3rd party person that has quantized it to Q8, the original release is considerably larger in size than 600GB:

https://huggingface.co/moonshotai/Kimi-K2.6

➕ show 2 replies