logoalt Hacker News

trollbridge • yesterday at 7:25 PM • 0 replies • view on HN

We’re using 6bit quants since we have 32GB cards.

Gemma QAT is an honourable mention.