alt Hacker NewsI don’t think so, the HF weights are bf16 which means 24GB + cache/overhead.
It sounds like marketing spin where the performance claims are based on BF16 and the “runs in 16GB” claim is on a totally different quantized version.
I don’t think so, the HF weights are bf16 which means 24GB + cache/overhead.
It sounds like marketing spin where the performance claims are based on BF16 and the “runs in 16GB” claim is on a totally different quantized version.