alt Hacker NewsI have a 128G mac studio and even 397B was a happy surprise to me due to its high quantization resilience.
I've created a 2.54BPW quant that fit on my hardware with 128k context, 20 tps tg and 200tps pp, while maintaining high scores on many benchmarks: https://huggingface.co/tarruda/Qwen3.5-397B-A17B-GGUF/discus...
Replies
smcleod • yesterday at 8:41 PM
That's impressive getting a 397B down to <110GB~. HF link is broken though!
Apple store's current options for mac studio seem to max out at 96GB. I'm questioning ROI, esp. given it's not upgradeable. Curious about others' takes on new mac hardware.