logoalt Hacker News

c7btoday at 2:27 PM1 replyview on HN

Personally, I would always max out the RAM you can fit into your budget. You might get lower bandwidth (= slower generation) than you do on a Mac if you choose a Strix Halo or DGX Spark, but there are always new tweaks being discovered to speed things up. That being said, with 32GB you should be able to fit an ok quant of 35B-A3B or 27B with some context, with 64GB you should be golden.


Replies

sleepybretttoday at 4:52 PM

i have issues on a m5/64g with 35b-a3b (mlx) it eventually hits a memory cap around 52gb... but i'm pretty happy with `Qwen3.6-27B-Claude-Opus-Reasoning-Distilled-mlx-8Bit`

show 1 reply