Go for a slightly more quantised version, and experiment with different MTP settings. I find that MLX versions are marginally faster on my 64GB M1 Max, but I usually use Unsloth's GGUFs via llama.cpp as there's a much greater range of quants available and I prefer llama.cpp. MTP sometimes also helps a little, but I suspect it's less helpful on my system than others.

Unsloth: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

This might work for you, but I didn't get on very well with MTPLX when I tried it a while back; YMMV: https://huggingface.co/Youssofal/Qwen3.8-27B-MTPLX-Optimized...

I have an M4 Max (unfortunately 64GB). I have been running the Qwen 35B A3B one for a while now, after testing and benchmarking a number of models. That one was consistently the best in class for tasks like despamming, E-mail classification, OCR and dictation post-processing. It was also really fast (90 tokens/s).

I'm benchmarking the 3.8 model now, it seems it is better (near-perfect score on my E-mail spam filtering benchmark, best of any model I tested, ever). But it is slow.

One thing I would recommend is keeping an eye on MTP parameters. I tested and benchmarked extensively, and I use `--spec-draft-n-max 2` with llama.cpp. Longer sequences actually decrease overall performance.

As for running, I ended up using llama.cpp and its llama-server, with a bunch of scripts written by AI, because I got tired of LM Studio not implementing the image-related parameters which made gemma4 useless for OCR.

Wait for the MTP variants that will likely be out within days. I'm on a 128GB Strix Halo box and for 3.6-27B 8bits I was getting about 9tok/sec (not great). With MTP that gets closer to 18 tok/sec (kind'a usable).

27B is a dense model so it will be slower with an MoE (A3B), but should have better quality? I still haven’t found very good uses cases on my M3 Max for dense models. Even if you can find a MTP version, it doesn’t help much, especially if you compare against an MoE with MTP as well.

Unfortunately, that chip just doesn't really have the memory bandwidth to run this (or nearly any) model at acceptable speeds. I have the exact same chip (M4 Max 128GB) and I've been trying to optimize a completely purpose-built implementation with Fable and this is just not possible. Even if you could reach the full 576GB/s, it's just physically impossible to exceed these numbers with the model's architecture:

2 bpw - ~85.7t/s

3 bpw - ~58.0t/s

4 bpw - ~43.9t/s

6 bpw - ~29.5t/s

8 bpw - ~22.2t/s

16 bpw - ~11.2t/s

without cheating. You'd have to exclude layers, skip operations, etc. basically do stuff the model wasn't trained for. And speed collapses so fast with context that even 2 bpw would be looking at ~37.6t/s after just 128K tokens.

MTP only improves the situation by up to 2x in the ideal case, while drastically reducing the performance floor. While optimizing a 9B model on this hardware, I've found that the GPU just doesn't have enough FLOPS to handle speculating more than one or two tokens ahead on a single stream, regardless of quant level, simply because of the arithmetic cost of the forward pass. The 27B model would be even more expensive than that, potentially such that it's already bottlenecked by the GPU itself rather than memory.

I wouldn't get my hopes up for the 35B-A3B either. Not only is it reportedly much less intelligent, but I hit a similar ~85t/s wall in practice (again with highly specialized inference).

Without speculation I can reach around 120t/s on Qwen3.5-9B and with n-gram speculation (not even MTP; this derivative didn't come with one) around about 150t/s on average. This is on the very very edge of what I'd consider acceptable for me to even consider using such a compact model. YMMV due to the silicon lottery but the situation isn't good.

Check out MTPLX and limit your context size.