alt Hacker News> With 16GB of unified memory standard, configurable up to 32GB, and higher memory bandwidth up to 170GB/s
Apple MacBook M5 Max has bandwidth up to 600GB/s, is 170GB/s for M6 the real limit? If so, wouldn't this slow down LLM processing?
Replies
Why are we comparing the M6 to the M5 Max?
What was the speed of the base M5?
If you want to do anything beyond getting a small single-core speed bump the M5 Pro is a better, more capable purchase. At least until the M7 comes out.
The M6 version is also cheaper than the M5 Max by a significant amount. I wouldn't be surprised if the M6 could keep up on LLM prefill (compute bound), but the memory bandwidth means that decode is going to be a lot slower, most likely?
It's a base M6 chip. The Pro doubles that, and the Max doubles again. And the Ultra doubles yet again. This is how the entire M series has always worked.
So the theoretical Max would be 680GB/s.
It's also absolutely wild that you apparently cannot spec one with more than 32GB, for any amount of money? Wow.
Not that I blame Apple here, for once. Seems like they're trying to cope with the limited quantity of RAM they can procure. And with the paltry bandwidth (170GB/s) of these CPUs they're a bad choice for anything but tiny local models anyway.
There are rumors that Apple is not releasing Pro/Max versions of the M6 with the intention of focusing these research efforts on the M7 instead.
Even then, 170GB/s is a bump up from 153GB/s of the M5. So we see a 11% uplift already.