alt Hacker News10 grand for 256GB memory. Likely double that for 512GB, but won't be available or finalized until October. Thunderbolt 5 is highest bandwidth external IO available at 120Gb/s. 1.2TB/s claimed max internal memory bandwidth.
Not exactly "future proof" for >1T parameter models but good for targeting specific lower-parameter models, or if you can rely on pipeline parallelism and run a cluster.
Replies
That's in the same ballpark as two 128GB AI machines like the Asus GX10 or DGX Spark or Strix Halo. And, it seems very likely to perform better than either of those for inference. And, 256GB brings some pretty good models into play.
But, that doesn't make it a good deal. It just means the Apple tax doesn't apply when stacked up against AI machines and with memory prices being so out of whack. I'm still planning to wait until the RAMpocalypse ends before I buy any more hardware.
There is no "future proof" for >1T param models, there is no present or future where you can run a model that size on consumer hardware.
> 10 grand for 256GB memory.
A NVIDIA RTX 6000, 96 GB at 1.7 TB/s, is 13 grand.
This 256 GB at 1.2 TB/s Mac is extremely competitive, it will be sold out everywhere.
10 grand for 256GB new Ultra sounds too cheap in today’s crazy DRAM market, it feels too good to be true.
3 of those thunderbolt 5 ports, so you can do a fully connected 4 machine cluster topology.
It will always be wasteful to have a GPU sleeping next to you 99% of the time. Something like OpenRouter is the solution imho, for me at least. I realize that that some people care about privacy more though, and I respect that.
> Not exactly "future proof"
Computers are never "future proof".