alt Hacker NewsRunning LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.
An RTX 6000 pro Blackwell is a pretty good card
Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.
An RTX 6000 pro Blackwell is a pretty good card
A M3 ultra mac Studio can run models that do not fit in similarly priced computers with multiple Nvidia GPUs. And it will use a lot less electricity while still having good enough performance. Except the pre-filing perfs that are quite poor on the M3.