logoalt Hacker News

mountainrivertoday at 8:58 PM1 replyview on HN

Running LLMs on Macs is still terribly slow. They simply lack the optimizations other platforms have.

An RTX 6000 pro Blackwell is a pretty good card


Replies

speedgoosetoday at 9:04 PM

A M3 ultra mac Studio can run models that do not fit in similarly priced computers with multiple Nvidia GPUs. And it will use a lot less electricity while still having good enough performance. Except the pre-filing perfs that are quite poor on the M3.