alt Hacker NewsCan't really run it as well, though. My "mini PC" is an M4 Max with 128GB of unified memory and the memory bandwidth is still sorely lacking for inference (although it's far better than any non-unified consumer architecture!).
Replies
bahmboo • today at 2:52 PM
To be fair it's "only" half the throughput of a 4090 and a third of an RTX 6000. Significant but not an order of magnitude.
➕ show 2 replies
Yeah this is such a funny thing going around. Try to run or train a small/medium sized model on a MacBook. It doesn’t go very good compared to a dedicated gpu
This is likely the right path in the future but it isn’t there yet today