logoalt Hacker News

aurareturnyesterday at 11:58 PM0 repliesview on HN

Prefill is gated by GPU compute, not CPU compute. A Epyc/Xeon CPU will never beat an M5 GPU in prefill. If you want something that can match M5 Ultra in prefill, you'd need to add beefy Nvidia GPU. However, the problem is that the GPU memory is separate from the 384GB system memory. And here lies that advantage of Apple Silicon. It's unified memory and accessible by the CPU and GPU.