logoalt Hacker News

wtallistoday at 4:43 AM0 repliesview on HN

Sure, if you want to make a comparison where the price tags aren't the same order of magnitude, then a recent server is obviously going to be powerful. But since the baseline of this comparison is a laptop and several Thunderbolt SSDs, the kind of servers or workstations with 1.5–2TB of RAM that you can reasonably compare against would have to be the really old ones, barely new enough to support that much total RAM.

And despite the theoretically high memory bandwidth of recent EPYC CPUs, approximately nobody who can afford one is doing LLM inference on them.