logoalt Hacker News

dofm • today at 1:02 AM • 1 reply • view on HN

The RTX 5090 is the ideal, but it's not the only 32GB GPU that can offer workable performance; the AMD R9700 is catching up.

You can surely build an LLM server with an R9700 and 32GB of mainboard RAM for less than the 64GB M5 Max Apple Studio, which is probably the next most competitive model up. It'll be better than a Strix Halo but less flexible.

Replies

the__alchemist • today at 1:24 AM

Of interest:

  - 4090: Going for $7500
  - 64GB M5 Max: Starts at $3,800
  - R9700: $2700
Compared to $20 or $100 per month. My thought process reading this thread:

Baseline of $20 or $100/month Claude or OpenAI subscriptions. Alternative posed of running models locally. A guy says he did it and results are lackluster on a 12GB video card. A number of people post that that card is insufficient; use 32GB+ cards. I look up prices of 32GB cards, and they are all very expensive. So from a practical perspective, people are suggesting you spend thousands of dollars on a video card to get results that are almost as good as the subscription plan they are on. It doesn't add up, unless one of my assumptions is incorrect.

➕ show 2 replies