alt Hacker NewsCould you elaborate on your experience with local models on your card? I've been thinking of upgrading to 9070 XT, and was thinking the 16GB would be okay-ish to at least run something usable locally, no?
Replies
I’m the parent of this thread, the person with the with the RX 9070.
My understanding would be that if you’re interested in this sort of card for AI that you should go with the AI PRO R9700, which is basically the professional version of the RX 9070XT but with 32GB of memory.
It’s significantly more money but not crazy like a 5090.
I just happen to have the 9070XT primarily for gaming purposes.
I’m not quite sure how to describe my experience using it other than “rudimentary,” and a lot of that is on me for not really understanding the best way to set it up.
If you have been using cloud hosted models, you will be severely disappointed with what you’d be able to run on 16GB VRAM. You will spend most of your time fighting with the model to fix its mistakes.
Usable certainly. But my impression is that useful models still need a bit more than 16GB. Something like Qwen 3.8 27B is useful but squeezing it into 16GB requires fairly aggressive quantisation which will make it unreliable (e.g it'll get stuck in loops) and won't leave enough space for a long context (which qwen 3.8 really likes)