I use Llama.cpp and OpenAI-style API of course (who doesn't, unless you use Ollama perhaps?)

Ollama sucks, they dropped ROCm support for my card out of the blue, and the Vulkan replacement wasn't ready. The developer just shrugged and didn't care about the people affected so I moved to llama-server. Also, here's some other reasons to avoid ollama: https://sleepingrobots.com/dreams/stop-using-ollama/

And really, ollama are trying to sell their cloud platform and are always running behind llama.cpp in new features like the KV Cache quantisation.

So numbers, I don't really measure it from day to day, I'm not really interested in benchmarks at all. But I just did a quick test, llama 3.1 8b q8_0 does about 70tk/s (with Vulkan up from 45 when I was still running ROCm), and qwen 3.5 9b q8_0 does about 45, I think that includes its time spent thinking, not sure. Overall I'm happy with that.

Smaller sizes and quants are of course faster than this.

It's not terribly much faster than a Mac Studio, but it's one hell of a lot cheaper of course. And for the price of a Mac Studio you can get a lot faster hardware. The only thing where the studio excels is if you need large amounts of vram like 64GB and up and run the models to go with that (I run multiple small ones and spread them over multiple cards)