logoalt Hacker News

kennywinkeryesterday at 5:43 PM2 repliesview on HN

It literally is usable now. A 5060 for $800 can run qwen3.8-27b 4bit at >40t/s, and the model beats opus 4.6 (max).


Replies

TomBombadildozeyesterday at 6:03 PM

Beats Opus 4.6 at what exactly? It certainly isn't code.

I use a combination of a Claude Max subscription and local inference, including qwen3.8-27b, 4bit. I have found qwen to be absolutely useless at anything but very specific, surgical code changes. In my experience, for anything even remotely nuanced, a frontier model is required.

show 2 replies
fshryesterday at 8:57 PM

> 5060

I think you're referring to a 5060Ti 16GB, yes?

32k context is easily done there. 64k can work with a more aggressive quant, but you lose a bit of speed.

show 1 reply