Qwen3.8-flash-next starts being "functional" for me above 4 bits weights and int8 kv. Preferably 4.5 bits.

Likewise those 20t/s they talk about are often at very short prompts and use very favourable MTP setup. The real number (on prose etc, where mtp is less useful) is in the teens.

Even 20t/s are below my usability requirement for an interactive model, but if that is all you have it is brilliant.

However if you want to use it for work similar to what people use 90eur/month claude code subscription for you need a mashine like what I have. Ryzen 7950x 192GB RAM and 6x rtx3090.

Cost today around $20k.

If I was building it today I might go for xeon ddr3 platform for cheaper ram, higher total ram bandwidth and more pcie lanes, but i use what I already have.

So I have this and I still pay for 90eur/month claude code subscriptions and $30-50 a month for open source cloud models. Why? Because if I wanted to do similar amount of work I do with claude code I'd need at least 2 if not 3 such mashines and this being winter (crappy solar output) my electricity bill would be over $1000 a month.

I once calculated my claude code use would be worth thousands of $ per week if I paid api prices. Simply speaking Anthropic subsidises these subscriptions a lot. Why would I not use it?

Of course all the disadvantages of cloud and opaque hosting have hit me many times, but free money is free money.

Because retail models now dominate the price-performance frontier by a huge margin. An open-weight model wins at just 3% of price points (down from 41% just a few months ago).

https://village.dev/blog/closed-ai-models-now-dominate-the-p...

Comparing the sort of models that would run on that to frontier models like Opus 5.5 or Astra is silly

It is weirdly inverted from the previously standard economics where hardware would keep getting cheaper but those efficiency gains would keep at least partially eaten by software.

> I don't see how anyone justifies a claude subscription.

Certainly I see the benefits in terms of privacy and security. But if you're trying to justify local models on economic grounds, you also need to factor in electricity costs.

(Also, people will say that "less than a year ago" represents an enormous capability gap. But people were already ranting and raving about how capable the models are and how cooked the industry is, considerably longer than "less than a year ago"; so.)

I think this is why some CEOs panic and want to kill consumer hardware as soon as possible.

Sad part is that hardware manufacturers are fully on board with this because for them the datacenter deals are quite lucrative. In the long run, this will probably bite them in the butt though.

And I think they suffer bad management if it cannot serve solutions to both problems.

> I don't see how anyone justifies a claude subscription.

I have hardware to run Qwen3.8 Flash Next (the model you used in your example) much faster and closer to full quality.

I do not understand how anyone thinks it replaces a $20 Claude subscription in performance or quality. At best I some times have Opus give well defined tasks to Qwen3.8, but I have to be very selective about what is handed over. I also use a lot of Opus tokens reviewing and fixing it.

For real work (not fun projects or experiments) I won’t even bring the Qwen models in to play at all. The gulf between Opus and Qwen3.8-Flash-Next is that big.

So I hope everyone takes comments like this with a huge grain of salt and healthy dose of skepticism. If your work has a high tolerance for slop and you can wait a very long time, the local LLMs can do a lot.

However claims about replacing a Claude subscription with Qwen3.8-Flash-Next? The only way this makes sense is if someone is choosing on ideologies like privacy or avoiding big corporations first, with quality a distant second.

I get downvoted every time I say this, but it’s objectively true. The open models that can be run on consumer hardware are far behind Opus, and quantizing them to Q3 takes them even further behind than the benchmarks show.

Also 100 tok/s prompt processing and 20 tok/sec is painfully slow for real work. The slowness is multiplied by how much extra work the less intelligent models have to do to get tasks done.

While not directly comparable technologically the level of anti-self-hosting animus in this thread reminds me of what happens when someone shows it is perfectly possible and in many ways preferable to self-host mail services. I say to each his own, I do self-host all my services - no inference 'cause I don't see the need yet but if I ever do I will - and this has served me well for more than 30 years now. Others shy away from everything self-hosted, that's their choice and I assume they have their reasons which seem just as valid to them as mine are to me.

What I don't see mentioned by any of the outsourcing advocates here is the value of keeping your data out of the claws (pun intended) of commercial and governmental entities. It is hard to put a price tag on that but the difference between handling your own data and handing it over to those who specialise in mining as much information from it as they can is staggering no matter whether it is the family photo album, your startup's proposed business plans or the engineering department's design database.

I run my own local models and there’s no world where “normies” can do this.

In fact, I just switched from a 96 GB ram laptop to a 32 GB ram laptop specifically so I can have GPU acceleration because 20 toks can’t replace the 6000toks or whatever astra does

Can you give example of hardware config ?

E5 doesn't support AVX-512 which is quite important

> I don't see how anyone justifies a claude subscription.

I want more than "functional"

Cancelled my x20 sub after the most recent usage reduction, will probably cancel my openai sub at month end as well... it really makes no sense anymore, they need to increase usage by like 10x to make it worth it.

Of course not. I want to be automated out of a job after paying for expensive hardware to run a model that doesn't work.

The whole AI world is run by sociopaths that want to make human brains economically uncompetitive comparedto computers. If you think running local models will save you, you're either very pessimistic about the future of AI, or you haven't thought it through.