I really hope the future is local models, because the idea that all companies will need to buy tokens from a handfull of tech companies to run their businesses or make their products is awful.

Interesting, I'm also comparing 3.8-flash-next (4-bit quant, llama for inference & pi as harness, on a 128gb strix halo machine) with opus 5.5 in claude code and while qwen is a good and fast model, it's nowhere near for coding, even haiku 5.5 feels better.

I could see it used as a sort of "sub-haiku" for easy stuff to leave more tokens for the bigger claude models, but I won't be cancelling my Claude sub any time soon.

I used Ollama with Qwen 3.8 27b on my MBP for a flight to try some experimentation and found that it was shockingly good at one of the more complex PDF processing tasks.

Like seat-of-the-pants good as Opus 5.5 but a bit slower and required just a bit more prompting to get what I wanted (which is fine for an app pipeline).

I am in the same boat. Sun servers were still the best that money could buy when they lost to the Linux revolution. Linux was _good enough_ and destroyed their margins.

That jives with my own datapoints. Of a circle of around 20 very active devs Anthropic went from being the must-have tool to on the chopping block.

Around 50% of that group says they plan to cancel their Claude subscriptions now that their local or open model setup is performing “good enough.”

These companies have no moat at all.

it feels like this is a bit similar to the famous dropbox comment.

I guess maybe another anecdotal comment, but I'm an engineer myself, have a home server for simple stuff, and I pay $200 a month to chatGPT for a pretty simple usage and really don't see myself go and run my own open source model, especially at the speed at which they evolve and change.

99% of the world will not setup their own open source model, just like 99% of the world is not setting up their own servier with ssh copy whatever the dropbox comment was saying.

I’m migrating that way as well, but I’m still struggling in creating a nicely tuned harness to achieve what I really want:

Opus for planning and the very difficult parts, Qwen for the mundane ones. It still involves a lot of manual model switching (but opencode and Pi make that relatively easy).

Curious how much ram you need to run this at acceptable performance

I've got a prototype cluster of 5 devices running Qwen3.8-27B assigned different roles and sharing outputs between each other. Beats Claude Opus MAX in SWE and Terminal benchmarks.

How much does such a box cost?

This. The problem is now the same monopoly cartel problem.

Strix halo is double the cost because of...

And 4 is about to drop.

There's basically no point in paying for these subscriptions anymore. Sure they are marginally better at some things. But they are also marginally worse at others and the implications suck. I stopped all my subscriptions and keep like 20 bucks in open router just in case. Never need it, but it's there...

The non-anecdotal part is to me as interesting as the anecdotal part. The price of RAM has gone up a great deal which is going to make it harder for others to get this sweet setup that you have.

I see lots of people talking about running Qwen as a local model just curious what's the difference with say something like local deepseek?

How does it compare to haiku?

What inference engine do you use ?

Do you ever worry whether these open source models have been trained to embed backdoors or leak secrets in very specific circumstances at some point in the future? The kind of thing that's hard to know ahead of time and even detect after it's happened?

I feel like the industry hasn't solved that alignment problem yet... We have no way to verify the entire models landscape of intent.

Paid models have an incentive not to do that because if the market discovers it there is a huge economy penalty, but the free open source models don't have that disincentive.