alt Hacker NewsInteresting, I'm also comparing 3.8-flash-next (4-bit quant, llama for inference & pi as harness, on a 128gb strix halo machine) with opus 5.5 in claude code and while qwen is a good and fast model, it's nowhere near for coding, even haiku 5.5 feels better.
I could see it used as a sort of "sub-haiku" for easy stuff to leave more tokens for the bigger claude models, but I won't be cancelling my Claude sub any time soon.
Replies
Here's a stress test of sorts that I've had it working on the last couple of days: We have a C -> microcode compiler written in C (most of that was work done 'back' in the Opus 4.6/4.8 era). I gave it the docs for that compiler and asked it to implement it in OCaml (since it's not very common, but also is a good implementation language for compilers). It can use the first compiler as a golden to compare against. It's essentially: C language parser -> AST -> IR -> [optimization passes] -> microcode (+ some verilog code). It's basically at the point now where the original compiler has been. Not only does this generate microcode it also has to generate some Verilog code that loads and runs the microcode as well as run simulations in Verilator to confirm functionality. I asked it to come up with some new optimizations that the original compiler doesn't have and it came up with a couple that in some cases reduce microcode lines by up to 30% (less microcode in this machine means more performant). These are ideas that Claude didn't come up with when it was working on the original compiler back in the 4.6/4.8 era(and I recall prompting it several times asking it if there might be other optimizations we could perform).
So all in all, I put it at Opus 4.8 level, at least. And in some ways it's exceeding 4.8.
It's obviously nowhere near opus 5.5, but it's crossed the same threshold opus 4.5 did: it's good enough. If all model-as-a-service companies shut down tomorrow, I'd be happy with qwen 3.8 flash next q4 running in my strix halo