alt Hacker NewsGemma is amazing with tools for anything that is not crazy complex. I think a lot of people have a wrong perception of it because Google's new prompt format broke implementations like llama.cpp and it took quite a while to get everything sorted. But even the tiny variants running on edge devices are surprisingly capable when used right.
The frontier will probably keep moving for a while, but it will be increasingly disconnected from normal human use. In the future, if you're not trying to solve a research level math problem, you'll probably do it locally and fully privately. Which also means the payday when they will fundamentally no longer be able to reach a billion users with frontier models will come soon for the labs. Even if they do get their IPO out, it will probably crash and burn at current valuations.
Do you guys actually work with these models?
I have to use GPT 5.4 Mini at work. It benchmarks higher than that Gemma 4 model.
In my experience it's next to useless. It cannot even move 20 existing lines of code from A to B without breaking them half of the time.
If you tell it to look something up in your dependencies, it's 50/50 on whether the answer is correct, incorrect, or it simply didn't perform the search at all.
I find it next to useless, and I'm mostly better off doing the work manually.
It's a night and day difference to even Sonnet, not to mention the SOTA.