alt Hacker NewsThis model seems to be pretty benchmaxed. In actual use, it doesn't survive the comparison to models in the benchmark.
This model was released as a "stealth model" in OpenCode Go for a while as "Space Bunny" (they mention this in the blog). So people have been using without knowing what the model was exactly, but it failed to impress anyone as far as I can tell. This is despite the model being served for free (with some asterisks). GLM 5.3 Flash and DeepSeek V4.1 Flash, even before last week's update, seemed to be much preferred. Stealth releases happen often with OpenCode subscriptions, and GLM 5.3 Flash (stealth name "Ox Alpha") impressed a lot for example.
The perverse incentives around benchmarks are now so bad that benchmark scores are effectively meaningless. Models can score highly and still be crap to use. It doesn't matter whether it's Minimax or Google, or anyone really.
Replies
I don’t get the endgame. If your model underperforms in real life usage, the backlash comes quickly. I guess I’m asking what exactly the perverse incentive is? A day of press at most?
> The perverse incentives around benchmarks
I am not sure. The people who actually care enough to look at benchmarks will run evals, and find out quickly. If you post silly benchmark scores too often, you will simply be ignored. You are paying with your reputation.
This is my experience with all MiniMax models: They're good on paper but fail to live up to expectations in practice. IMO, at no point has any MiniMax model been the superior choice over others, all things considered.
"When a measure becomes a target, it ceases to be a good measure." https://en.wikipedia.org/wiki/Goodhart%27s_law
I used Space Bunny from some providers that offered it for free and it was a very much a "you get what you pay for" experience. If the release is comparable, I'll be sticking with Deepseek v4 for sure.