alt Hacker NewsIf your benchmark puts DS4.1-Flash at the same tier as K3 or K3.1, I'm sorry to say your benchmark is broken. It really isn't, as much as I wish it was.
Replies
haizhung • today at 7:17 AM
I tried both of them for agentic coding and didn’t feel like K3 was adding so much benefit to justify its almost 10x price (on the modem provider I’m on).
In fact, I barely see a difference between the results at all. The models being tied smells correct to me.
I didn’t try 3.1 yet, though.
We allow early voting. This is against what most arenas do, but is on purpose. If the model is faster to do simple stuff, it should be chosen. In this sense, DS4.1-Flash is better on many tasks.
You can remove early voting from leaderboard and they are no longer the same.