Could you check how many tokens the different models spent to do the task? With all the comments about Kimi K3 Not being token efficient I'm curious if your test confirms that
Correction on my earlier post, Kimi was through Pi, not Kimi Code. For Qwen I used Qwen Code and for Fable I used Claude Code.
Not sure wtf is going on with the Fable stats (a lot tokens, virtuall all of them were cached - I guess heavy system prompt?) but both claude code stats and ccusage tool output match.
For the web app task I mentioned:
* Kimi K3: 9532k input (9172k cached), 114k output - cost $5.5
* Qwen 3.8 Max: 18020k input (17836k cached), 114k output - cost $6.3
* Fable: ~14m input (all cached??), 196k output - cost $30
Correction on my earlier post, Kimi was through Pi, not Kimi Code. For Qwen I used Qwen Code and for Fable I used Claude Code.
Not sure wtf is going on with the Fable stats (a lot tokens, virtuall all of them were cached - I guess heavy system prompt?) but both claude code stats and ccusage tool output match.