Kimi K3 Is Competitive with Fable; Kimi K3 and Fable Is SoTA

If you haven’t been really running and testing these models yourself, they are all benchmaxxed. No matter how close they score to frontier on whatever metric, they always fall apart in real world tasks and their token efficiency is ridiculously bad. Fireworks has incredible incentive to make this claim in a headline, because Fireworks hosting K3 for you is pure profit for them, unlike when they host closed source models.

This benchmark is probably also self promotion of services. Fireworks happens to make a router. Using the router gets better performance.

https://docs.fireworks.ai/deployments/routers

Very interesting. They test Kimi K3 and Fable on a set of approx 1000 tasks grouped into 5 areas (SWE, Legal, etc).

They put a router model in front that predicts whether Kimi or Fable is going to give a better cost for a correct result. (They believe that ultimately such a router model should be continuously trained on your own workloads so it makes the best decisions for you).

Their router chose Kimi the majority of the time (72% in one category, all the way to 96% in another category), leading to cost savings in every category (from 1.5x to 50x depending).

When DeepSeek was released, it had an immidiate and significant impact on the US stock-market. Now when its becoming common knowledge that China is almost at parity with US SOTA models with good momentum, why is there no sentiment change on the market?

What is routing? How do they decide which is better? The only way to come up with a routing model for your workload is to send queries to all the models and then come up with a way to say which solution was better, often trying out multiple times for the same model + query to account for other statistical errors.

This makes you, an ai inference user an unwitting AI company with a non scalable product.

The biggest mental trap people have fallen for is the notion of “best” and, always using the frontier model. Instead you should just bite the bulet and choose the cheapest or the best. This routing dance is just tokenmaxxing in disguise.

Edit: Another thing concerning me is that the models themselves are not concrete behind their endpoints. Models can be arbitrarily dumbed down by reducing their inference resources. Once Anthro/OAI release a new great model, they are fully incentivised to dumb down their current models to drive traffic to the new shiny more expensive one. In fact they can do this for any reason. Once they switch things behind the API interface, how useful is your meticulously tuned router? Not much at all.

I will accept a 5% drop in benchmarks for a model that talks to me like a human.

Anthropic looks like Roman empire fast-farwarded, getting to the other side of the peak even before the IPO.

What's the data governance and privacy controls on using Kimi K3 if I subscribe to their coding plans? I want to migrate away from Anthropic

I wonder why there's so much resistance here against chinese models. Sure at my employer claude is used, but at home? I am just happy using my z.ai sub for 20x i got in September last year, coupled with the 39 dollar tier of kimi. I use them in pi, with a collection of extensions i curated myself for this iterationm of models, and a couple glue extensions we have made.

At home i feel way more productive, the speed of my queries are second to none with kimi 2.6, and handing review over to glm5.2 means i can juggle the small models in my brainstorm to commit workflow.

Genuine question: can these posts be paid to hype the open source models? If yes, what would be the purpose?

On my work tasks, FastAPI Python and Springboot Java on a modern SaaS product, the only open model that can do tasks well and efficiently is Qwen3.7-Max.

In all my experiments, both GLM-5.2 and Kimi are busy grepping around the codebase for ALMOST 70-80K tokens before writing anything and when they do it typically breaks the code… it feels to me that these models are good but only when you write out a super detailed spec of the task just like it was done a year ago… Qwen3.7 just… does it

Kind of cool to see.. that said, there's more to a tooling experience than benchmarks and specific models. Cursor, Claude Code, Codex, etc. add to the mix. Things like Open-Router and backend options make it easy enough to test.

The tools, libraries and languages you are using can also dramatically affect results. Even on state of the art models, I find, for example, the output of SQL for complex interactions, or C# for that matter to be sub-par, where I find Rust results to be pretty great, with JS/TS falling in between.

At the best, it can feel amazing and productive, at worst, time consuming and annoying that you could have done it faster yourself. YMMV in real world use.

Note: I'm a proponent of human in the loop gatekeeper/reviewer usage of AI, and I'm not able to even consider Chinese models for my own use, and not able to use anything at my day job.

Is there something specifically with Kimi that's better here? As far as I know Kimi pricing is about the same as Sonnet 5 -- what happens if you use that model and Fable instead? Or Grok 4.5 which is even cheaper?

I love the Chinese models.

I use DeepSeek exclusively and now Kimi K3 offers a great planning assistant for more advanced coding tasks.

DeepSeek v4 Flash is extremely fast and is able to handle pretty much anything I've thrown at it (I use mostly Rust, PSQL, Angular and Terraform).

I self host Bifrost as my LLM gateway, though I wish LLM vendors would do monthly/daily automatic billing (like VPS providers do) rather than prepaid + auto-top up.

It's annoying maintaining a non-refundable minimum balance across vendors, I would rather be billed for my exact usage.

OpenRouter helps, but I don't really like it as a service and not a fan of the mark up.

Seems that “oracle routing” is a term the authors invented. Also sounds like they’re sending requests to all routes but assuming the readers request will route to the “best” model. The closest thing to what they’re describing is semantic routing using a NN search in a vector DB to make a routing decision, but the efficacy of this approach isn’t a slam dunk.

Hmmm, a company that hosts open models is telling us how good open models are...

That's incredible. Hope the chinese keep it up!

Anti-China: K3 is propaganda and benchmaxxed, no matter how anthropic and openai reactor for these, it just a smoke signal.

Pro-China: K3 is good choice for better and affordable choice to smash down the Big three ruling.

SoTA means "State of the art". I wish it didn't take me 5 minutes to figure out what SoTA stands for.

Anyone have routing harnesses like this describes with Claude Code? Or other good routing platform recommendations?

(yes, I know this article is about an oracle router)

As always, benchmarks rarely paint the whole picture. It also seems like this article is somewhat biased, eg when Fable and Kimi are close but Fable wins it’s “dead heat”, but when Kimi wins it’s “Kimi wins”. GPT 5.6 seems to be missing as well.

I am really eager to give Kimi K3 a try, but I’ll reserve my judgement until I’ve worked with it for at least a few days.

Nothing makes me happier than seeing AI becoming a commodity rather than the winner takes all bullshit Anthropic and OpenAI have been chasing with hundreds of billions of investor money

What's the reason the current SOTA wasn't tested? GPT-5.6-Sol-Max is the actual SOTA.

Interesting. So the latest in the technology now is this model routing thing. Cursor estimated Composer + Fable works much better than Fable alone. And here K3 + Fable is supposedly better. Interesting.

These routers can be interesting on a company level to optimize for cost and quality, but for individuals who mostly work on the same tasks, i doubt it. You want to leverage the cache and switching models within a task seems not cost effective to me.

Mythos/Fable was the state of the art back in March, if not earlier.

A third the cost, open source, and won't refuse every other request because of some vague possible connection to cybersecurity concerns.

If only you could run K3 locally that would be the magic bullet to make it a true magic bullet!

for people in mainland china the only option now is Kimi K3+GLM 5.2 for their day to day work as Fable is blocked. for me i use Fable + codex when i was out of mainland china, it is good when you have options wherever you are in this world

I enjoy all the fun of these new models as much as the next guy but I truly don’t see a circumstance in the near future where my $200 a month with the frontier labs doesn’t get me more than enough consumption of what I need. Local models, chinese models, etc are all very fun weekend projects to tinker with but until something changes (entirely possible!) with how much you get with one of the subscriptions I just don’t see why I would move. What am I missing? Is it simply that a subscription is no good for production use cases? I kinda feel the same way with choice of coding harness, openrouter, etc. why would I use anything other than frontier if I don’t have to pay any more pretty much no matter how much I use? pls tell me if I am holding this wrong haha

Kimi K3 showing competitive performance with Fable while both sitting at the SoTA level on fireworks.ai is a huge milestone. Really interesting to see how the landscape is shifting here.

The article is about the best you could theoretically do with a perfect router. The takeaway is that trying to build a good router is worth doing. But it's unlikely to be a perfect router.

I have several thoughts about this, which I'll just iterate:

1) US export bans have made it so that Chinese companies have to compete using less-than-state-of-the-art hardware. This has forced Chinese companies to build more cost efficient models. Whereas, US companies have moreso tried to be state of the art by spending more money than anyone else on state-of-the-art hardware.

2) Xi Jinping has called for more open AI models (not to be confused with the closed models of OpenAI), and I'm happy to see a powerful world leader advocating for open-weight AI models. Whereas, the US seems likely to just ban models.

3) My impression is that, if China surpasses the US in AI development, there will basically be nothing that the US does better than the rest of the world--except for military spending--we spend a lot, but we don't necessarily spend well (something something Iran). I mean, if the US is no longer a tech leader in the world, like... what are we a leader at? Manufacturing? Healthcare? LOL. Are we a leader in any industry or by any metric? I wonder if China is attempting to remove the last jewel in the USA's crown with these AI releases.

4) It must be refreshing for companies to have access to a new model that isn't going to get pulled because the government bans it 2 days after release. And it's open-weight so it wont go away--amazing--what a shift in the market.

5) If it becomes clear that open-weight models are the future of AI, will that pop a huge bubble in the US economy? Maybe. But, on the other hand, these companies aren't just training AIs, they are also building data centers which will remain valuable no matter what happens.

If I had an oracle, why would I need an LLM?

the per-token comparison keeps missing that k3 spends way more tokens per task. if it burns 3x tokens to reach the same result as fable, cheap per-token stops mattering

Openrouter also features routing. Routing is indeed an option if you don’t need consistent behavior and allow switching models.

cluade model, claude opus 4.6,7 perform very well.and stable, understandable ,i miss that in kimi, its slow, doesnt gives feels like claude

Was this an out of sample test of the router or was it trained on these specific use cases/eval suites?

Your account <...> request reached organization TPD rate limit

Shifted to K3 and it is like a fresh air. While Fable and Sol very good at _generating_ code i even cannot force sol to just read all relevant source files. As result it reinvent existing things or assumes too much about internals of other, which lead to incorrect uses. Even with hard planing mode GPT burned 33% of week tokens for 3 hours producing no result and even cannot find root cause, but K3 fixed it in a minutes. Hilarious that while software not started with empty cache Sol handcrafted empty cache to let it start. Having full procedure right in the MEMORY.md. K3 found this, and downloaded cache correctly. But while using gen5 models i always feeling myself ignored. Any commands, steering anything - just ignoring. At first i added lots of hooks, no "?", expect in rust code, bash hook ban for find|grep|tail, with notice to use ltsp but then it started to ignore strategically. Also whole "thinking" thing is hidden from claude. regenerated thinking summary is incomplete and not useful. While being very verbose kimi k3 doing good job providing whole train of thoughts. So looks like claude degradation started from 4.6 comes to a logical end. Maybe i missing something and changing existing code beyond bug fixes is not a way to go. But there is no currently stable way to generate code on hier of specs and lean models i'd like to.

For model routers, do they have to retrain the routing model every time a new LLM is released?

I'm skeptical. According to arena.ai, Fable 5 dominates almost every category: https://arena.ai/leaderboard Kimi K3 has an edge in WebDev but struggles to reach top 10 in many other categories.

The irony is the Chinese are being very democratic with their models, while the USA tries to do central control.

Glad to see centralized control fail on the grandest scale. Maybe we can learn a thing or two.

The Kimi K3 felt pretty good when I tried it out.

we put a router model in front of two other models so the router can decide which model is better at deciding things. next we'll need a router for the router and eventually the entire internet is just routers routing routers to other routers

what are my options if i want to use a router like this ? who provides one ?

They forgot to compare and incorporate GPT-5.6-Sol.

It is not SOTA. Give me a break. Sure, run it on Cerebras to get speed but that’s pretty much its advantage.

Why SoTA (uppercase “T”) instead of SotA (lowercase “T”) ?

“State of [T]he Art” versus “State of [t]he Art”.

If not SotA then at least SOTA, which is more accurate.