alt Hacker NewsHow do you define the model buckets, and what happens when a session genuinely needs a model that isn't in the bucket the HMM picked?
How do you define the model buckets, and what happens when a session genuinely needs a model that isn't in the bucket the HMM picked?
You can think of buckets as models with similar capabilities. So for example Deepseek 4.1 Flash will not be in the same bucket as Astra.
The latter is an interesting question! In practice because the session continues, we can see it's going down the wrong path and escalate. Basically no decision we make when routing is entirely unsalvageable (but we do have a performance penalty for every incorrect decision we make so of course we try to avoid it).