https://github.com/NVIDIA-NeMo/Switchyard#routing-strategies

Looks like your great question doesn’t have an answer, but looking at the routing strategies things get even more confused, since the proposed ones tend to rely on extra llm calls to determine which model to pick.

The nice thing is that it makes sense for specific setups, less conversation oriented.

As an example, you need to classify batches of data, and have many fine tuned models. Or you need to do speed to text and need to pick which whisper to use.

You can write your own strategy, in that case an harness with subagents would be able to leverage this, picking the right model and then keeping its session sticky, but overall the lack of concern for caching points towards use cases where you do not gain much from it.