alt Hacker NewsI've not used Open Router but I'm wondering what are the reasons to route LLM queries at large scale?
Isn't the trend increasingly to let an orshestrator agent decide which model to use in terms of problem difficulty vs. size and thinking budget?
Once you fix those parameters there's little to optimize over, no? I mean you want to find the cheapest one across providers but that's not a recurring decision.
Replies
Mashimo • today at 7:59 AM
> reasons to route LLM queries at large scale?
Convenience. Paying only one provider, but get access to different models hosted around the world. Need a model for a project where latency matters? EU hosted and ZDR for another customer? Cheapest for personal use?
You get everything under one roof.
I suspect only a very small subset users are sophisticated/interested enough to do that for themselves on a regular basis. More will likely hop between agents based on cost and the perceived nature of the task, without hard data (or an orchestrator model) to support that decision.
In the second case OpenRouter is very useful; in the first case, maybe it’s an area they move into in time - providing and owning/optimising the orchestrator?