alt Hacker NewsI love OpenRouter, long time user. Stripe will hopefully be a good custodian.
I just want to point out some features of OpenRouter that make it more than just a model selection and routing endpoint and that I find incredibly useful:
0/ Default routing is to the cheapest provider, but they're usually not the most performant. I'd guess 99% of OpenRouter integrations never tweak the default routing. Here you can setup cheapest with performance minimums:
https://openrouter.ai/docs/guides/routing/provider-selection...
You can also stack model selection in priority
1/ Using broadcast you can push all your analytics to clickhouse / s3 / snowflake and a bunch of other compatible destinations. Setup a clickhouse server ($5 VPS[0]) and send all your traces to it:
https://openrouter.ai/docs/guides/features/broadcast
customise your own observability in your dashboards from there. Superwin
2/ Model router is also a natural home for llm security - OpenRouter has the beginnings of prompt injection detection:
https://openrouter.ai/docs/guides/features/guardrails/prompt...
there is also PII detection. This will show up in observability as rejections/blocks etc.
There are so many model routing solutions (same with observability, security etc.) but they're all 80% solutions - OpenRouter really rounds out with well implemented features that you need when deploying models at any scale and I gladly pay the toll.
[0] not sure if these exist any more but clickhouse is resource efficient
Replies
I have been using openrouter only superficially. I didn’t know about the broacasting feature. Thanks for that.
They might be popular for indie developers, but nobody doing serious AI or in a corporate environment is using them.
And if they are, their compliance team is about to strike them down. The VCs forcing this acquisition do know this.
- Why would you add a penalty of 50 ms at a minimum? And that is not the p95... Just run LiteLLM in house and you dont even really need that.
- Their capacity pools are shared across the whole user base, a massive batch processing by another of their customers and think what that means for your response time...
- So instead of negotiating corporate rates with OpenAI or Anthropic, you would be using an intermediary and topping up the corporate credit card... for a 5% markdown ? Really?
- They can see all your critical corporate data on the in and out
- They present some pink SOC 2 promises but then wash their hands and defer to you and the providers. Its just the Bolt and Uber model the drivers are not our employees....
- They are a man in the middle proxy that is a massive security liability for your corporation
- They have no support for private cloud points
- No geofencing guarantees
- No intellectual property legal indemnification unlike what AWS or Microsoft or Google offers
- Its a provider roulette inconsistent with hosts providing different quantization levels causing random shifts in response quality
- Support via a Discord server...
The only reason they were not shutdown yet by Anthropic or OpenAI is because they have the same VCs, as those two. That would mean said VCs investment would go to zero. Oh and those are the same VCs that own Stripe...
Just setup a private proxy tier using something like LiteLLM, even if you really dont need it. Just code your enterprise apps to have have fallback loops on the core hyperscaler providers like AWS Bedrock or Azure Foundry...
> features of OpenRouter that make it more than just a model selection and routing endpoint
All those features show it's just a model selector and router.
> Model router is also a natural home for llm security - OpenRouter has the beginnings of prompt injection detection
Maybe I'm biased from the perspective of a "harness provider", but I think Model routers often have too little context to act as well-informed prompt injection prevention. e.g. it lacks context of where which part of the message(s) originates from and sanitization/safeguards were already performed on the application layer.
Something like OpenRouter's "flag" mode is fine, but usage of auto-redact or auto-block should really only be used if there is significant risk exposure through your harness or otherwise they are a constant source of bugs.