logoalt Hacker News

fc417fc802today at 12:46 AM1 replyview on HN

> This behavior makes it so you don't benefit much from the caching

I don't believe this is correct? AFAIK once it routes you to a provider for a given conversation that choice is sticky unless you hit technical difficulties. (It's more complicated than that, they recently added named routing strategies that you can append to the model name.)

IMO the relevant metric is cache TTL which isn't typically published AFAIK.


Replies

RussianCowtoday at 6:41 AM

This is very much NOT my experience in practice, even though it's how I would expect it to work. OpenRouter will happily bounce you between several providers (none of which have downtime) even within the same session. Requesting specific providers is the only way I've been able to hit a cache rate above 90%.