logoalt Hacker News

Bolwinyesterday at 10:20 PM3 repliesview on HN

Cache hit % on openrouter is not a good metric, it's mainly driven by openrouter's own provider juggling than the providers themselves


Replies

andaiyesterday at 10:50 PM

OpenRouter randomizes which provider gets your request by default right? I think you have to pass a specific provider in the request to prevent that. (Or set up a preset or something.)

This behavior makes it so you don't benefit much from the caching, unless you pin it to a single provider.

show 2 replies
dakolliyesterday at 10:30 PM

This is not true, there isn't even a way to see a cache hit % model for a specific model, that wouldn't make any sense. You are confusing what I'm saying with cache cost, that has nothing to do with effective cache hit %. I'm talking about when you click on a specific provider for a specific model, you can scroll down on the view and see their cache hit % for that model [0].

These cache Hit % are accurate, I've done a ton of testing of this myself. The cache hit % is one of the most important metrics as far as estimating cost. There are many providers with cheap cache reads, but have an effective cache hit % of 30%, making their cheaper cache pricing meaningless compared to another provider who charges more but has a 85% cache hit percentage.

[0]: https://openrouter.ai/deepseek/deepseek-v4-flash-0731?endpoi...

scroll down on the provider/model card and you'll see a field called cache hit %, its different for every provider/model.

I don't use routing on openrouter, I strictly use models with a single provider and no fallback, at least for use with harnesses its pretty dumb to route requests to multiple providers you are busting your cache every other request and increasing costs by 20-50%.

show 1 reply
ralusekyesterday at 10:54 PM

> Cache hit %

I thought you had to actively manage caches, do you not?