alt Hacker NewsYour question is worded kind of confusingly, but all caching is handled on the inference layer, and by all major providers. In short, caching should work as long as you are sending requests to the same model and provider.
Your question is worded kind of confusingly, but all caching is handled on the inference layer, and by all major providers. In short, caching should work as long as you are sending requests to the same model and provider.
Apologies. I guess my main question is just is there a benefit to routing directly to a provider through its own harness vs just sending chat requests to those providers using OpenRouter or similar as a proxy?
As per your response it sounds like at least caching would happen for any provider regardless of the request's origin.