It's because the cache comes from the prefix. `f(f(f(A,B),C),D)` is quite different from `f(f(A,B),D)`.
Yes, that's my understanding- though I realize that OpenAI has made changes to what you can do with caching over the API recently, so you may be able to do some fancy stuff to get around this.
alt Hacker News
Yes, that's my understanding- though I realize that OpenAI has made changes to what you can do with caching over the API recently, so you may be able to do some fancy stuff to get around this.