alt Hacker NewsOpenAI does cache the most frequent questions like this.
Replies
dimitri-vs • yesterday at 9:22 PM
Doubtful. Considering features like memory, response style, custom system instructions, etc. can heavy alter a users output.
OpenAI does cache the most frequent questions like this.
Doubtful. Considering features like memory, response style, custom system instructions, etc. can heavy alter a users output.
There is no way to cache the answer to questions that differ by even one token reliably without introducing potential problems.