logoalt Hacker News

embedding-shapetoday at 2:52 PM2 repliesview on HN

Say you put the current time down to the second in the system prompt, which is the message that goes in front of the entire conversation, then basically nothing will be cached, every agent turn needs to ingest the entire session over and over. Contrast to not doing that, and the backend can leverage caching all the way up to the latest message, as nothing until then changed.


Replies

esperenttoday at 3:13 PM

Surely other agent CLIs are not dumb enough to invalidate cache on every turn over something so obvious?

show 3 replies
theanonymousonetoday at 4:20 PM

Yes, of course you can destroy it. But how far can you "improve", beyond decent "common sense" behaviour.