logoalt Hacker News

tym0yesterday at 8:24 PM1 replyview on HN

That's not my understanding of how most agents work. This is what a chain of request/response looks like:

  Your Prompt 1: Prompt Content 1 -> cache-1
  LLM Response 1: <Thinking>Thinking Content 1</Thinking> Response Content 1
  Your Prompt 2 (client side): prompt-1 + response-without-thinking-1 + Prompt Content 2
  Your Prompt 2 (server side): cache-1  + response-without-thinking-1 + Prompt Content 2 -> cache-2
  LLM Response 2: <Thinking>Thinking Content 2</Thinking> Response Content 2
  Etc...
So reasoning gets dropped from context and you still get cache from the accumulating requests.

Edit:

I've realised I was incorrect, the thinking doesn't get passed back and forth but the latent snapshot does which result in using memory just the same.


Replies

mike_hearntoday at 7:52 AM

Modern protocols loop back the reasoning tokens in raw textual form via an encrypted parameter. You can't see them (modulo the recent attack), but you do resubmit them.

show 1 reply