alt Hacker NewsI agree. Compaction sucks, so I made tools that let the LLM selectively delete (and recall if needed) chunks of its context. You might want to try context bonsai if you're routinely hitting the auto-compaction wall.
Replies
adyavanapalli • last Sunday at 2:11 PM
This is actually brilliant. I've seen other approaches where the context is selectively pruned, but never in a recoverable way.
P.S. Love the "bonsai" name btw!
➕ show 1 reply
znnajdla • last Sunday at 1:53 PM
I have a hunch that OpenAIs proprietary compaction endpoint actually does something similar on the backend.
➕ show 1 reply
I've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!