alt Hacker NewsI have memory disabled in all my Chat UIs (even though it tends to creep in, looking at you ChatGPT). It’s very helpful until it scales with time, at which point it becomes useless due to staleness or mis-application across contexts.
That being said, in coding over a longer time horizon, having the agent continually re-derive decisions/laws/facts/etc from your code is wasteful of tokens and time, and if your code doesn’t consistently apply them you can’t know the agent will make the correct choices.
You need memory of these important facts to avoid this expense or potential incorrectness. Memory does not itself scale though, without maintenance and pruning, and that has its own impacts on cost and correctness like the Chat memory.
“Damned if you do, damned if you don’t” at least until the agent can itself maintain its memory accurately - or some other non-human effort can achieve that.
I like a lightweight ADR system.
e.g. docs/decisions/README.md (index with a blurb about each decision), docs/decisions/01-some-lesson.md (some architectural decision/pattern that you or the agents discovered).
ADR files have important sections like "rejected solutions" and "acceptable risks", and they're live files that can be refined and pivoted over time or retired to docs/decisions/archive/.
It's also nice to give each top-level bullet point some stable ID like "R1" for rejected solution #1, I3 for invariant #3. Agents use this stuff intelligently all the time like "This could be a time to reconsider D4/R2" = ADR #4, rejected solution #2.
The essential part being that your system ratchets into increasingly better decisions and invariants over time, and there's a place to actually put this stuff.
It's essential for automating high-quality software and something we couldn't be arsed to do much less update before AI.