I think this is the wrong approach because everything is external to the model. You end up creating an ad-hoc externalized model scaffolded out of coarser systems, RAGs, files, and so on.

This leads to, if useful at all, to this process of ad-hoc recall inference which has to happen in time. By itself this is not a problem.

The problem is that the model has to be constantly injected in context with the newest version of the "memory state" at each turn or relevant turn.

The newest coherent memory state is also a problem. More or less 50 years of not failures, but not success either. This may be even deeper problem than the externalization problem.

I think we are just in the very beginning and we are slapping database stuff to the transformer hoping it will work, but these deep neural-net architectures categorically show that they are not databases.

There will be a synergistic middle ground, but its shape is still not clear.