alt Hacker NewsI disagree. LLMs might create incoherence not because of context size limit is too low, but because it's too high. The limit is vastly higher than for humans.
But to overcome the limit, one needs to adopt simpler, more coherent models of reality, which is what I believe humans do. But there is little motivation to do so if your context window allows you to understand the situation in full.
So it seems there is a tradeoff. Some solutions (proofs of complicated statements) might require context of certain minimal size. So human solution doesn't scale.
Maybe we should try to adapt the context window to the system being worked on, to ensure that the system architecture has the structure viable in the minimal context size required.
Humans get an understanding of large scale systems into their head by developing and refining abstract models, and committing them to memory.
LLMs don’t do that. Any abstraction they do happens at training time; they cannot dynamically construct new models at inference time. Nor can they commit them to long term memory, because LLMs don’t have such a mechanism. Which, in turn, precludes progressive refinement.
We get around this by using agentic harnesses to pack everything into the context window. (Which itself is something humans don’t really have.) This is why large context windows are so important for most real world tasks: it’s the only way LLMs have to handle information that wasn’t in their training corpus.