alt Hacker NewsHumans get an understanding of large scale systems into their head by developing and refining abstract models, and committing them to memory.
LLMs don’t do that. Any abstraction they do happens at training time; they cannot dynamically construct new models at inference time. Nor can they commit them to long term memory, because LLMs don’t have such a mechanism. Which, in turn, precludes progressive refinement.
We get around this by using agentic harnesses to pack everything into the context window. (Which itself is something humans don’t really have.) This is why large context windows are so important for most real world tasks: it’s the only way LLMs have to handle information that wasn’t in their training corpus.
I am not sure you're correct. Yes, from the physical perspective of LLMs design, the above is correct. But they are also systems that evolved to do things through training; so you cannot discount the possibility they can emulate building of new abstractions in another level.
LLMs already have the concept of "reference" - i.e. I can say a name of something, for example of a function, and they understand that the properties associated with the name itself are not relevant, but rather whatever the name does refer to.
But this is pretty much what you need to build new abstractions. You need to name a paragraph with the definition (explanation) of a concept, and be able to recall the paragraph by the name in order to do inference.
So with references, they can get around the context size limitations, like humans can. It's slower than learning things "by heart", but I think LLMs are perfectly capable of that; and the evidence is, they are able to navigate codebases larger than their context, which often contain many new abstractions.