you have a premise at the heart of that:

> understanding anything in the way humans do

i'm not sure it's clearly established LLMs can't be a model of some part of “the way humans do”?

to be more specific, i'd argue LLMs “understand” awfully similarly to a brilliant (polymath) with early dementia or Alzheimer's

no executive function, no short term memory, and absent both of those, conversing with that person about the past or with an LLM about topics that had been "in their training sets before a cutoff date" is surprisingly similar, right down to: introing a topic precisely the same way, you'll experience the same conversation; convo loops if bits are too quantized (looping and lossiness / recall / context-length are correlated); and ofc opening a new session is like the first one never happened