alt Hacker NewsYour assumption is that LLMs will ever leave this uncanny valley.
Maybe unforeseen breakthroughs and different architectures are achieved. Given LLM fundamental shortcomings grounded in mathematics and information theory, I highly doubt they will and we will always need to deal with these issues in some capacity.
I think that bigger context windows help, but I feel that for AI to cross this chasm, it needs to be able to encode more abstract context knowledge. I think the holy grail here is online learning.