Is this unsurprising? It's a fancy markov chain. It's like using a slot machine to diagnose medical conditions. I guess it's a slot machine with really good marketing.
People are reading way too much into it, talking about "emergence" and anthropomorphizing it to insane degrees.
Hey, 2021 called, they want their knee-jerk criticisms of LLMs back