Reminds me of Gary Marcus's argument: LLMs (the forest) aren't genuinely intelligent, but the LLM providers can - by feeding off from the vast amounts of user data - make them simply memorized enough to turn every out-of-distribution challenge into an in-distribution retrieval and mixing task without having 'AI' that is ever truly intelligent i.e. can generalise.