The end goal though (in my understanding) has never been for an LLM to regurgitate knowledge it ingested during its training. The end goal is to use the patterns and correlations found in internet data to generate an emergent prediction and problem-solving machine.

Whether that's possible is something we'll discover, but no one is throwing billions on AI companies for the hope of them building a giant natural language queryable internet information repository.