alt Hacker NewsIt might be a mistake to assume tomorrow’s training looks like today’s. Unsupervised learning is a thing and a very hot research topic, precisely because it avoids some of today’s big problems with acquiring the vast amounts of training data necessary.
Unsupervised leanring has been around for years and is already how the current wave of models are trained. It doesn't mean no data, it means no human provided labels of the data. So you still need creative new human ideas to move LLMs forward. LLMs != intelligence.