alt Hacker NewsIs that really human input, or just grounded input in general?
Most human-written texts on the internet refer in some way to the real world, or at least to things outside the text itself: They might be descriptions or reactions to current events taking place, or blog posts or wiki entries discussing some aspects in detail. Even completely fictional texts reproduce basic assumptions about the world, culture markers, etc.
In contrast, LLMs learning on their own output have no connection at all to the outside world, the data just reinforces all assumptions the model already has, whether or not they are true or useful.
What is theory without observation? Science was turbocharged once we recognized the need to make testable theories and then actually conduct tests, always ready to toss a theory, no matter how beautiful it was or who came up with it, if it failed the test.
LLM's don't "think" that way. They don't form hypotheses or theories. We may need an entirely new class of AI algorithms to successfully cut humans out of the loop, and that may be a very fortunate thing. The corporations currently at the forefront of AI appear to be riding a wave of gold-fever and are completely lacking in patience or good judgement.