alt Hacker NewsWhy couldn't an LLM, if it was smart enough, generate and consume its own data?
I know the answer: because it leads to model collapse. But why is that? Wouldn't a smart model not collapse? It's seeming like they keep getting smarter because we keep pouring more of our own knowledge into them, not because they are actually getting smarter. And yes, sometimes a dumb but persistent bruteforcer can make new discoveries.
Replies
It's because LLMs are entropy generators. That's not a bad thing for what people are doing.
But to prevent model collapse you need a way to pump down the entropy. Much like in thermo, it's an expensive and slow process.
If it is smart enough to generate data it can consume to train itself better, it is already smart enough to not need to do that.
They are already trained on generated data I believe
> if it was smart enough
and i think this is exactly the crux;
the really big models need really big datasets
and current gen LLMs get a lot of training data beyond "all books + all of the internet"
the objection is then that producing this additional data would already confound it with pre "virtual cutoff date" knowledge (since the training data probably implies mathematical and SWE concepts that were developed post "virtual cutoff date")