alt Hacker NewsCan you back up this with hard data and evidence?
Most research converges to the idea that RL on synthetic data makes models worse, not better.
If what you claim was anywhere near that relevant, than we would've long achieved singularity by simply feeding increasingly better output to the training of the next model in a loop. Yet this doesn't work.
25 million turns on Claude output is a small amount, yet an expensive one (we talking hundreds of $ millions) that is better spent on compute.
There's no evidence such a process works, but I'd like to know more if I'm wrong.
Replies
Back up what? That distilling from a more capable model into a less capable model pulls the student model's capabilities up? What. Why the fuck is this even a question.
Look up literally any distillation works. Because this is just distillation but on one-hot token chains instead of richer logit KL proxies.
And no, I'm not claiming than you can "close the loop" and get RSI on the cheap just by distilling forever. I'm claiming that distillation is a very cheap way to bring the performance of a less capable model closer to that of a more capable model. It doesn't give you "a more capable model" out of thin air.
Which is why Chinese labs rely on Anthropic to provide that "more capable model" to them. They take the capabilities Anthropic trained for the hard way, and train for them the easy way.
It's a "fast follower"/"improved capability density" trick, not a "singularity tomorrow" trick. There are a few "distillation pump" tricks that get closer to what you have in mind, but they're still more about "extract more training signal out of the same set of data" than about "unbounded RSI".
> Most research converges to the idea that RL on synthetic data makes models worse, not better.
You are missing a mountain of nuance by generalizing the existence of a hole there.