> Any increase in fidelity must come from somewhere...
Presumably, it's just that we had the training process wrong and it was actually more efficient than we gave it credit for. It's just that our interpretation of the output was incorrect. Asking the model to train itself simply has it self-reflect (via attention on the sequence length as it reasons) to unlock that which was already trained.
This could be an answer to some objections to training where people claimed that AI models need more data than a person to achieve the same result.