Likely won't improve much. They trained on every text already.
most of the gains from the past year and a half have not been from web data, but from synthetic data and agent rollouts with RL.
alt Hacker News
most of the gains from the past year and a half have not been from web data, but from synthetic data and agent rollouts with RL.