But despite getting better and better at not making stupid mistakes, the models are still just next-token predictors. There’s plenty more room for them to improve without a totally different architecture.