The gains from increased parameter scaling are sublinear: there's no more hockey-stick improvement to be seen going in that direction. That doesn't mean some improvement isn't possible - it's just going to be increasingly not worth doing.

Also, I think the fact that small open-weights models are catching up to the frontier rather than the frontier rapidly pulling away is evidence of this. In fact, by far the most dramatic capability increase story over the past two years has been the gains made in the small-parameter regime.

One might think, "hey, this agentic coding thing was a pretty big deal!", but I think it's a bit of a distraction because models only recently became optimized for this specific use case. It's not like they suddenly gained so much general intelligence that they magically had the ability to use a coding harness. No, the labs started spinning up a bunch of RL environments and generating rewards over long-horizon trajectories of combining these tools. It's an excellent application of LLMs but care needs to be taken interpreting how much "progress" has been mae in terms of raw generalized capability.