> The top model from 2025 looks silly compared to the top model of the first half of 2026. Do you feel like progress has stalled?

I do. Pre-training is where the industry saw the “emergent properties” of LLMs arise and, for a time, people thought you could just keep scaling up bigger and bigger models but then the incremental gains from doing this did plateau. Labs will still do bigger models (Bytedance has a 10T planned), but these are sparse architectures and they aren’t going to have mind-blowingly greater intelligence. Fable didn’t either.

Scaling pre-training tokens also flattened out. What is still delivering gains is scaling RL on verifiable tasks. But that’s not general intelligence - it’s fitting models to specific tasks, which ML has always been good at. More importantly, most tasks to which humans apply their intelligence don't have computationally verifiable answers.