I don't think it's a matter of "staying ahead"; the proprietary frontier models are better, but the trouble for them is that open weight models are good enough in increasingly many cases. This raises the floor on the frontier companies and cuts their total addressable market by commodifying the easier LLM tasks. This is really the central argument of tfa :)

They are running out of novel, clean training data and compute. There is probably a limit to how much improvement can be squeezed out of LLMs. Recent improvements have been more about orchestration and "reasoning" loops (i.e. iteratively feeding context back through the model).

Diminishing returns on both intelligence and training, mostly