This, a thousand times. As the ratio of code to human writing necessarily [1] goes up, they become not just smarter, but more precise and technical, which requires them to use more jargon. You could say they become more nerdy. Hence, text generated by these models will become more easily recognizable, at least by default, when not asking them to twist themselves into something else via prompting — which degrades intelligence. This is a good thing, in my book, given all the slop we already have to contend with.

Of course there will be models trained on much less code and technical writing, and they will create more natural sounding prose, but they will lack the deep intelligence of frontier models. Seems like a fair tradeoff.

[1] watch the first couple of minutes on this bycloud video on scaling training data mixtures: https://www.youtube.com/watch?v=aD93kfArOik