alt Hacker NewsI've recently seen this mentioned more and more, both on HN and on reddit. It seems these output patterns are getting worse. It's not just Claude, my impression is that all of the current models have this style issue. Their writing can get borderline incomprehensible.
Is there some feedback loop or compounding happening with each model generation? Maybe newer models are ingesting too much AI content? If the ratio of AI generated content in training data is getting higher and higher (because the amount of AI generated content is increasing in general), maybe this is a compounding bias, poisoning the training?
Replies
That's most likely what's happening. SNR will constantly decrease as LLM content is so much quicker and cheaper to generate, which makes it more statistically significant, which will make it more "relevant" for future models. A positive feedback basically.
They are increasingly being trained on generated tasks and even (parts) of the pre-training data is 'distilled' (e.g. Clibmix as an open-source example), so there are many ways in which the vocabulary can seep into the model.
We are seeing more of their "thinking". Lowering the refinement of the output to get closer to profitability. The nature of the LLM is that it generates huge amounts of text, then it iterates them down into a compact, hopefully accurate prose. That refinement is the really hard part and computationally costly.
The going conclusion is they’re getting models ready to talk to other agents, not people.
It's likely/It could be an effect of more reinforcement learning in training compared to earlier. You need loots of RL to learn to code well.