I've used LLMs quite a lot (Claude, GPT) and have never seen this behavior. You've got something else going on.