alt Hacker NewsWell for the ding dong that downvoted me, here's a technical explanation of why this is a plausible theory
* current LLMs work by choosing next likely token, with a small randomization built in (emulating how humans sometimes substitute words)
* one implementation of the text marking is to "rig" the dice roll above, but do that on based upon a certain key
* so by looking at word substitutions in text, you can use a key verify it was generated by an llm.
* the algo is designed not to substitute proper nouns and some other asterisks... no idea if it's deterministic or not.
So yes, this is completely possible and even fits what happened.
The phenomenon of tortured phrases in published papers has been going on since at least 2020 - long before LLMs were a significant factor, and certainly before watermarking their output was a consideration.
The general working theory is that these phrases are the result of a "text spinner" tool being applied to a manuscript and blindly replacing random words or phrases using a preset list of synonyms (which are often imprecise and/or inappropriate). https://arxiv.org/abs/2107.06751 found that many of the commonly observed phrases were characteristic of an online text-spinning tool.