I'm guessing - probably some textual variation on Benford's law? [1]. Trivial for compute, painful for a human.

- "Ensure distribution of vowels is in >99th percentile of human work"

- "Ensure the distribution of the letter "s" is within 99th percentile of human work"

- "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters.

- "Ensure there is a cross-linguistic 'typo' (colour vs color) at 1/N words, where N: 1000 = Model1, 2000 = Model2, 3000 = Model3.

- "Ensure the distribution of tense error is within 99th percentile of human work"

If more than 3 dimensions have a score >99% percentile of human, let's call it watermarked...

- 1) https://en.wikipedia.org/wiki/Benford%27s_law

It essentially looks like the difference between two different runs of the model with the same prompt but different seeds. The watermark is essentially a small bias in the model such that when there's multiple different tokens that could conceivably follow the previous token, the model will only pick some subset of them (the subset is derived from a hash of the previous token). This bias can then be checked for statistically (without needing access to the model and without needing the whole prompt), and for longer text where there's enough freedom in word choice you can show that it would be vanishingly improbable to accidentally follow the rules in the watermark.

IIRC, watermarking text could be as simple as training the model to use specific words/phrases more frequently than what you would expect to find in human-written text, to the point where it's highly statistically improbable that it wasn't AI generated. I assume similar logic could apply to code in the form of functions/code styling.

That's probably an over simplification. Also a solid defence that can be used against complaints about the way AI writes text.

non visible text is extremely easy to filter with a git hook, a post tool call hook, or just a script. I doubt they are doing that