Text AI watermarks will always be trivial to remove

AI watermarks feel like they're approaching the problem from the wrong side - no matter what it will be possible to remove the watermark (Via manual rewriting, local LLLMs, etc). Instead it seems like we need "proof of human creation". And the only way I see that being possible is hardware-attested proof of keypresses. Which obviously has huge privacy implications, but how else would you actually know that a piece of content was produced by a human pressing keys on a keyboard? The proof would also need to include timestamps for the keypresses, so tell if someone is just copying from another window.

The goal of the AI act is not to determine if an "oh yeah!" comment was AI generated. The target is long papers that falsely claim human review and can have real significant consequences.

E.g. research paper, law makers, lawyers, state policies, notaries,...

These are much longer content and thus statistically they will disclose a better guess at AI generated content.

Asking another AI to paraphrase will not erase the mark (which they are unaware about) but rather cumulatively add their own mark and make it easier to detect.

The problem is not to use AI, but to endorse the responsibility of the content you (as a human) deliver and somehow make sure that fake-news, biased content or unverified output is detected as early as possible.

I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it.

Currently I can recognize AI text because I read thousands of ai generated text. I know that 110% of yahoo finance news is generated. I don't want to read an AI generated personal blog, but if I do what's the problem really? Other than the companies distinguishing AI text for getting better training data, how do people benefit from watermarked text exactly?

The article mentions you can always simply use a smaller, local, un-watermarked LLM to rephrase the original watermarked text. Which is true, sure.

But if we're talking about deterministically taking some watermarked LLM output and having a function removeWatermark(text), it won't necessarily be "trivial" to remove, because the watermark function itself need not be public. Only the API that tests for the watermark need be public, right?

Anthropic's magic watermark could be, like the article mentions, something like "every 7th semicolon has a N% chance to be a comma where N is the sum of the last X characters mod Y, and every character in the bit range q1...q2 has a Z% chance to..." etc etc etc. And if Anthropic controls those variables, it would be very difficult to determine the rule, even with some pretty advanced analysis (I would assume). And keep in mind, that example rule I mentioned is pretty naive, too. I expect the actual rule would be way more advanced and not so straightforward as "swap every <charX> for a <charY>"

Yeah, but it's better than nothing.

People underestimate the value of rules that only take malice and a little knowledge to break.

And they tend to exaggerate that underestimation if they... don't like the rule.

If everyone is using ai (per reported revenues) and nobody like the outcome (banned here, dismissed there), then what is the future of AI?

"What would it even look like to sign ChatGPT outputs? There’s no artifact to pass around."

It would look like a lot of little signatures on little bits of text, and then larger signatures on a collection of those chunks once the larger chunk exists. It's not that hard. It's just a lot of signatures.

But this scheme could only ever prove that this bit of text was made by a given AI, and validate anything else ever included in the signature hasn't been tampered with. It's not hard to work up a scheme that proves (within reason) a text was generated no earlier than some date by incorporating some sort of information that could only have been known at that date so that could be validated. But this isn't even a step in the direction of proving that something was made by a human. And that's assuming the private keys stay private, which is its own tricky problem. If a private key ever leaks anything signed with it becomes invalidated.

Anthropic the other day announced future models "launched in the EU" will include watermarking https://support.claude.com/en/articles/16266773-how-claude-m...

I was assuming it was something like SynthID rather than just sneaky invisible unicode but it's hard to tell from the description.

I done a C2PA implementation for raw text, if that can ease someone’s life: https://github.com/dualeai/c2patxt

here‘s what i don‘t get about this whole discussion. AI companies already store all prompts and responses for future training.

just make an API that returns the string distance between a previously generated paragraph and the query?

that would sidestep this whole problem class.

regulators could even specify how that has to work.

what am i missing?

Good. Tracking and surveillance have no place in the modern world.

Society can overcome this problem by changing the way we think about text. Raw plain text should be banned, all text is cryptographically signed by the editor, gui element, or tool that created it.

For SynthID and similar solutions, there is much I don't understand ...

Here's what I grasp: The AI system scores each token and then selects tokens based on those scores. If we encode something in the token selection routine ('in order choose the 1st, 3rd, 1st, 5th, 2nd, then 1st highest scored tokens'), we can identify AI-generated text by comparing sample text (ST) to the expected text (ET) for that prompt.

1) How do we score the tokens for the ET without the original prompt? Even a Markov-like process needs to start somewhere.

2) To recreate ET don't we need to maintain, until the end of time, the AI state - entire model and code - at the time of ST output?

3) Doesn't #2 require maintaining all states for all AIs? Often you won't know when and from which AI system the ST might have been generated. What happens when an AI vendor goes out of business?

4) To recreate ET, don't we effectively have to rerun the prompt? Won't rerunning it for every verification increase most costs of AI output by an order of magnitude? Most of what AI vendors do would be ST validation.

Yeah but it's like saying "Masterlocks will always be easy to pop off with a hammer". Of course, but by doing so you are actively engaging in fraud, which then puts the onus on you and whoever you are attempting to deceive.