alt Hacker NewsAn ill advised undertaking IMO: Any attempt to build AI generated content detection systems is flawed, since the outputs of such a system may be used to train an even better fake data generator. This leads to an equilibrium state of digital uncertainty: nothing in the digital realm can be deemed as real anymore - only as digital.
Replies
This is what I've been saying for years, then I was surprised by Pangram actually working.
Sometimes it pays to attempt the impossible.
One thing I've observed is that people who use AI rarely want to hide that they're using it. With text or even images, it's sort of like a dialect. AI-generated/assisted stuff is becoming somewhat like a fashion trend where people tend to use it and it becomes part of them, rather than something they want to pass off as entirely human-made.
So I think there's very little incentive for an arms race to occur in this case.
there's some truth to what one says but after hearing how Pangram works I'm pretty impressed at the methodology
I think the founder once said something like "well you can try to fight our detection and do a bunch of stuff, you just might find yourself with a pile of extremely garbled text that is less than useless".
I'm not all that worried about this, I do have a very small concern about there only being one gig in town doing this. Creates a situation where they could be co-opted to label legitimate images as fake.
This is how ai is trained anyway - generative adversarial networks
In theory you are correct, in practice it works. In theory you could use it as a training signal, in practice you cannot. In theory, theory and practice are the same, in practice they are not.
How is that a downside exactly? If we can’t tell then we are no worse off.
I doubt AI labs have beating Pangram as an optimization goal.
> the outputs of such a system may be used to train an even better fake data generator.
[citation needed] - my understanding of how entropy works in real photos vs. synthetic images (AI or not) is that this is extremely difficult without severe degradation of image quality.
[dead]
AI detectors can't work in theory, but Pangram works well in practice. This is because the AI companies are not trying to fool Pangram. They have no incentive to; in fact the incentive goes the opposite way. They want to watermark their outputs to leave no doubt as to whether something was generated by their AI.