logoalt Hacker News

wongarsuyesterday at 10:15 PM2 repliesview on HN

You can detect if texts from a year ago used AI based on statistical patterns. Nobody is taking issue with that. But once you tell people "we will run these tests to detect if your future submissions are using AI" you create an adversarial environment and your statistical methods will continuously break. Not because statistics is broken, but because you are trying to hit a moving target that doesn't want to be hit.

That's not like detecting thoughts via fMRI, it's like detecting tomorrows malware with yesterday's malware signatures. Or like researchers making a vaccine against the common cold

And the obvious proposal to fix that has been made multiple times in this thread: don't make take-at-home tasks part of the grade. Instead of trying to punish what you can't reliably detect, take away the incentive to do it in the first place


Replies

lich_kingyesterday at 10:38 PM

> You can detect if texts from a year ago used AI based on statistical patterns.

I don't understand your argument. The vendors for these detection tools can acquire recent samples from all frontier models just as easily as you can use them to write essays. There's nothing that requires a one-year delay.

show 1 reply
oytisyesterday at 10:31 PM

> you create an adversarial environment

Do AI vendors specifically train models to circumvent AI detectors? Why would they?

show 1 reply