logoalt Hacker News

jsrozneryesterday at 5:30 PM1 replyview on HN

I reviewed for ACL (big NLP conf) and another similar conference recently. Lots of crap (though there always has been). One superficially well-written paper was likely AI-plagiarized (i.e. the AI sampled an idea from prior work and rewrote it) and got desk rejected for it.

Reviewers were also totally unengaged. Of 20 reviews I read (from my reviewers or from reviewers on the same papers), maybe 2 were mediocre, and the rest were crap (though likely not AI).

The notion that science will somehow benefit from this is about as stupid an idea as you can have. Science relies on skepticism. AIs are not skeptical, and many folks are submitting papers because they stand to gain something, not because they are motivated to do good research or develop new understanding. Fields are being inundated with garbage that is maximally indistinguishable from real work (that's the training objective for LLMs). This in turn maximizes the cost of identifying bad work.

This is the same enshittification process that we see everywhere else. You get spam phone calls because there is no reason for a spammer not to call you. "Researchers" are submitting spam papers because there is no cost to doing so with some possible gain. Absent intervention, this eventually drives the community value of the network to zero (or potentially negative, if friction costs to switching are high).


Replies

Der_Einzigeyesterday at 7:29 PM

And yet the quality of work at ACL is still higher than NeurIPS, and that's with ACL this year having over 40% of all posters looking nearly identical due to everyone claude coding their posters...