alt Hacker NewsThe frontier LLMs are getting pretty good at checking this sort of thing. You could prompt them to not only verify the references are real but that they actually state what the article claims. Some human review will still be needed but I'll bet this approach could find a lot of academic fraud.
Replies
> The frontier LLMs are getting pretty good at checking this sort of thing.
No, this is career ending high stakes. it requires old school "actually check a record of reality" type methods, like a database query or http get to one of the many services that hold this info.
why is the standard response to "this tech isn't reliable enough for this" to run its output through the same unreliable tech?
The device-fixer started breaking devices instead of fixing them. Tell it to fix itself!
Your approach is good for catching stuff that human reviewers might miss not as a first line default-only unit. The whole reason this is happening is because humans are not doing their job. Your solution (humans not doing their job) is just increasing the scope of the problem.