alt Hacker NewsMore LLM allergies.
Look at this PR: https://github.com/qgis/QGIS/pull/67731/files
The code change is so simple that Claude today could easily write this code. How will they know if an LLM wrote the code? They won't. And this policy will age like milk: Each passing month, coding LLMs are improving. It will become more difficult to "detect" an LLM-authored submission.
The best LLM policy that I have seen for an open source project is Linux. Roughly: You can use LLMs, but you are fully responsible for the code it produces. Linus acknowledges both positives and negatives to LLMs.
Replies
> How will they know if an LLM wrote the code?
The problem isn't just the code - the documentation for the change explaining what it fixes, how, why, what testing has been done, and so forth is often either lacking, far to verbose, or both (massively verbose without actually containing much useful information or other sense).
You raise a valid point for the future: when the average quality of submissions is good enough that it is impossible to tell the difference between human (or human aided by AI but giving its output sufficient review/rewrite) and LLM/agent submissions, then the issue will have gone away. But, given the problems projects like this are seeing it is obvious that the technology (or the people using it, probably both) are not nearly there yet.
> The best LLM policy…
That is a great policy. If only the entire world would follow it, or in fact even ever read it.
If the only reliable way for now to not be suffocated under a mass of bad AI submissions is to refuse anything that even vaguely sniffs of AI, then that is what many projects are going to have to do. If that offends you, then fork the project and patch your changes in, or vide code one from scratch yourself, then you can keep maintaining that along with all the submissions you get for fixes. Let time tell which the right decision is :-)
LLMs are still being detected at the same level since the day coding agents were actually something, hence, it's not improving for 3 years. Why would they be undetectable next month?
> How will they know if an LLM wrote the code? They won't
They don’t need to know. The policy is not a linter that will scan each PR. Just like the law does not monitor everyone 24/7 for criminal behavior. It’s enough that the rule is in place and it’s enforced when they catch an infraction. No need to be perfect as long as it lowers the pain.
The Linux kernel has the benefit of a big team of maintainers, and the GitHub integrations with these LLMs make opening these low-effort PRs incredibly trivial on GitHub vs a mailing list
You are missing the point
> AI stream is a lot higher than what human review can take seriously, leading to either a longer list of un-reviewed PR's or not fully reviewed code and quality issues. Both thing we don't want in the project.
It’s not about detecting AI, it’s about reducing the amount of code to review.
Agreed that's the best policy for the real contributors, but how do you fight back against the folks using your reviewer's good nature to test their coding agents?
Also, Linus is also comfortable closing 100 PRs as "this looks like garbage". Many communities are softer and try to be more helpful with new contributors.