alt Hacker NewsWe have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
Replies
> They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
This is funny to me. Coding isn't a main part of my job, but I know someone whose it is. And he says the exact same thing about his colleagues. And not just about PRs, but also comments in code in general.
Are these generated descriptions of LLM submitted changes or of human changes? If a human, shouldn't they be putting the motivation into the PR?
It gets better if you tell it what you expect, but maybe even better is to keep some examples of "this is a good PR description" and feed it into the LLM generating another.
Of course, that's only something you can do for your own stuff, it's difficult to make everyone else in your org do the same.
With GitLab at the moment the description I get is just "Close JIRA-123".
90% of what I want to see in a PR is "why" and an LLM is entirely incapable of knowing that.
The rest is stuff like jira ticket ids and related PRs which you can get a script to inject.
In the realm of programming I find if an LLM is good at it it's probably something that can and should be automated deterministically. It truly is e-duct tape.
I don’t know. From my experience I get pretty good descriptions of PRs if I ask the right questions and provide some context.
I am so very tired of 2 page long PR descriptions for a 5 line change.
[flagged]
Id rather have ai descriptions than an engineer lying to their PM. Mist engineers I've worked with are lyers and they usually form groups incase you're wondering why they arnt called out
I hate to be pedantic but you can finetune a skill to shape the PR message the way you like it. That being said, I did have exactly this issue you mentioned, but the defualt output can always be tuned.
Maybe on your team, but I don't think AI PRs are universally disliked. The people that submit PRs without understanding them are universally disliked.
Have you tried writing in AGENTS.md or whatever to exactly explain what you like/dislike about the PR descriptions?
My main gripe is with Claude deciding to make 200 lines of code in a PR I need to review, instead of 3 lines of code somebody who understands the original algorithm/intent would do. And coworkers just YOLOing changes without understanding them. Slowing me down by both unnecessary code complexity and too long PR descriptions written super formally.