alt Hacker NewsI have a contrarian take on this: if most people working on the codebase are using the same model, then these comments serve a very useful purpose: they tell the model's future self that despite its tendency towards this solution, it is incorrect and here's why. It encodes useful context right when the model needs it next to avoid making the same error.
I imagine this style of commenting is partially responsible for model improvement in SWE work, a trained trait through feedback, since it makes the comments actually useful. Not to you or me, but to an LLM.
Some of these errors are obvious to a human, that's why we find the comments unnecessary. But if everyone's using LLMs and said model tends to make this error, then they are very useful. It's like a comment you would leave for another human to help them avoid a footgun. It's just a LLM footgun, not a human one.
Replies
The problem is that the list of things one shouldn't do is infinite. The things one should do are narrow and get narrower along two independent axes of clarity/wisdom: system requirements and engineer seniority. LLMs overindex on any words given, so you only want to give them negative guidance around truly repeated, almost common-sense pitfalls. But in a large distributed system often the changes come from all different angles, and each agent will find it's own unique failure modes. Those comments will dilute critical context for diverse agents far more than they will help on average, at least for the systems I'm working on, given current claude code chattiness.
One way to clear some of the low-hanging slop is to just have a separate agent code reviewing and pruning any comments that don't stand on their own purely in the context of the diff, but it still doesn't catch it all.
I agree, but that information should be in the commit, which the model also has access to. Of course it's more token intensive to look through commits, but "good" models might know when some more research about a particular bit of code they find weird would be a good idea. Thoughts?
If LLMs are any good, like many of you claim, they won’t need these comments. They will ingest the code and know what was done. They won’t need comments about what wasn’t done.