logoalt Hacker News

bjackmantoday at 10:00 AM0 repliesview on HN

My experience is that basic generic agents are useless but an agent with extensive prompting about your usecase is extremely valuable.

In my case using these prompts:

https://github.com/masoncl/review-prompts

Took things from "pure noise" to a world where, if you say there's a bug in your patch, people's first question will be "has the AI looked at it?"

FWIW in my case the AI has never yet found _the_ bug I was hunting for but it has found several _other_ significant bugs. I also ran it against old commits that were already reviewed by excellent engineers and running in prod. It found a major bug that wasn't spotted in human review.

Most of the "noise" I get now just leads me to say "yeah I need to add more context to the commit message". E.g the model will say "you forgot to do X" when X is out of scope for the patch and I'm doing it in a later one. So ideally the commit messages should mention this anyway.