alt Hacker NewsThat sounds good. But it doesn't work in practice from my experience.
The exact same skill in the hands of one person will have vastly different outcomes in the hands of a different person. The review skill I built myself has been shared with folks. They (say they) use it before they put up a PR. I still catch things with the same skill. That is evolving as I catch the model(s) at BSing. Even if I would re-share it all the time, when I catch it, these other people using the same skill wouldn't catch it.
And that is exactly why building a general purpose review agent (or skill) is harder than me having my own evolving skill. If I have to "properly" release a skill/some automation, I will likely err on the side of not having too many false positives. That is harder than still relying on a human to review the AI reviewer. The same people that were really good actual human reviewers of a PR are probably the ones that are good doing the same thing while AI assisted. And the ones that really needed these other human reviewers and processes to help them be productive are probably going to produce a vibe coded mess if left to their own devices with nothing but some AI.
Have you tried running the same review skill back to back in fresh sessions? My experience has been: 1) build complicated thing with LLM, 2) run code review, 3) it finds like 8 things, 4) fix them, 5) run another code review, 6) it finds another new 8 things, rinse and repeat.
I'd guess people truly are finding things pre-review. It's just that LLM review seems to have a limit to the number of problems it can or will find. While at the same time, LLM-written code can be an almost limitless source of bugs and errors.
It has no sense of when an architecture is doomed and needs fundamental changes. Instead, it will happily continue to chase individual bugs nearly to infinity.
But, maybe your review skill is better than the ones I've tried.