This is very cool. But also, it’s a benchmark/result for a very specific experiment/task: identity resolution
Despite the title, the article is not making claims of, or demonstrating general usage improvements or general coding agent improvements
Agreed, though many kinds of tasks are likely to benefit from this pattern, YMMV.
Not hard to test. I wired a “judge” into my existing Jev setup and… it works. I was able to trim my prompt down to the basics without “engineering” and have my entire test suite pass. Best I could get before was 1 failure, with an extremely brittle prompt. Like, adding a comma where grammatically appropriate broke 10 more tests.