A normal agentic loop will have the agent using a type system and basic tests to do some basic validation of changes. A good agentic loop would give the agent a very easy way to verify if it’s on the right track. I think agents are better than many humans at writing error free code (runtime errors, not bugs. The code could still be buggy or incorrect.)
Once agents run autonomously for longer periods preventing vision drift becomes as important as correctness.
Epiq solves this with an architecture that supports workflow auditing, allowing you to time-travel state in a filtered view to reconstruct what happened, when, by who, and where intent started drifting, while also allowing you to correlate the evolution of the board with the corresponding commits.
Fundamentally though, an agent cannot produce great code, because great code requires intent, which is the opposite of the statistical mean.
You will get a solution that works with a proper workflow, but you won't get one that scales or would be truly maintainable. Which is also what you get with random midwit drive-by contributors, but faster. I'll give it that.
[dead]
That has been my experience as well. Having the agent have full access to what it is building feels like a cheat code. A recent example: Godot (the game engine) has a feature where GPUParticles can spawn another particle system when they collide with something. Today (as in, download the engine right now and see it for yourself) this is broken, particles spawn beneath the collision box and the faster the particle is the worse it is.
So I asked Claude to first figure out a way to instrument GPU particle code so it could itself check his solutions. After that I left the agent running for 3 hours and it came back with a solution (continuous collision detection) and a bug fix (particles get their velocity applied an extra frame after colliding). I'm sure someone would find something to complain about the code (which is why I haven't upstreamed it) but it looks visually perfect and I had plan to fork the engine for my project anyway