alt Hacker NewsEvery modern (and not so modern) software development method hinge on one thing: requirements are not known and even if known they'll change over time. From this you get the goal of "good" code which is "easy to change code".
Do current LLM based agents generate code which is easy to change? My gut feeling is a no at the moment. Until they do I'd argue code generated from agents is only good for prototypes. Once you can ask your agent to change a feature and be 100% sure they won't break other features then you don't care about how the code looks like.
Replies
> Once you can ask your agent to change a feature and be 100% sure they won't break other features then you don't care about how the code looks like.
That bar is unreasonably high.
Right now, if I ask a senior engineer to change a feature in a mature codebase, I only have perhaps 70% certainty they won't break other features. Tests help, but only so far.
I am constantly getting LLMs to change features and fix bugs. The key is to micromanage the LLM and its context, and read the changes. It's slower that vibe coding but faster than coding by hand, and it results in working, maintainable software.
> Do current LLM based agents generate code which is easy to change?
They do. I am no longer writing code, everything I commit is 100% generated using an agent.
And it produces code depending on the code already in my code-base and based on my instructions, which tell it about clean-code, good-practices.
If you don't get maintainable code from an LLM it's for this reason: Garbage in, garbage out.
This is the brake on “AI will replace all developers”.
Coding is a correctness-discovery-process. For a real product you need to build to know the right thing. As the product matures those constraints increase in granularity to tighter bits of code (security, performance, etc)
You can have AI write 100% of the code but more mature products might be caring about more and more specific low level requirements.
The time you can let an agent swarm just go are cases very well specificed by years of work (like the Anthropic C compiler)
I'd add in "code is easier to write than it is to read" - hence abstraction layers designed to present us with higher level code, hiding the complex implementations.
But LLMs are both really good at writing code _and_ reading code. However, they're not great at knowing when to stop - either finishing early and leaving stuff broken, over-engineering and adding in stuff that's not needed or deciding it's too hard and just removing stuff it deems unimportant.
I've found a TDD approach (with not just unit tests but high-level end-to-end behaviour-driven tests) works really well with them. I give them a high-level feature specification (remember Gherkin specifications?) and tell it to make that pass (with unit tests for any intermediate code it writes), make sure it hasn't broken anything (by running the other high-level tests) then, finally, refactor. I've also just started telling it to generate screenshots for each step in the feature, so I can quickly evaluate the UI flow (inspired by Simon Willison's Rodney tool).
Now I don't actually need to care if the code is easy to read or easy to change - because the LLM handles the details. I just need to make sure that when it says "I have implemented Feature X" that the steps it has written for that feature actually do what is expected and the UI fits the user's needs.
> Do current LLM based agents generate code which is easy to change?
Yes, if that's your goal and you take steps to achieve that goal while working with agents.
That means figuring out how to prompt them, providing them good examples (they'll work better in a codebase which is already designed to afford future changes since they imitate existing patterns) and keeping an eye on what they're doing so you can tell them "rewrite that like X" when they produce something bad.
> Once you can ask your agent to change a feature and be 100% sure they won't break other features
That's why I tell them to use red/green TDD: https://simonwillison.net/guides/agentic-engineering-pattern...
We won't be able to be sure of 100% with LLMs but maybe proper engineering around evals get us to an acceptable level of quality based on the blast radius/safety profile.
I'd also argue that we should be pushing towards tracer bullets as a development concept and less so prototypes that are nice but meant to be thrown away and people might not do that.
The clean room auto porting, after a messy exploratory prototyping session would be a nice pattern, nonetheless.
No. They’re great at slopping out a demo, but god help you if you want minor changes to it. They completely fall apart.
Irrelevant because you are not going to make new changes by hand. You will use AI for that.
All the hype is on how fast it is to produce code. But the actual bottleneck has always been the cost of specifying intent clearly enough that the result is changeable, testable, and correct AND that you build something that brings value.