alt Hacker NewsI think most of this is correct, in spite of potentially being built on a bad assumption.
The assumption is that LLMs should be writing the code and human engineers reviewing and verifying the LLM output. And that this pushes the cost of producing down. And I fundamentally disagree with that.
Every time I ask LLMs to write code, even with Opus 4.8 (haven't tried it with Opus 5 yet), what I get ends up being totally rewritten. LLMs still aren't good at writing maintainable code. Can they write plausibly functional code? Yes. But it won't survive the long term. People using LLMs to write all their code are gambling on them eventually getting to a point where the LLMs can fix their own code. It's possible, but I wouldn't necessarily bet on it.
Where I have found immense value from LLMs is in code review. Repeated review by LLMs catches an amazing amount of potential issues. They really shine on security review, but are very effective with any kind of review.
The other thing that the "LLMs write code camp" misunderstands is that writing was never the bottleneck. Understanding was. And understanding the code is still the bottleneck. But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing.
Most of the time previously spent writing was actually spent updating and deepening our understanding of the system under development. There's no replacement for that understanding in a world where LLMs are doing the writing.
But if you flip it: humans write, LLMs review, then you still get a major gain -- not in speed, but in quality. And you keep the understanding loop intact. I would propose that this might be the best way to deploy LLMs.
Replies
How do you handle the fact that an LLM can't seem to help itself from disgorging page after page of words no matter what it's asked to do? I've never seen an LLM say "this looks good as-is; I would not spend any more time on it; what's next?" it will always seem to suggest using another pattern or additional abstractions or other yak shaving.
But to be fair human code reviews have the same problem. It's like reviewers feel they have not done their job if they don't find something wrong.
In my domain since circa Opus 4.5 LLMs write code as good as most engineers given well defined small enough chunk of work. They refactor. They write tests. These days LLM also debug/troubleshoot better than most engineers.
Are they as good as handcrafted code by 0.1% of top software engineers. Generally no. But neither is 99.9% of real code.
LLMs also are good at code reviews. What they'll miss is often the big picture but they can still catch plenty of issues. I still want to see a human in the loop in my domain.
Totally agree that writing the code was never the bottleneck. We're not seeing massive productivity gains even if some code is written faster. It's not just about understanding but also various other activities that happen in large companies and teams.
Also agree LLMs can be used to gain quality but realistically most orgs are going to aim for "fixed or decreasing" quality at lower costs.
I have not experienced this "llms dont write maintainable code narrative" I just tell it the shape of the entities and apis I want vaguely and the mental model and the output is excellent.
I’ve found that doing the full requirements capture, planning, writing, reviewing, gardening loop with frontier models has worked quite well since last October, and phenomenally since Fable 5.
The key I’ve found is human peer review. The reviewer jumps on a live call with the developer, pulls up the PR with transcription on, and asks questions. At the end of the call, the transcript passes back into the coding agent and the PR is polished up, becoming more self-documenting, and the humans are left with some degree of common understanding of what’s going on.
I’ve been operating my team of ~15 this way for 9mo to great effect… there is simply no going back to the stone ages.
I write code in Go in an established codebase. 90% of the time the LLM gets a basic api/feature right and fully e2e tested as long as I give it enough business context in the prompt. The other 10% of the time I have to do some follow up prompts to either change the behaviour or change the approach of a given step in the flow or just to point out that the wrong pattern was used and please rather use our codebase standard. I haven’t actually hand-written code in well over a year. Is this way faster? Absolutely. Does it lead to better code? Yes, because I have time to write all the regression tests that keep the behaviour as I wanted it when others come bungling around.
“Not in speed, but in quality” — unfortunately though, not a single CEO, executive, company, VC, investor or anyone with the power to make decisions cares about quality instead of speed.
> The other thing that the "LLMs write code camp" misunderstands is that writing was never the bottleneck. Understanding was. And understanding the code is still the bottleneck. But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing.
This was my stance a couple of years ago, but now I've given it up.
It turns out writing actually was the bottleneck. You can understand perfectly well what you want, but writing it is long and tedious to the point where you find excuses not to do it. Particularly with version 2, the step where you have an OK system and you want to improve it. Quite a lot of changing the code is just useless busywork: re-wiring old functions, moving imports around, searching for locations that benefit from extracting a common piece of code. And each time you do one of those, there's a decent chance you did something even more trivial like forgetting a semicolon or calling the wrong function.
Now that I have an LLM helping me, I can see why. The critical decision is a terse declarative like "we need to have several TCP connections instead of one, and just use the sequence number to arbitrate". A human junior programmer could perfectly well understand what this meant, but he would have to go through all of the above to get to the final product. Now, I can just tell the LLM and I will get what I want, even with the things I didn't explicitly state, without spending attention.
This means I can use my attention on the things that matter. So instead of spending today thinking about how to arbitrate between the TCP connections and tomorrow thinking about pre-calculating my outgoing orders, I can just do both today. I don't waste the good waking hours chasing minor bugs, I just think about the large structure.
I get the feeling the best programmers of years past were actually masters of the little things, which led them to be able to look at the big things. Essentially it was cheaper for them to get to the top of the mountain, where you can see the landscape. Kinda like how the kid who was good at mental arithmetic in primary school was also good at calculus at the end of high school: if you don't have to concentrate on the little things, you have time for the big things.
There is the flip side of where the code is no longer read by humans and is becoming the prevailing way software is shipped in tiny businesses. You don't need to code to be maintainable since you will never maintain it, the AI will and the quality will naturally improve as models improve. For example 5.6 sol and fable are showing signs where you can feed garbage in and it will spit out something pretty decent, definitely not the quality you'd expect from a senior developer with millenia of experience, but that of your average grunt worker turning words in an issue board into code.
However, sometimes then I tell it to write an app with detailed instructions and it spits out garbage so your mileage might vary.
" But understanding is truly gained during the writing loop. The understanding you gain from pure reading or code review is marginal compared to the understanding you gain while writing."
Debugging code step by step is how I understand complicated code.
You're right about understanding. Where we might disagree is the conclusion. I think it's possible to build the understanding without having to type out syntax by hand (debugging, writing tests, etc). Maybe we're missing much better verification tools? LLMs will likely play a big part in those too (explain the codebase, walk me through A, B, C etc)
I've come to realize that the smarter LLMs get, the less they understand the point of abstractions.
I've been using it for a unity game for the past few years. Nowadays it will go sleuthing into packages and assembly and make decisions based upon what it sees there.
It will make comments about why it's doing something based upon a function call 3 methods deep.
God forbid any of these details change in a minor version update.
I hate saying this but unironically opus 5/gpt 5.6 are writing maintainable code on the order of 500-1000 lines
many of us work with swarms of cheap off-shore contractors and have to rewrite or finish their code already as it is. The llm is much lower friction
> LLMs still aren't good at writing maintainable code.
I find that depends on the target language. They can be good at writing maintainable code, but not consistently across every language.
The languages beginners usually gravitate towards are especially hard for LLMs to produce quality output for. Presumably this is due to the training data including all the unmaintainable codebases written by beginners in those languages, which hasn't allowed the LLM to converge on recognizing what a maintainable codebase looks like in those languages.
I think this is outdated. If you follow spec-driven development, get the model to do all the planning work upfront, review and iterate the plan, write clear markdown file documentation on the abstractions and patterns you want to follow, then you have every opportunity to tell the model how you want it to write the code. If you use Opus or Fable 5 it will then write the code better and faster than you will.