alt Hacker NewsUnderstanding is the new bottleneck
Comments
We have LLMs try to generate descriptions of PRs for us and they're pretty universally disliked. They're always overly-complex descriptions of the mechanical changes and have no sense of motivation.
Also, a huge reason to understand the code yourself is to make sure the LLM isn't wrong, but this doesn't work if an LLM is itself generating the understanding.
I agree with the problem but not the solutions.
The problem pre-dates LLM's: writing code that "works" but breaks the underlying model. Because it works, it always sounds reasonable and doesn't raise any flags.
Only someone - human or LLM - who holds the model as the standard would see that this working solution breaks the model.
(In theory, the model is to preserve scaling, flexibility or some other systemic feature not immediately invalidated by this working code, but as always the model itself could be bad.)
LLM's are not bad at giving an account of the model; indeed, fighting with the LLM over what the model is can clarify things. But LLM's will happily hold on to a stream of inconsistent statements as their model, so they are not the authority.
It's more like this:
We've always lacked understanding. However, it didn't feel like a bottleneck; in spite of lacking understanding, we developed huge, complex systems that became hard to maintain and that nobody understood completely.
Now we want to scale that orders of magnitude, but when we do that, we feel the pesky lack of understanding.
We previously worked around the lack of understanding by making the system gradually incomprehensible in small increments, upon each of which we observed it still working, more or less.
If the whole thing materializes in one day, that doesn't work; the approach is gone.
You can now bring into being something which statistically resembles the old kind of system that was iteratively evolved. But the thing has no such history. You can't go back to play archaeologist. It looks like something that would have had users, but it never did. It was never in production anywhere. Nobody ever submitted feedback, or a bug report, such that it was fixed or improved. There never existed a simpler version of it that several ex-maintainers understood perfectly; there are no such ex-maintainers and no such understanding. There is no documentation trail, or other historic trail if surrounding activity like discussions and negotiations which led to things being the way they are.
"I read the code." -Mitchell Hashimoto
Great code needs great understanding and agents need excellent guidance. Even in my current solo-dev work, I can't imagine making a production commit I haven't read until I understand it. I own the consequences of my code; that's a responsibility AI agents can't take.
In the end, LLMs create garbage code that no one understands, they break things that should not have been broken, that would not have been broken if it was done slowly with understanding along the way. To reframe it as "understanding is the bottleneck" is just more LLM salesmanship. LLMs have their limits and when you hit them you're stuck. LLMs are the bottleneck. But the idea that "LLMs are the answer to the problem created by LLMs" is absurd.
I am surprised by the title and the story. Understanding has always been the bottleneck; there is nothing new about it. The argument goes like we humans should understand so we can verify and participate. How bold! Maybe we should have been doing that all along...?
I found the linked article "books don't work" from the quiz part highly interesting
https://andymatuschak.org/books/
It explains a lot and works really well.
I tried out in ChatGPT with a simple prompt:
> ...paste link... Give me series of quiz see if I really understood the article well. Ask & answer one by one in turns.
Really fun experience.
I am so dying to read more about the new/current/real bottleneck!
Where is the bottleneck? WHERE?? Tell me! No evidence needed, just lay it on, man to man, thought-leader to thought-leader!
Low value comment, so I apologize, but it's funny to note how many bottleneck articles there are now as a result of AI adoption. Lots of new bottlenecks.
Improving code understanding is the main focus of my work and thinking right now. If we want to make advances I believe that we should rely more heavily on one key quality of the program code: It is meant to be executed.
Here are some ideas:
1. Time travel debugging. Reading a PR just like a wall of text is difficult, but what if you could step through the PR and see the state at a given line for some test executions? Time travel debugging can make this possible. You would collect a debug trace and use it to overlay the PR diff with additional controls and information to resemble a debugger's UI. I was part of the team behind Codetracer (https://github.com/metacraft-labs/codetracer) who is trying to work in this direction.
2. Test suites and coverage. We don't use them enough for understanding right now. The test suite encodes what features the code is supposed to have, and the coverage tells us where in the code those features are implemented. I'm playing with an idea about this here: http://atlas.vihren.dev When we intersect coverages for the different test cases we can arrive at code segments which represent "atomic behaviors" present in the code. They form a mathematical structure which can be represented as a graph. I am currently exploring what value we can extract from it for the benefit of both humans and agents.
I think the ideas presented in the article are interesting, but at times the Notion references were a bit much. The author disclosed that they work there, fine, but at times it felt like it was more about showing off a Notion feature (e.g. embedded interactive HTML).
For me the solution has been to throw away the code I don’t understand. I let the agent write the code, and if when I read it it seems unclear or needs a lot of explanation from the agent, I just throw it away and start over, or do it by myself.
Analysis and synthesis are two sides of human intelligence. Current form of generative artificial intelligence is very powerful in synthesizing information and producing artifacts, human will lose this battle.
Understanding is about analyzing and we still have some tools to help us, such as type checking, testing, etc. To some extent this can be automated but needs to be maintained automatically to match the flood of synthetic artifacts.
“Synthetic analysis” is an oxymoron and could lead to hallucinations and irrelevance.
I love the idea that understanding is the new bottleneck. Because if we just ignore the potential horrors of cybernetic augments, it suggests the next challenge is how to teach things better. And that’s such a valuable thing to improve.
I have a soft spot for when I find a teacher or textbook or interactive website that makes something click. I live for that click. I crave it. I crave seeing it happen in others. How optimistic I could be if understanding becomes the primary target.
Understanding has always been the bottleneck, everywhere for everything. And now, with this new realization, are you going to finally realize that Communications and your skills with it are basically everything?
> Why? Why understand?
Because (in the just-barely-possible universe where anyone is stupid enough to make me a manager) I will fire you if I ever find out you don't.
What possible good can come from letting people deploy things over which they do not, in any sense, have cognitive ownership?
A small bone to pick, but describing an AI that is operating autonomously as creative seems wrong - at best, this process is accretive, because the AI is adding and adding, but has no ability or incentive to shape its output toward something a human would find of value. Value is subjective (individual), changing over time. An AI doesn't know when it needs to be taking away - removal is a key part of the creative process.
While the tips are good to handle the volume, I still think this sets code owner on a dangerous path.
AI have limitation and hallucinate. Complex code will be explained in hallucinated way. At some point AI will be unable to write more because the arch has become too complex or the volume of code will be to high.
The article I would like to read would suggest how to force LLM to architect the code like a solid tower instead of a pile of unstable mud.
Understanding was always the bottleneck. The way LLMs speed up your work is by letting you get code without taking the time to understand it. If you want to understand your code, LLMs are a net loss.
If you want to move faster with LLMs, you need to act like a manager and stop caring about what the LLM did. You just need to do the manual testing and make sure it works.
How does everyone feel about the “don’t read the code” stuff that folks are saying? I certainly do not support it but I’m curious to hear what other folks thoughts are
"new"? The reason for all the old bottlenecks is that you had to understand as you went along.
If you start with a spec you understand at the beginning then you don't need the LLM to generate high-level information about the changes at review time.
The grilling (grill-with-docs) skills [1] are amazing for ensuring you produce a through spec that covers all the edge cases. The /code-review skill from there helps ensure that the code changes meet the spec.
I use an intermediate detailed plan stage (done by a more expensive model) before implementation. Information from that plan is posted on the PR to give pretty much all the intermediate level context reviewers need.
I do like incorporating the idea of this article into my flow- that the spec and PR context could be presented in a more educational way.
My personal view is that programming languages are amazing tools for understanding. Some more than others, but even the worst—the most verbose, the lowest level—are better than they have any right to be.
So I think that we are leaving a lot of power on the table if we treat generated code exclusively as something to understand, rather than something to understand with. The techniques Geoffrey presents are great, but they should come alongside approaches that use code itself to develop and articulate conceptual models.
There is a body of knowledge in unit testing, integration testing, static analysis, model checking, formal methods, fuzz testing, … (what other techniques for building assurance in our code have I forgotten). And LLMs can be put to use towards all of these methods. But sure, we just need to think harder to solve all our problems. Velocity of code goes up. Velocity of testing can also go up. It’s just not as fun or glamorous.
I've been having a good time with Spec Driven Development, and it directly addresses the issue of needing the understand.
The whole idea is that you specify exactly what you want in some SPEC.md file. You can of course nest them, have multiple, etc, but the core idea is that the SPEC file is the source of truth, and all the code should be able to be generated by a competent agent into the working product you want. The SPEC file(s) should contain all the details and behavior you care about, and anything you don't care about is up to the agent to decide. If you don't like what the agent picked, _put it in the spec file_.
Critically, _you_ must write the SPEC file. You ensure understanding by doing so. You can of course ideate with the agent, but it's your ideas, in your words, specified by you. This also makes it a great source of documentation when you come back later and have to remember wtf is going on in this codebase.
I guess it depends on the context and environment, right? In many corporates, the bottleneck for me has always been specs and testing. I have daily examples. I had to explain it to leadership like this: "There are far more ways things can go wrong than right".
Understanding has always been the bottleneck.
In a team: yups.
Me with my LLMs: still.
Thanks for the post Geoffrey. I have been thinking about this a bit and wanted to come to these sorts of conclusions, you have saved me a lot of work (lol I have no ego that I have to figure it out I am happy you did).
Cog debt even on simple PRs is big and also cog debt when using AI to do organizational research e.g. what team do I ask?
Understanding has always been the bottleneck. Sometimes AI helps with it like explaining things pretty well with diagrams. However, in general I agree that more code is being generated per developer and it's difficult to keep up with the phase of new changes and understand it.
Understanding was always the bottleneck. It’s just now the understanding is backloaded instead of frontloaded.
ie. understanding it AFTER it’s already generated rather than before you wrote the code by hand
Claude make an entire app for me to describe this security contract change by pretending I’m in a Zelda game and only use funny metaphors because i’m bored and can’t read typescript
Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.
The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.
This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.
Absolutely. It's far easier if the thought originated in your own mind.
The default long responses of LLMs don't make it easy.
This talk is also available on YouTube: https://youtu.be/WkBPX-oDMnA?is=ojFaLX2onMn3ARhi
I've been using Geoffrey's /explain-diff skill in my replace-github-with-tailor-fit-personal-software journey, and I'm liking it. I recommend at least giving it a try.
Is there a markdown version of the `/explain-diff` skill? The page says there are HTML, markdown, and Notion versions, but I just see HTML and Notion
If you try to spec the problem with all the painful details for the machine to understand, you will end up with a rust codebase.
absolutley agree, but better understanding needs a better review surface that syncs with Github, which is why I created pyor.review, It's a blast compared to how I used to review code on Github and now I use it everyday.
reading everyone and their dogs post on "x is the new bottleneck" is the new bottleneck
Understanding is always a bottleneck regardless of human or ai. But now we are at a different scale.
Sorry but actually what.
Flash cards? Games with micro worlds? To understand ones own codebase???
We figured this out 10, 20, 30 years ago. Small, atomic commits. Small PRs. Lots of manually written tests. Documentation, ideally with the PR.
There is no "understanding" 60k loc highlander PRs. That's an entire feature.
We had it so good once.
To put it another way, AI is like a calculator or physics textbook.
You can say, "I don't need to be able to do basic arithmetic in my head. I have a calculator!". Or, "I don't need to know how to solve this kind of problem. I have a textbook and I can look it up on demand!".
Having to reach for a calculator constantly slows you down and makes simple equations hard, while also severely retarding your ability to do estimates and sanity checks. Not practicing on basic problems prevents you from developing the mental tools to solve more advanced problems, or being able to develop methods for solving novel problems. If almost anyone else could use your calculator and physics textbook to get similar results, what use are you?
Some companies are pressuring their employees to let AI do everything without slowing down to gain understanding of what it's done. These are the companies that most people won't have a lot of use for in the near future.
Understanding has always been the bottleneck. That's why LLMs aren't actually helpful: they speed up the part which is easy (typing characters into your editor), but are neutral or even harmful on the part which is hard (understanding the problem and how best to solve it).
Hot take: I look at the level of abstraction that matters most to me. When I encounter cognitive debt (usually due to sleepy sessions where I’m mostly “encouraging” Claude), I ask it to step back to clarify the overall purpose. If I get really stuck, I have it visualize the processes involved. Usually, the hard part is giving specific enough feedback to get a specific enough response within a much broader set of working material.
damn, I'm from those times when "good code is(was) self-documenting"
I guess a wall of text is the way now
"So I asked Claude to make me a video game — a command center where I do the port myself, step by step, watching the visible effects and the file tree evolve. It produced a UI where I click buttons to run the port step by step, with my old site and new site running side by side."
It's excruciating that this person is so close to reinventing moldable development and just keeps on skipping around it.
Yes, you should build tools that answer questions about your code, runtimes and systems. You should have tools that trivially allow you to incrementally and very immediately develop tools for inspection and getting clear answers. Going a roundabout way through some non-deterministic database to try and get there seems like a waste.
Breaking: water found in ocean
I thought this was going in a different direction along some of the thoughts I had around LLMs for programming tasks specifically. While this talks about understanding and how to ensure you're keeping up with what is changing, I feel that this is maybe more aligned with how a PM/PO should understand the work being done and not necessarily how an engineer should.
I took a note a few months ago and my point was I think more engineering specific, although it might be my own lacking abilities/skills that caused this realization. "Your capacity to learn/recall and map information is the new bottleneck. LLMs can act as learning amplifiers but correctness isn't as important for their output as critical thinking on the side of the consumer - YOU."
My point is that, I think if an LLM outputs 50,000 lines of code, your ability to go through what has changed, how it has changed and where the changes have occurred is the bottleneck. I see the approaches here, sure, "summarize the changes" or "draw me a picture" or the more recently observed "build me a city building simulator to understand this", but I feel that misses the point from an engineering perspective. The difference in understanding the weeds such as DB transactional boundaries or tenant isolation (which I believe was a topic in a recent data leak), those aren't summarized that easily in drawings or if they are, if you are working at this granularity, then your 50,000 line PR will yield 50,000 pages of crayon drawings you now have to understand.
I guess, my point is that understanding is the bottleneck, but low level understanding and the ability to read/map/connect is even more so. Any developer with some experience will agree that if changes are trivial you can scan and pick up mistakes or flaws easily. So most SOTA models won't necessarily even make these. So what you're reviewing now is going to be one level higher or more in terms of difficulty, mapping multiple components or touching multiple surfaces. Your ability to make the links, reason about them and attempt to find flaws or logic issues is the bottleneck. In the time it takes you to understand, another 50,000 line PR is up.
I'm not sure how we're going to be solving this. I don't know if in the current state it is a solvable issue, maybe another 6 months? Maybe another 6 years? Maybe this is fine and we will settle in a sort of place where your mediocre engineer will be responsible for tens of reviews a day signing off on method/functions/classes/interfaces being added, get paid 50k a year and doing the same non-thinking work day in day out while signing their name to the quality of the code being shipped while a senior/lead will be busy reviewing multiple of these. Think of the way an assembly line functions.
P.S. I hate to see this annoying tendency of transforming knowledge work into assembly line work. We keep trying to "fix" this without understanding what knowledge itself is. Maybe this technology will indeed yield software assembly lines, I don't wish to eat my words, but I'm still struggling to see how we will handle the nitty gritty of software work. Maybe the same way we handle building airplanes - as long as only a couple crash a year, we're sort of fine.
I think it funny how much average engineers are beginning to discover the challenges of engineering leadership and program management. This has always been the bottleneck.
It's why managers and PMs want to be in standup. It's why slack exists and engineers are constantly being poked on it. It's why execs always talk about not getting too far away from the work. It's how seagull management happens. It's why program management is a job.
All those behaviors engineers hated about their bosses that kept them away from being focused on the code...they're starting to feel what it's like on the other side and reinventing the solutions instead of just reading a book about engineering management. Maybe we'll rebrand program management to "understanding ops" or something.
I wonder what AI would say about us if given the tokens to complain.