alt Hacker NewsThe problem is not AI code, but not knowing about system architecture or intent
Comments
When you write something you constantly remodel your understanding through refactors and rewrites until you internalize it. By internalizing it you gain the capacity to reason about it (during critical downtime) and communicate it. An entire team that can communicate can solve problems together, from one guy's vision to products white boarding to engineering's infrastructure to UX and UI's artistry.
It boggles me we completely forgot that the world operated like this just 4 years ago
I am finding this, professionally, not so dramatic - but mostly because the industries in which I've consistently shipped code at scale involve review as a first principle, as in no un-tested, un-reviewed code gets shipped, because: safety critical/realtime requirements and certification specs, say so.
And having AI code to review is no different than any other code that ever was to review, so the review tooling is - as it necessitates - also benefited by lugubrious application of .. more AI. But: all AI is human reviewed.
So it's not a big impact. We just don't ship code that isn't 100% human reviewed, If that's insurmountable: you're doing it wrong. Use AI to make code readable again.
And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local.
Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.
The industry will prove this, itself, sooner or later: If you can't put your AI in its box for safe-keeping, you're doing it wrong, anyway... and should've already learned this practice as a habit, decades ago, vis a vis future-proof tooling... (See also: not logging everything you do with an AI? Big fail.)
Sure, the absolutely intoxicating addiction of Big Metal AI™ is going to put a lot of consumers in a deep, deep pit of Neo-Illiteracy - however: 'good' AI code is actually just good code.
I see this everyday. The problem is code is the wrong abstraction for the work we do. LLMs have solved coding, but they haven't solved systems, collaboration or system maintenance.
Edit: Since I seem to have touched a nerve - I've been working on a project to solve this: https://www.archme.io if you want to know my thoughts on the right abstraction
AI is going to make, long term, Software Engineering more important. Getting requirements, user feedback, tests, product vision. These are key differentiators now.
If you can really get a good set of requirements, go and write all of your test cases out, and then throw it at an AI that will one shot it. Its perfect.
AI is not for programmers, they will always complain and given time and resources can of course produce better atomic code. AI is for a combination of a business analyst, architect and product manager in one person who has capacity in coding but decided this is limited role in the whole software engineering lifecycle. With the right attitude and of course certain level of micro management over AI (which they had to do with human programmers as well) they can achieve the same or similar results with much less communication friction, less people to maintain that teamwork and get closer to solving issues they find rather than discussing philosophy of programming and other manifestos of the craft that stopped being prestigious or unique and that’s the key point of complaint when AI is given wrong people to use.
I feel related to it. I switched companies at the end of July. First 4 weeks AI was giving me the feeling of freedom. No longer I need to understand tens thousand of lines of legacy codebases. Never onboarding was so easy. Just ask Claude and it tells me what happens here and how.
But after 2 months it starts to backfire me. I still know nothing. I have some understanding of the system design and core components but I have zero clue about how certain things are done under the hood. Because AI read code for me and code for me and I take it as my own understanding.
In last week I end up limiting my AI usage and forcing myself (it is really hard) to read and code at least a bit by myself to start having any idea about what is going on here.
In this moment there are a million ways to do things wrong. But there are also some ways, maybe less than a million, to do things right.
Here's an anecdote about a way to do this wrong.
I have found with AI coding methods that there's a line where it becomes a hail-mary (in the American Football sense).
A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.
This moment with AI code is indicative that you can't put together a coherent plan so you just tell the agent to "make it good". It used to be that the results here would suck, but now the agents are really competent, so the results might be good.
But at that moment, that's your cue to back up. Because as soon as you take a solution that's so far detached from your understanding, you're underwater. The hail mary is not part of a larger game plan. It's the last play of the game. There's nothing after.
So as soon as you reach that moment in your coding, you're signaling that you're done understanding not just the code, but even the way it works at a high level. If you're still going to work with this code after, then back up and work with the AI to get more understanding of the problem.
Yeah we forgot the goal of programming isn’t just to tell the computer what to do, it’s to program the programmer into thinking a deeper understanding of the problem.
It's not a given. It's about willpower and care. LLMs are the ultimate crutch. I over-rely on the crutch, more and more people will over-rely on the crutch. But it is still inherently a psychological problem.
You can still know things and get force multiplication out of LLMs, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.
Each to their own, and the market will decide in the end? If a company is going to (implicitly or explicitly) incentivize everyone to directly ship Claude’s output straight to production, and prioritize features above everything else, that’s a choice. Providing that I have a long enough runway, I’d love to have this company as my competitor? Yeah, year 1 will suck, because they’ll pull ahead, but if I can stay in the game until years 2 and 3…
Aside: this reminds me of running a “negative split” in a long distance race, where you aim to run the second half faster than the first. It’s very hard to do this because you have to be willing to let everyone else in your pace group pull waaay ahead, and running above race pace in the beginning feels “free” with all of the adrenaline. But if you do manage to stay disciplined, it’s a fantastic feeling to reach half-way with gas in the tank, and then start to reel in all those runners who sped by in the beginning.
We've had team changes and product manager changes and lack of documentation for so long that this was the state of our team anyway: no one knows why it was done and no one wants to break it.
> But the final boss is, and always will be, maintainability.
I think this article misses the mark in a few ways, but this is the biggest. I see AI as the ultimate solution for maintenance, in three ways:
* AI does a great job of refactoring and so tech debt becomes shallow. If something is not architected right, it can be fixed much more easily than ever before. * Bug fixing is also a great AI strength. In the future there won't be a backlog of all the bugs that never got fixed. AI can fix them as quickly as they come in. * AI doesn't get bored, doesn't get tired, and doesn't care how crufty the code is. It is happen to maintain any application without judgement.
I'm understanding my role as a developer in the AI era through the lens of the company hierarchy at my first job. In 2018 I began my career at a management consulting firm which also did tech consulting for a very specific industry.
The hierarchy was as such - associates at the bottom who did all the manual coding work and got into the weeds with the tech, consultants on top of them who orchestrated the work and kept the wheels moving, and managers on top of them who had a more higher level view of the system and knew why decisions were being made (partners were involved with future deals and driving the business forward). The managers who knew how the system worked had no idea about the actual code underneath it all because it was abstracted away from them through multiple layers of hierarchy. The actual code which was mostly written by 21 or 22 year olds who largely had no idea what was going on other than the fact they had to make it work or risk the ire of higher ups was a mess, unless the consultant orchestrating it all had real technical and system design chops.
I was an associate then and more often than not I had no idea what i was doing but i was fortunate to work with consultants and team leads who did. Right now, I'm a software developer akin to a consultant, with more ownership of the code but no associates to guide because all of that work has been pawned off to AI agents. I see my role gradually morphing into what the manager role was at the consultancy, where i have a bird's eye view of the system and know where the project is moving and why decisions are being made, but with a very limited knowledge of what is actually in the code.
These year and a few next ones will be the years when everything was possible.
We have both the tools and the skills.
Later, we will loose the skills because of AI and the depletion of natural resources will lead to the scarcity of the tools.
I don't really understand all the negativity - I've delivered amazing robust solutions at about 10x my previous rate I think and can make dramatic cross code changes on awful legacy code bases with ease. I just added some complex prev/next navigation across a load of detail pages pages that rely on the current search, a new PDF generation system that uses a queue and workers and absolutely tones of changes to a legacy ERP system that would be unworkable without AI.
Dealing with legacy messes I used to get frustrated and bored of making the improvements, now it's easy to clean up code bases and write loads of tests. I asked Astra to come up with a plan for Playwright testing the whole App, I have not built it yet but the flows suggested were fantastic as was the ephemeral database we plan to create for CI.
I built my friends portfolio website almost entirely vibe coded in 4 hours and it looks unbelievable, we added so much slickness (he's a designer) just prompting together. I used a CMS I had never used once before and it was so so easy to do without any of the usual need to read docs about everything.
I've done so much devops now I'm actually fairly confident that me and an AI can do anything you want in terms of deployment/infra and scaling from AWS to Terraform to whatever.
Anyway my main concern about this technology is not that it is crap at coding it's that the improvements in how it codes and thinks are absolutely dramatic which is extremely scary - it has come so far in a year I wonder what the next year will bring.
I don't mean to be mean, but the problem is that AI has exposed how much programming is designed around human usability. C/Rust/Go wrap assembly. TypeScript wraps JS. Java/Python wrap C.
Do you need all these layers of abstraction when the human is no longer looking at the code?
Best analogy is forgetting how to use a slide rule following the advent of calculators. The former was made to make hand-calculation of logarithms easy. The latter does these calculations directly (obviating the need for a slide rule at all).
I think what humans still need to learn are the theory and domain fundamentals for their industry. If that industry is computer science, that means algorithms, calculus, linear algebra, etc. I think the future of CS is then (a) theoretical human-drive design and (b) prompt engineering to implement and verify that design.
It would also be helpful to have domain knowledge outside of CS as having the skills to build something is nearly commoditized (outside of the above fundamentals).
I think it is not needed to own every line of code anymore , but you should still have control and idea about the architetcure of the system. You should understand what componenets are there and what is the repsonsibility , how are the orchestrated and what are my quality gates where i messure if what i requested match the results. Thats why i build archkeel (https://github.com/rapiddweller/archkeel) and datamimic (https://github.com/rapiddweller/datamimic) ... to increaes the transparency and review surface for human ... to make the results easier to judge ... i think when we stop knowing anything, we can also stop burning token and ressources
There is no "problem".
The architecture and the intent don't matter to business folks, it never has, and with LLMs it matters less than ever.
Vibe code into production is satisfactory regardless of architectural understanding or intent. If there are issues, just have the LLM spin up some agents to play whackamole until the issues are pushed beyond visibility.
Ultimately, the idea that we can hang onto fleeting engineering disciplines misunderstands where the industry is going, regardless of any assessment of the LLMs capabilities.
This is the perspective, not code, coding is solved. The wrong input will produce wrong outputs, so a mediocre dev using AI may produce mediocre results. AI can do everything asked for, code, tests, reviews, fixes, etc but still it can not think, for now
Intelligence is the new currency, but it will be short lived
The post seems to be in line with some of my own direct observations. I've been pondering recently whether for many devs the use of AI is the death of the mental model.
We all interact with systems through mental models, but if many devs are just prompting claude when something doesn't work, they might read what claude found, but lose out on the exploration, debugging, and work that builds and reinforces the correct mental model and discourages the wrong one. And if devs are missing out on the mental models, will they actually be capable of driving efficient solutions to problems as the mental models get worse.
No the problem is allowing changing of system architecture or intent because the AI just did it and being totally trained to mentally surrender to the whims of it.
My boss-boss asked me to do a presentation on a topic I actively research for quite a while now.
Scheduled a quick call to align me on what he expects - normally he wouldn't do that but he has attached a big agenda written by Claude what the presentation could show, and invited two other product colleagues of mine.
I came to a Miro board of the Claude Agenda, put into Miro using the MCP.
Honestly, just tiring. Asked my colleagues if they would just put the Claude agenda into Miro with Claude, what they need me for when an AI could just narrate it.
I agree writing completely new systems with LLMs is now near impossible to keep in your head, however, the onus is STILL on you the individual engineer. You are mistaken if you think that's changed.
So, if you're pooping out code, and committing it because tests still pass, and that's all you know, you're in for a treat. When an executive wants to know why a b0rked feature lost their department millions of dollars, guess who will have to answer for it, and its not the LLM.
My advice is to find ways to keep on top of how it all works, and if you're the only one who cares, well, then, that makes you even more valuable, not less.
I some very real way this has been true for a very long time. Does anyone know how a AMD Ryzen processor works? I mean does anyone understand in any detail how it works from machine code down to the transistors?
Does anyone single person understand what’s happening when compiling a large C++ code base? Meaning, can anyone track the basket cast C++ language constructs down to the Clang IR to the optimized machine instructions? From there can any one person follow those machine instructions all the way through to the actual registers etc to actually running the code?
Clicks on blog, sees AI generated template, leaves.
Enough, already.
After experience on both ends of the spectrum, I arrived at the position that what we basically need is to be a "Responsible Human in the Loop (RHITL)" [0]
> You may not write the code by hand but you understand it enough to investigate and fix it when it fails. It is how I think we should leverage AI instead of becoming a meat proxy.
[0]: https://raahelbaig.com/entry/responsible-human-in-the-loop/
You could argue the same with high-level vs low-level programming languages. But I do get that English is so abstracted and non-deterministic that we're kinda clueless of what's going on. At the same time, I do think that maintaining code does require some level of understanding. And unless you straight up one-shot something perfectly, you still want to test manually and tweak with a few more prompts, adding a level of 'how does this work?'.
This is also freeing up time to explore things that before you wouldn't have been able to even start. New fields in tech are opening up. It's all about the model, compute, plugins, third parties... and more to come!
And this is not limited to code, but also how the world works. It's all being abstracted into prompts. Funnily enough, I'm also learning that way...just taking less time to get to the point. But this is not the first time we go through this. Eg. Google vs a library. And like anything, if no one knows anything anymore how do we distinguish from one another? There's a level of wanting to understand in order to distinguish ourselves from the rest in the serendipity of everyday life.
Call me naive, but something tells me we're going to start being much more open to just exploring the world with all this time we just bought ourselves thanks to technology. We were always gonna get to this point and there'll undeniably be bumps ahead.
I'm curious how much people who code this way now are spending in tokens every month? Where I work currently, we operate on a $200/month token usage limit. For me personally, this has prevented me from just endlessly prompting claude to fix bugs.
Even with unlimited spend, it seems immensely beneficial to dig into the code base and fix a certain amount of bugs oneself. Oftentimes this is ends up being quicker than having to type out a detailed explanation of an issue in plain language, with the added benefit of maintaining intimate knowledge of the code base.
The problem is not eating McDonald's for every meal. It's that we've forgotten how to make healthy choices.
You could say the same thing about the transition from assembler to C in the 1970s and early-1980s, albeit that was at vastly smaller scale of impact. It’s not that nobody knows _anything_. We still direct the machines, just in a different way. And if what comes out the other side satisfies our needs, does it matter what lies beneath?
Tail risks have always existed in software development. The tail risk of a bug introduced by some dev who quit five years ago is similar to the tail risk of a bug introduced by Claude six months ago. Deal with it by building better visibility into how your systems work. Demand that your agents write good documentation to accompany their code-writing.
If you’re doing it right these days, it means you’re thinking of a much bigger picture and containing downside risks as boldly as you’re expanding the frontier of upside opportunities.
Feel like this has always been a problem, especially with large legacy codebases. Very few people ever knew everything. We just reach the problem faster now; it used to take years of churn and turnover.
How can one learn system architecture while mainly vibe coding?
I'm not convinced that this matters anymore either. Knowing architecture was a vibe coding strength about three months ago. The modern models seem to handle it very well.
"Nobody is resolving bugs" is weird. Coding agents are great at resolving bugs! So much of what I see posted here seems more about how coding agents are being misused rather than anything inherent to them.
This problem has existed long before AI became useful. As technology becomes more accessible, less knowledge is required to operate it. As that knowledge becomes less necessary, fewer people acquire it. Eventually the abstraction becomes so effective that entire layers of knowledge disappears from common practice.
As a person who's a solid generalist with over 30 years in various roles, I am completely and utterly shocked at how little foundational knowledge people in "senior" roles possess across a wide variety of technical fields. I'm often treated like some wizard or oracle for knowing things that everybody in the field used to know, I just haven't retired yet.
AI didn't create this phenomenon, it's just the latest (and probably the fastest) iteration of it. I recall another particularly large iteration happened when Windows NT 4.0 Server saw mass adoption. Suddenly, people who were effectively IT technicians were now sysadmins.
Edit: grammar
When you simply large amounts of information, the knowledge probably shifts to upper level reasoning, then complex reasoning develops on top of it again. But who knows.
Why was the title of the HN post changed from the actual title of the blog post? I thought it was clear in its meaning.
Agent Smith had it right
I think there is definitely gonna be a shrinking of critical thinking as lots of people are gonna be lazy.
Has t this sort of thought been the constant refrain of every generation at the advent of new technology?
The problem is always maintainability, which can only be achieved through human understanding of code.
Companies will pay hard and high...
The Computer Chronicles – Word Processing (1983)
https://www.youtube.com/watch?v=Jt0OoXluC8g at 4:08:
Writers disagree on what effect word processing will have on the quality of our written language.
Some writers are concerned that computer assistance may promote dry bland writing.pretty AI-sympathetic vibes in this thread. I generally agree with the author despite everyones opinion (including my own) in this thread that AI can produce great code. Just becuase the code is good doesnt absolve the human-in-the-loop from understanding and clearly documenting and communicating what the system does.
At the end of the day I'm not paying an engineer to send me claude all day. I'm paying someone to become an expert on a system, even if AI assisted. The product will be better if i have someone that deeply understands the system and where to point AI to. Someone that can understand the full big picture can also anticipate future needs - something AI cannot do at all.
I'm in EE/embedded/FPGA work and you can make an absolute hell of a mess with AI in that world, so maybe everyones opinions are more based around front end software or something. I think people forget there are fields that do not have the huge data training base that frontend/backend software does. a large majority of good designs in FPGA are proprietary at big defense corps, not on stackoverflow and github.
If you don't understand how your system works, your ability to make good decisions about future work on that system quickly degrades.
I don't think you need to review every line of code, but you absolutely do need to be able to describe how the system works and its high level structure.
As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here. You are responsible for a large system that has been worked on by multiple different collaborator (both human and agentic). You need to be able to make smart, informed decisions about that system, and talk with credibility to other stakeholders about what it can and cannot do and sensible next steps for the project.
In some ways I'm sympathetic to this: after all I'm trying to make money by being that person who does know stuff. But honestly that capability never seemed to be terribly valued by the people with the money. Anyway, it would be nice to see some examples where AI tools didn't "know" about architecture or intent. My experience has been they know plenty. More than most human developers. They do tend to follow certain fashions, which can be overridden by debate, or by introducing new fashion via skill text. But that's today. Surely these tools are only going to get much better in the next 10-20 years. I'm skeptical there's going to be a line of thinking that "it's smart but doesn't know the things humans know" long term. Whatever it is that we think only humans know today will be learned by the tools tomorrow.
>We’ve had to know everything about the product/business from day 1
I know this is being hyperbolic but I thought this was an odd post to include. I've met plenty of data engineers that don't have great knowledge of the business/product and SWEs that do have that. ¯\_(ツ)_/¯
I predict we will see more and more of these convoluted rationales (i.e. excuses). All these basically boil down to: “I know AI is crap, but I have found a way to make it useful. Trust me.”
I am gonna appeal to Occam’s razor here and say, the integral variable here is AI, and the only variable you need to know is AI. The problem is AI.
I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.
The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.
The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.
The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.
This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.
We were not building the feature because we wanted it. We were building it because we thought other people expected it.
Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.
Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.
Where does the market's "action" come from? What is the driver?
Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.
This left me lost.
Nobody in this ecosystem, I thought, is actually in control here.
Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.
So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.
It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.
Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.