Ask HN: How to deal with shipping AI-written code you don't understand?

I’m surprised I haven’t see anyone say this yet, but I don’t let the LLM design the code at all, I let it implement a design I have created.

By that I mean that, rather than let the LLM do the entire structure, I take the time to create the full hierarchy of abstractions and actors that I want my code to do. I decide, at the UML level, how abstractions will sit and interact — and then I make the LLM implement those gaps.

The key thing here is that, by doing this, I DO know how the code works. I may not know how each unit is implemented, but I know each unit’s responsibilities and how they interact with one another. I’ve found that LLMs are HORRIBLE at writing decent abstractions, regardless of how much prompting and skills I’ve given it. It will always get the job done, but it will often be unmaintainable and require bigger and heavier refactors as I introduce more and more requirement changes.

But by flexing my basic design skills, I don’t have those issues. Future refactoring fit into easy abstractions that I’ve designed. And by documenting it up front, I also have what becomes a living document on how this work will turn out.

It takes a few hours up front to plan ahead, but it makes the LLMs job easier and use less tokens, and it makes the code better long term. I don’t care if one unit is suboptimal, as long as the abstraction is right — because at worst I can change an implementation for a better one later.

Testing.

Lots and lots of testing. I usually interactively test with the LLM. Now that I'm spending less time, coding, I get to spend more time, testing.

In my experience, I test more, with an LLM, than I did, with my own code. I don't find coding errors, but I do find implementation errors.

On another note: I ran an LLM on a bunch of really mature code that I wrote, and could swear was "perfect," and it found a number of issues.

Nobody's perfect. Trust; but verify.

I’m slowly coming around to the idea that if it works, I’m good with that. I never used to personally check all my colleagues work, either, I just trusted that they’d done a good job.

That’s not to say I blindly trust an LLM - my project is an autovectorising compiler. I have it write a bazillion tests, and I have real, non-trivial programs that it compiled. I have it run a ‘gate’ before releases that takes literally hours on 5 different architectures. To say this code is tested is … masterful understatement.

I think what helps is a long planning session, going into details, working out corner cases, designing test-cases, you know work and effort. Once that’s done it’s easier to let it work through the night and do a major production feature.

I still review the code, but hand on heart I couldn’t say I’d reviewed everything, sometimes I miss something in a complicated check-in. I try to keep checkins small and contained, but sometimes that’s not easy to manage alongside the “don’t break stuff” rule as well.

So it comes down to: I’ve been working on this for 6 months or so, and so far this has worked out for me. You know, kind of like getting used to a colleague’s idiosyncrasies over a period of time.

I will say that in my experience the code quality is fairly high. Maybe that’s down to the effort put in ahead of time in the planning stages, I don’t know, but I don’t seem to suffer from the ‘slop’ that people complain of. YMMV.

This is how every product manager feels every day.

If you're an LLM prompter shipping products, you're outputs are more akin to a product managers, not an software engineer's.

I don't. I understand it, and if there's anything I don't, I ask it about it and research until I do.

It's not too dissimilar to life as a Principal level engineer pre-AI. You technically oversee the work of many engineers on their own systems, but you couldn't possibly review all the code yourself. You appreciate that human code review reduces, but does not eliminate, the number and severity of bugs shipped to production. You don't have the low-level details in your head, just the high-level ones. You design how all these engineers interact with each other, without stepping on any toes.

Time will tell whether we were right to move from a system where we trusted humans to understand the low-level system, to one where we trust fancy calculators. But so far, it's working.

I think there's a balance to be had. Leaving everything to agents when it comes to throwaway prototypes, one-off test env. scripts, extra tests I'm fine with. Doing the same with features - absolutely not - especially when it comes to financials. Seems like Meta is pressuring engineers into becoming expensive spec generators because they want to be in the cool kids club. Time will tell if that's a good play but to me it just sounds like a quick and dirty way to obliterate their engineering culture.

Ask the AI how it works!

It's totally fair to load up some open source project into an agentic IDE and instead of asking it to write something ask questions like "how does X happen when I use feature Y?" or "can you explain how client code could accomplish Z"

All the time I see an AI use a language feature or library feature that I never used before or I see some code I don't understand and it does much better than a lot of humans do at explaining what it did.

Technically it's just a role change. You're now the project manager or something, not the developer. It isn't a project manager's job to understand the code, only to ensure it does what it should and doesn't do what it shouldn't. And to ensure the developers, etc str unblocked and doing well. Embrace the new role.

I can think of so many times this has happened before. People getting used to MVC, Dependency Injection, ORMs. Sometimes the code is so large you only work on a part of it, so you have to get used to a lot of not knowing.

So how do we build confidence anyhow when we can't know everything?

I think first, standard checks - does it compile? What do static analysis tools say - code quality, passes SAST analysis? Linters? Style checkers?

Do the tests run? I'm not comfortable with the tests I write let alone the AI.. by adding fault injection (fuzzing, chaos), and mutation testing, I have a better degree of confidence in the tests and code.

Spend even more time getting to know the product instead. Maybe I don't know the code as well as I used to, but I know the product backwards and forwards. That helps me have some new confidence that I didn't really have before. Knowing the greater whole.

I lost my ability to fully explain the systems I work on around the time I transitioned from next-edit-prediction AI running in my IDE (early Copilot) and working primarily via Codex/Claude CLI's.

The further away I got from the implementation details, the fuzzier the semantics became. To the point that now I could not in good faith say I understand the system I have built from the ground-up and worked on the last ~4-5 years.

The solution to this, in my mind, is not to revert back to reviewing implementation details. The economics of that from a business/productivity perspective don't play out well.

The only viable solution in my mind is adapting code/languages themselves to the shift by making programs formally verifiable. That way you can be sure that the code "does what it says on the tin", and provably so.

I feel so strongly about this I've written on it:

https://gavinray97.github.io/blog/design-by-contract-and-eff...

I'm miles away from all of this as a non-user of AI and it seems like a mania to me.

The time that I would have spent typing in the code is now time I use to test the code. I really like to poke at things and ensure that my understanding and reality match.

I agree with the advice to design your system before implementing it. It is pretty easy to iterate on a design with your favorite coding assistant, and this is your chance to be ultra-picky and make sure you get exactly what you want. This was always where the "software engineering" was done, but in practice it's easy to have a vague idea and refine your (internal) list of requirements / tradeoffs as you type in the code. I find it harder to do that when the LLM is generating the code, just because one turn can change so much.

Keeping it under control requires discipline, just like before LLMs were a thing. You set a bunch of engineers loose on your codebase with no vision or supervision, you get spaghetti. You just get a little bit more control now. Agents don't feel shitty for the rest of the day when you tell them to do it some other way, so you are free to review as critically as you want without hurting anyone's feelings. I think it's a good thing.

The same rules from 10-20 years ago:

have an architecture

Be consistent. All parts of the app should look like all other parts of the app.

Dont allow clever code

break things into modules

do true unit testing, meaning test the smallest unit of code possible.

Well, it's simple. Don't do it.

I was struggling with this today. Working on a feature and AI review flagged something so I need to fix that. I iterated multiple times throughout the day and was completely exhausted cause each iteration was always complicated with the fix being ~1k lines. Finally landed on a one I felt at least a little comfortable submitting for review but a work day like this is super draining and I felt like crap the entire evening (I am about to go to bed, I am probably gonna have nightmares about pressing enter and enabling auto mode on plans that make no sense in a true nightmare sense)

You hit deploy and hope for the best, if it goes bad You just prompt the model again....

Just kidding, it's de same as with any other Open source plataform You install from their repo. It will be hard at the start but then You Will be very aware where it starts to fail and how to fix it.

The traditional answer is the same for engineers going into engineering management because managing a team and managing a set of agents is highly similar, only managing agents is far easier. One only ever has the illusion of control, and so you have to set your mind to containing the worst possible outcomes and looking at the net efficiency and efficacy of the team with the understanding that they will, on average, never be perfect.

I built and use https://smalldocs.org, which is a rich document format (mermaid diagrams, nice formatting, etc.) for agents. So I’ll be like “sdoc me the architectural ideas for this feature” and it helps.

Maybe a bit unorthodox but I have it plant 1-3 bugs into the code it writes (depending on the size of the MR I'm working on); they range from simple one-off issues to something more structural.

It then gives me a hint on the area or a typical code flow that would hit the bug - whichever gives me less info. I can obviously ask for more hints as I read along.

Obviously all honour-code based since I could just ask it to fix the bugs itself. I think this is something that can't be fixed systemically - either you're going to bother reading and thinking through the code or not.

Other than that I guess reading the tests - to make sure they fail without the code / fix, to make sure the setup makes sense etc.

1) Need to triple down on infrastructure (safeguards, tests, sandboxes, etc.).

2) Ask agents to do more than just code.

They need to fully demonstrate and prove that their PRs are rock solid. Ask for screenshots, videos, code reviews, detailed architecture diagrams, etc.

It would be an insane amount of work for humans, but AI can do it easily for each PR.

Merging any PR from an agent should be a no-brainer if there's real proof that the feature is built correctly and working.

If there's still doubt, then the agent needs to do a better job of proving that the feature is built correctly, doesn't introduce regressions, etc.

I've been spending a lot of time iterating on feature implementation plans with Claude (at work) and GPT (for personal projects).

I then make sure they write a lot of tests and that if functionality is changed or added/removed then tests are updated alongside.

Generally I try to review changes myself and ask it to explain the changes.

It's not perfect but I feel like the planning step being iterative isn't all too different from my old research -> think about implementation -> implement loop.

At that point, reviewing any code changes goes hand in hand with how I coded before LLMs.

The major difference is I usually have it make any changes for things I don't like.

Maybe a related question:

How do you keep the IEC 62443-4-1 auditors happy?

Test driven development and the testing pyramid.

If the agent has written tests and crucially - witnessed them fail before starting the work to make them pass - you have a lot of confidence.

Review of your own work's easier too - test and production code tend to mirror each other, giving you a much more complete picture. Confused by the production code? Go read its tests for a bit, then come back.

It's also easier to understand the change if you were there for the build, watching the transcript, checking in every so often. Doesn't have to be your primary task - leave it on a second monitor, it'll catch your eye naturally.

And finally, on top of test-driven development, have the agent first (before the first test driven development cycle) build out an end-to-end test covering the work it'll do. Once that test is green (test driven word for "is passing") you're done with your work and can clean up at will.

Play with it before shipping, get it real nice. It's never been easier to clean the tech debt up and make the UI perfect.

Eventually you'll be ready to package it up into a PR!

Now's you can leverage agents and get creative. Make your colleagues froth to review your work. Consider making a Loom video to walk them through it. Consider reworking it into a commit series; it only takes a few second for your agent and you can tell a story for humans. Maybe you want to sequence it according to your struggles; maybe it's better to show the different layers emerge with each commit - it's up to you. Consider also that your agent can easily do one-off visualizations of the change for your colleagues if you think it'll help them, and you can attach those too - mermaid charts or even whimsical AI generated images. All this is because motivation is driven entirely by other humans' motivation and you want them frothing with excitement every time you drop a review. Make it fun, it takes seconds and gets your stuff merged fast while making your teammates smile.

This is a brand new job and we will work out how to do it together :)

It’s very disconcerting. I think this will largely be solved by going all in on AI and use it to test and review the code in an adversarial manner. That won’t be cheap but it’s the only way I see this work. But a lot of companies will probably save money and use AI mainly for writing code and then skimp on review and testing. I bet we will see some huge blow ups with AI written code and hopefully the industry will learn.

It also took the industry quite a while to accept that server code needs to be secure even if that requires a lot of extra effort. AI will probably be the same.

This kinda makes me think about the transition from writing machine code by hand to using a compiler. Did people feel similar at the time that they didn't understand the machine code that the compiler was writing for them?

My Jira tickets are written by AI started just feeding the ticket into AI (shrugs)

No I only do that for unit tests so far

I went through like a mid-life crisis when the thing I learned/spent last 13 years on became meaningless.

Maybe the new meta should be reviewing new code at the end of the week/month rather than at each merge? But even that might be too much of a bottleneck. The alternative is to talk to your agent to understand changes at a high level without reading every line of code.

Peter Naur in 1985 wrote that programming is the act of building a working theory of the system in our minds, and this often takes form as tacit knowledge in the head of programmers. Even if you handed the complete codebase and perfect documentation to a completely new team, they would often need to go back to the old team and ask questions.

And before agents, we often built this working theory of the system by writing the code by hand. It helps us figure out the "grain" of the codebase so we have a better idea of how to extend it.

So what happens when agents are writing the code? The main avenue with which we built the working theory of the system in our mind is now taken away, and we're left feeling like our understanding of the system is blinded.

I ended up more or less changing my entire workflow. Four main principles:

1. Interrogation/interview-based design and specification

2. Make the decisions that matter and leave the rest to the agent.

3. Use agent to generate explainers, quizzes, and simulations.

4. Use verifiers and guardrails in the coding loop.

Interview-based design and spec

When I tell the agent what I want to build or extend, there's naturally ambiguity. But the agent knows what is ambiguous to it, so it can ask me. /grill-me is one way to do this. You'd think this would be commonplace now, but it's actually not. Some people are still wondering why they're getting bad results. It's because they're not resolving the ambiguity.

Making the decisions that matter

Not every decision matters in a product. Agents are fast at writing code and pretty good at some of the nitty gritty details, so I let it make those decisions. Also, if a particular subsystem is commonplace and undifferentiating, such as login, then I also let it make architectural decisions for that subsystem.

For decisions that are product differentiators, where that differentiation is supported by technology, I make decisions on the stack, the architecture, and the approach. This is the design phase. Because agents tend to make the mainstream decisions, they almost always take a midwit approach which doesn't leverage the technology in a way that makes a difference for the customer.

Some try to build this understanding from code reviews. I personally don't think it's the way to go, because it's not motivating. I find that I'm more motivated to learn how the system works today, because I want to change how it works tomorrow--and it sticks in my head better too (no different than pre-agents).

Generating explainers, quizzes, and simulations

As I'm doing design for an improvement and I don't know how a part of the system works, I can ask questions of the agent in chat. If I still don't get it, I can ask for diagrams (either ASCII or mermaid). But if it's an entire subsystem, I have a skill in which I can ask for visual and interactive explainers. It'll generate a markdown page (or html) that has these interactive components that help demonstrate a concept.

To help make the concept stick, there are quizzes after each section and at the end.

With the computational simulations, I generate a throwaway tool in which I can explore the state space. This doesn't happen as often, as I try to keep state spaces small, and for those explainers tend to work well.

With the design simulations, a same-model agent and a cross-model agent and I will talk about an explainer, and we'll name things that we understand or notice about the design and the intended direction we want to extend it. Then, we'll use subagents to generate options and then discuss the pros and cons. I find this is a good way for me to understand the dead-ends of a design.

Using Verifiers and Guardrails

I define guardrails as mechanistic checks that doesn't change with the feature. such as linters, typecheckers, and pre-commit hooks are all guardrails. Verifiers are mechanistic checks that will be different with each feature, such as unit/integration tests.

For more complicated states or concurrent parts, I'll ask the agent to use formal methods to write a proof. Then use the proof to generate a trace (a series of steps that demonstrate a happy or unhappy path). Then I run that trace in unit tests (kinda like property-based testing--so I guess this can be called trace-based testing), and the code has to adhere to the trace.

I also wrote a design verifier, where the requirements in the spec can be converted into a series of mechanically checkable rules. Then during testing, the verifier fires up the headless browser and uses the DOM measurements of the page to see if it matches up to the rules. This was completely impractical before agents.

All of this so a) I can be sure that the agent is doing what it's supposed to with the decisions that it's making b) I can be out of the coding loop, so that I'm not up at 2am begging, "you fixed this, but you broke that. pls fix." The agent can check for itself.

---

If any of this sounds helpful, lemme know. I've been wondering if other engineers care about understanding the code that's being written, or if people just don't care anymore. I've spoken to a few, and it seems to be split. And it may just be a matter of degree and detail. Some like it at a higher level. Some like it nitty gritty.

This is OT and a cheap shot, but if I were an engineer at Meta I would have felt pretty uncomfortable well before AI showed up...

I've been playing with this https://getchronosphere.com. It will (attempt to) rebuild a PR in a coherent journey rather than a massive blob. It's been somewhat helpful up to this point, and still improving.

Im currently building a ui framework for unreal. I dont enjoy ui work so I wanted a way to have to write less ui.

For me I dont look too closely at the implementation but I look at the api where if a human was using my framework how is the code they'd have to write and engage look at. Then I also do plenty of performance against a comparable target in my case slate (default unreal) and keep overhead minimal.

So good performance and reasonable public facing api are the 2 main pieces im focused on. Less with the internals.

Care about the seams and interaction and the things you can measure or count.

I'm currently working on a combined Aero sim/CAD/CAM software for designing, simulating, and performing DFM optimization specifically for Group 1 to 3 winged UAVs.

What I do is ask Opus/Gemini to explain the parts I don't understand. And I do some of my own research to verify accuracy of the explanations.

In addition:

* Every new feature has a Feature Design document which I review, modify, and approve. And a corresponding Implementation Plan which I also review and approve.

* Every fix has a RCA document and fix Implementation Doc which I review, modify, and approve.

* There's a global high level Architecture and Design document with additional sub component design documents which are always kept in synch after a new feature or bug fix is done. Synching it is always a step in the Implementation plan.

This does adds some overhead but it allows me to have have check point gates. This has often allowed me to catch misunderstandings and align properly before any code gets edited by the AI agent.

Having the above docs also helps when it comes to referencing them as context to properly ground new features or bug fixes.

I think the solution is leaning on static types and hard to abuse abstractions to make programs correct by construction. It does mean that you’ll still have to use your brain to design the internal code architecture, and just use LLMs to fill in the boring bits between the safe abstractions.

>This new standard of shipping with AI is moving so fast that if we're going to spend our time checking each line of code, it's going to take forever.

"This new way of piling bricks is so fast that I can no longer afford to check the buildings integrity".

I just don't understand how the fuck we ended up with this narrative. Somehow the machine making things twice as fast forbids us from spending time ensuring quality? Surely if it does things faster you have more time and tools to check!

I'm half convinced this frenzy is the marketing work of AI companies at this point.

Force to always use TTD, at least validate the tests.

Recommend reading (All though a little self serving): https://www.linkedin.com/pulse/ai-code-quality-jorge-varas-a...

I shipped code I didn't understand when I copy pasted from Stackoverflow before. At a level if it works it works.

I like starting with defining outcomes, determining invariants, and then consider TLA+ or Lean models, or both. It's generally not good to ship stuff you don't understand, but if you work on big legacy code bases from 20-30 years ago, there's probably no one who understands all the code all the time.

It's at least nice to be able to prove something about the code, rather than blasting it with tests which only prove that the tests that are written do not fail (and perhaps get you some amount of code coverage — necessary but insufficient for anything like quality).

How to do all of that in your environment? That can vary greatly!

Formal verification, pick a language that has it integrated and prove relevant post conditions.

AI still does not know how to even write code that makes impossible state unpresentable so testing won’t help.

I love using AI, but any feature it creates I test the hell out of it, i become the QA testers I hated and do all sort of ways to break the app, and after 100 or so hours, I have a bug free and fully working app, now will this app scale as I add more features that to be seen.

Product doesn’t care. And most of us will get replaced by agents. I guess around 30% of the engineering workforce will keep their jobs which will be mainly to keep the product up and running, doing whatever it takes (so using agents to fix stuff basically).

The ones who get to keep their jobs will be rebranded as Maintenance engineers with a lower salary. VPs and Heads of will keep their cozy job (bc there are not so many of them), and they are not that dumb to fire themselves.

Code understanding doesn’t matter anymore (or won’t matter soon)

If you are a senior engineer then there is no need for you to be shipping code that you don't understand. People can definitely generate code a lot faster than before, but you also have a magic genie that can help you critique and improve the code that you're reviewing, including when that code was written by agents.

An experienced senior engineer with access to a coding agent is the most powerful code-understanding system that has ever existed. There's no excuse for shipping or approving anything you don't fully grok.

I had same problem, but after a while I just accepted that is inevitable

if you have to maintain it, you will still have to get an understanding of it when it breaks.

not understanding the detail is quite similar to a coworker writing the code and another coworker approving and merging the changes.

That's the neat part, you don't.

Or at least, I don't think people should.

To me this completely goes against the whole point of engineering.

IMO If you don't understand the code, then you're just a manager now. Soon your salary may also reflect as such, assuming you get to keep your job.

When you code, do you understand the asm code being built? What are the cache line access patterns? Most software developers have already given up the details of how things work, and lean on tools to handle them.

Which is fine, usually. Depends on what you are doing.

The things that always mattered were the core system, the trunk of a tree if you will. The branches less so, and the leaves not at all (though that is what people see and appreciate! The irony!).

Claude to Prod. Every day.

By taking responsibility of it, maybe by saying "Hey this is not a good idea", and if that doesn't work perhaps by switching jobs if that's an option?

The mental gymnastics developers are willing to go through just so they can avoid standing up for something is quite perplexing.

An engineer from Meta not knowing what they're doing is exactly how I imagined engineers at Meta. Thank you for confirming it's an actual reality.

On the other hand, not even surprised anymore most people even try to provide serious answers to this. Welcome to the pure insanity.