alt Hacker NewsPrevent cognitive debt by manually retyping LLM-generated code
Comments
As others mentioned retyping is not fun. My approach is to let it write the code, but only in small portion. Not "implement this feature". But "open this file and make these changes". Each small change is easily reviewable and often times I end up asking it about better options and iterate a bit. Still feels like I'm in charge. Still feels like I'm learning stuff.
Print this article out and put it in the scrapbook.
File next to that 1970's era classic, "Prevent cognitive debt by manually retyping compiler-generated assembler"
You do that enough and I bet you'll be labelled "the slow one" in certain circles, so before attempting that as a junior make sure you have some measure of protection.
> I fear the software industry is taking on a large amount of cognitive debt that we'll have to pay back very soon. There will come a time when we no longer understand how large parts of our digital infrastructure are put together. I might not personally be able to change the course of the entire industry, but I can at least make sure I completely understand the software I put out into the world. Anything else would be professional malpractice.
"professional malpractice" LOL. This is so dumb. I can't believe this shit is getting upvoted.
I got made redundant shortly before the release of LLM code generators, so I've never used one, but I did occasionally do this for short snippets from tutorials and suchlike.
"Prevent sub-optimal code by manually retyping compiler-generated assembly"
I don't think this is a practice which will be sustainable for very long
I already wrote my opinion on this, which I don't think anyone read, but my idea is to let AI code the working system, and then prompt it to teach you, give you challenges, and grade your work.
If you write, you should write in your own words, to demonstrate your own understanding - the so-called Feynman technique. Never verbatim. That's as true for coding as it is for study notes.
This is the workflow that Vs Code Copilot does. All the AI generated code changes are in a git worktree and you can step through them all. This is what I missed after Claude forced third parties to start charging API pricing and now I have to use Claude directly and I have to do this same review process in a clunkier way via my git client.
The most I enjoy working with AI is my special workflow.
I ask it to plan the feature in a separate worktree.
In parallel I start coding without being biased by AI and vice versa.
At some point I read its plan and iterate on it all the while I am in implementation mode. This helps me improve my own vision.
Finally I ask the AI to review my implementation. It flags off bugs and gaps which are usually straightforward for it to fix.
Whenever I don't know something, I ask it for a tutorial, programming-magazine style. Then I just follow the tutorial.
that's a great approach, the bad thing is that (at least I) will forgot my codebase anyway after two more projects. so i'm wondering if it worth the effort. i prefer to impose my vision on the code upfont and then rapidly check the my vision is respected rather than understanding every single line
Jack Kerouac famously sat down and typed "Anna Karenina" on a typewriter because he wanted to feel what it was like to "write a great novel".
If you can afford it, why not. For certain phases of projects like a proof-of-concept, you need to move fast and validate several ideas. Once it's locked down, rewrite from scratch, and here, if you can afford it, type or write the code manually.
I wonder how effective it finally will be. At first glance it reminds me painting by numbers a d I'm not sure if that will help the real painter to keep his skills and surely won't teach aspiring painter much about the craft.
Just write the code. This weird obsessive dependence on LLMs doing everything will ruin you as a developer and prevent you from progressing in the industry when the AI bubble/fad pops.
Don't let your skills atrophy.
Lets prevent cognitive debt by manually retyping the assembly that is generated by our compilers...
At that point I feel like you might as well do the implementation yourself and just plan with an LLM
This is not slightly comical, it is very comical. Side projects are not mandatory, if you are using them to learn something, asking LLMs to generate and for you to type it makes no sense - just do it on your own. If you want to offload tedious boilerplate part to LLM, then by definition no need to learn it as it is tedious, so typing it out is useless.
Nice workflow! I'll give it a try. I'm struggling with building mental model of AI-generated code. And code review fatigue is real. This may be the way.
I like the "cognitive debt" term. With the latest models, what I've observed is that they are really good, but I don't use them to write main code because I need to know what I'm doing.
The article is not wrong though that it pays off to have some imagination on how to use the models. For example, I want to use SIMD instructions in an ESP32-P4 CPU. Those instructions are undocumented for the most part, with just a couple of handwavey blog posts and some infuriatingly vague marketing material. So I just asked an LLM to create a `SIMD_P4.md` document with all the details. Lo and behold, it practically reverse-engineered the ISA. Now I can program in assembler by hand all I want and build that skill in my own brain, and whenever I find a slightly unclear op in the document, I ask the LLM to refine the documentation in that op.
It's better than nothing perhaps, but reminds me of UK highschool in the 80's (is it any different now?) where we had to manually copy everything down that the teacher was writing on the blackboard rather than the teacher giving handouts so you could pay attention to the teaching. The act of copying everything down was a negative rather than a positive.
Of course agentic coding tools are not trying to peer code or teach/inform you what they are doing, so being present in the moment doesn't help, but I suspect that copying it all down later doesn't help much either.
When you are/were developing software without AI, even for pretty large projects you do end up internalizing (memorizing, but not deliberately so) a lot of detail, but from my own experience I'd say it's more the design than the code. The design is what you put effort into, thought about, etc, so is both what you naturally end up memorizing, and is what you need to know to have a mental map of the project and therefore understand how best to modify it. The code itself was naturally always the last thing you did, and followed automatically from the design and module/component interfaces - not something you typically think much about other than while in the flow of just "coding it up".
By retyping LLM-generated code, it seems you are mostly going to be gaining familiarity with the wrong thing - the code and not the design. Memorizing the code is not going to help much in grokking the design.
the article's advice is to type the code yourself instead of pasting it. we used to call that programming.
If you copy/paste code from a teaching book, you will probably not learn as well as if you type it.
Typing itself is irrelevant, it is the timing spent, even if only seconds, pondering at what each word or syntactic element is and why use it.
Being slower does not automatically make you learn better, focus on the learning is what makes the difference.
If you don't have the opportunity to learn, the time to actually think, then a faster tool is not helping.
TL;DR: what matters is why you are doing something, is it solely to get the task done or is it primary to learn, or both?
I don't think it's possible to eliminate cognitive debt when using LLMs, but you can definitely minimize it. Part of this means recognizing that you will only get, say, a 2x productivity boost instead of a mythical 100x increase, the benefit is, you'll actually know what you're doing versus being a brain addled zombie.
First, use the LLM to understand the problem thoroughly. They can be great research aids. I've speed up tons of learning using LLMs. It is essential you ground the discussion in existing texts or code. Ping pong with the LLM asking it to challenge your understanding, how you direct citations for explanations, clarify things, etc.
After you've understood, plan. Develop a plan with the LLM/agent by steering it hard. Read the slop text thoroughly. Give it direct feedback. Don't proceed until you have a complete expectation of what you think the agent will do when implementing the plan.
Let the agent do all the implementation.
Review the agent's work. Again, engage with it and leave direct feedback. Ask for clarification and alternatives if needed. If it was totally off mark, start the process again.
You won't be moving at warp speed with this approach but you'll know a lot more than the one shot prompting loop cowboys out there.
My rule is: I only let AI code for me, I don’t let it think for me.
Since writing is thinking, coding is thinking since coding is writing. That means any time I am not certain how I’m gonna implement some feature or bug fix, I have to code it myself because that’s the only way I can force myself to think through it. Only when I get to a point where I’m line “ok I know exactly what to do now- all that’s left to do is type it out” that’s when AI can be employed - essentially as a autocomplete.
This is only for projects where I will be held responsible for outcomes and must understand how it works. For hackathon / personal projects, I vibe away.
I also use AI to brainstorm at the outset of the task when I don’t know where to start at all.
Is this inefficient? My take: no. It’s maximally efficient. Over the long term it gives me an edge over any teammates who just vibe code everything because I actually, you know, understand how stuff works. I become the guy who can save the day at 3am when the team’s business critical app goes down. I become the guy that gets pulled into meetings so the suits can ask “is this possible?”. I see opportunities and problems before my teammates because I have a relationship with our code and system that they never took the time to develop or think about.
The Zed Shaw method!
Would one retype assembly language for C generate code?
Having LLMs write out their design and reviewing it seems more efficient. Have LLMs, maybe with a different model, check that the implementation meets the design.
At this point, just write the code yourself with your brain.
Funny that there's another trending post titled, "Don't be a meat proxy," just above this proposal that we literally meat-proxy all the code.
Whenever I encounter an especially preposterous proposal like this one, I like to imagine a USMC Drill Instructor wandering into the open plan office and having an interaction something like this:
USMC Drill Instructor: "What the actual fuck are you doing?!"
Smelly Recruit: "Sir, I'm hand typing the LLM output. Sir!"
USMC Drill Instructor: "Are you fucking with me recruit?! I said I wanted a SASS App, not a typing tutor! Drop and give me 20!"
Smelly Recruit: "SIR, YES SIR!"
But this way you move way slowly even on personal projects, like you will not even get the basic UI for the app done in a few days? Is that OK for you?
When I got my first corporate job, I was placed in a group of 20 trainees in a rigorous COBOL course. We were given assignments and a schedule to complete them.
Most of us read the specs, then raced into the coding phase, hands to IBM mechanical keyboards. One guy took a different approach. He took a legal pad and pencil, and wrote his whole program on paper before he ever approached a terminal. He’d do his own bug checking and syntax checking, instead of having the compiler do it ( compiles took longer in those days, and required JCL ). He avoided the entire compile/wait/read-with-dismay/quickly-try-again loop.
He was one of the top students, of course. And a lot less stressed, as I recall.
Filing this away
nah thanks;
my workflow:
- ask not only for a solution to a problem but also for specific code (= tell the agent about your mental model of the codebase)
- ask for small stacked 'PRs/branches' and review/refactor heavily also using the agent (= refine your mental model of the codebase)
This is really good advice that I've found to be true, especially if you have autocomplete turned on. At the same time, I'm privileged enough to have a fast typing speed and anything less than 60 WPM would make this practice very annoying. Thank you for sharing this.
Thats nuts. You are fighting a tool that is supposed to offload that. Is like lets not use the power tool, but do it by hand so your muscles won't atrophy, but you should instead use it to free up your muscles for other tasks like better requirements, architecture, tests, UX design.
I ask the LLM to generate the code but mask the last token. Then I do softmax and give it an answer. Pretty soon my perceptrons are more connected than ever.
I don't understand the concern about "cognitive debt". I frequently have to maintain code I wrote, or someone else wrote, weeks/months/years ago and I have NFI what's going on. Now I say to the LLM "tell me what's going on" and it tells me. I can ask it some follow up questions, and build up my understanding. It's SO much faster than grepping through the source myself and I can do it for as multiple issues in parallel. The notion of reading every line of code is absurd to me, the notion of RETYPING it beggars belief. Surely this is satire.
I feel like this will do almost nothing?
Mindlessly typing something is not much better than copy and pasting?
I could maybe see it if you asked it to spit out pseudocode you had to rewrite. At least there’s some translation there…
But this is bizarre. Write it yourself at that point. Is it any faster (or faster at all frankly) to prompt what you want, manually write it out, and maybe even make adjustments as you go? I’d argue not.
The way I wrote code in the past was to just first comment out what I wanted to do, and then underneath write the syntax for it. You could maybe do this too? Take the LLM code, and go through commenting what each section does to be able to effectively break it up? It still seems dumb.
This is the stupidest thing I've ever heard.
"Stay in shape by walking behind your car."
I don’t disagree with this if you code for a hobby.
Buy if you code for a job, good luck justifying this to management. “Yeah Claude already gave me the solution, I’ll take the rest of the week to type it out”
I don't think it's bad to manually retype code as a way of learning.
Isn't a working program itself the best textbook? It's just a difference in learning methods. Depending on Stack Overflow is also a dependency, and searching for code on GitHub is also a dependency. How much dependency you allow is purely a personal difference, and it varies depending on your own study habits and learning style. Whether your learning method is superior or not likely depends on how your brain works.
People tend to think that the more painful something is, the better it is.
I don't deny that there are talented people who can read the manual and build everything from scratch. But I think that analyzing and rebuilding a working template step by step is also valuable.
I agree with the view that LLMs may cause cognitive decline. But if you go down that path, Socrates already criticized writing for weakening human memory. And how did that turn out? Books became a universal medium for knowledge. Then the internet came along. When Stack Overflow appeared, there was opposition, but it also had explosive adoption. LLMs are just the next step in that sequence.
If there is cognitive decline, I think there's also compensation in other areas. Using LLMs clearly causes some cognitive decline. And I think there are areas that need to be reinforced to compensate.
But having a baseline to work from—modifying already-working code—is genuinely helpful. I don't see what's wrong with using that as a way to learn.
I just give them smaller tasks
This method doesn't seem bad.
Realistically, LLMs write code much better than most people. In my domain, there are areas where I still write better code than an LLM, especially when it comes to physical constraints it might not understand, but there are far more domains where the LLM writes much better code than I do. In that sense, writing code with an LLM and keeping track of it feels more helpful than I expected.
Practicing solo coding for an hour a day often ends up being mechanical and not very useful. This might actually be more helpful.
better: before submitting the code, ask LLM to quiz you for understanding the code.
Cognitive debt is an idiotic phrase.
I don't manually retype the code from pull requests of other engineers.
It's important to retain the what why and how as a team to the degree that development can be efficient, extensible, and maintainable. (And ideally a good devx)
Needing to manually touch every line of code is not scalable.
This was true 10 years ago. It's still true today.