alt Hacker News> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
Replies
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
In the Linux kernel I've had a lot of luck with:
1. "find the code that does X"
2. go read that code
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.