alt Hacker NewsRespectfully, it feels like your position requires a very low, if not brain-dead level of incompetence on the part of LLM users, in order for your conclusion to be correct.
My personal anecdote: I used an LLM recently to basically vibe code a password manager.
Now, I’ve been a software engineer for 20 years. I’m very familiar with the process of code review and how to dive in to someone else’s code and get a feel for what’s happening, and how to spot issues. So when I say the LLM produced thousands of lines of working code in a very short time (probably at least 10 times faster than I would have done it), you could easily point at me and say “ha, look at ninkendo, he thinks more lines of code equals better!” And walk away feeling smug. Like, in your mind perhaps you think the result is an unmaintainable mess, and that the only thing I’m gushing about is the LOC count.
But here’s the thing: it actually did a good job. I was personally reviewing the code the whole time. And believe me when I say, the resulting product is actually good. The code is readable and obvious, it put clean separation of responsibilities into different crates (I’m using rust) and it wrote tons of tests, which actually validate behavior. It’s very near the quality level of what I would have been able to do. And I’m not half bad. (I’ve been coding in rust in particular, professionally for about 2 years now, on top of the ~20 years of other professional programming experience before that.)
My takeaway is that as a professional engineer, my job is going to be shifting from doing the actual code writing, to managing an LLM as if it’s my pair programming partner and it has the keyboard. I feel sad for the loss of the actual practice of coding, but it’s all over but the mourning at this point. This tech is here to stay.
Replies
If you measure the productivity of the system that is “you, using an LLM” in terms of the rate at which you can get actually-reviewed code completed (which, based on your comment, seems to be what you were doing) that seems like a totally reasonable way of doing things. But in that case the bottleneck is probably you reviewing code, right? Which, I bet, is faster than writing code. But you probably won’t get the truly absurd superhuman speed ups.
What would you say is your multiplier, in terms of throughly reviewing code vs writing it from scratch?
I don't know if it is incompetence - if anything i doubt it, someone else pointed out that pg also used that metric and i don't think pg is incompetent. However at the same time i think it is misleading at best.
My impression is that, as someone else wrote, we do not have an actual metric for such things as productivity or quality or what have you, but some people do want to communicate that they feel (regardless of if that matches reality) using an LLM is better/faster/easier and they latch to the (wrong) assumption about more LoC == better/faster that non-programmers already believed for years (intentionally or not, they may also deluding themselves) as that is an easy path to convince them that the new toys have value that applies to the non-programmers too (note that i explicitly ignore the perspective of the "toymakers" as those have further incentives to promote their products).
Personally i also have about 2 decades of professional experience (more if counting non-professional) and i've been toying with LLMs now and then. I do find them interesting and when i use them for coding tasks, i absolutely find useful cases for them, i like to have them (where possible) write all sorts of code that i could write myself but i just don't feel like doing so and i do find them useful for stuff i'm not particularly interested in exploring but want to have anyway (usually Python stuff) and i'm sure i'll find more uses for them in the future. Depending on the case and specifics i may even say that in very particular situations i can do things faster using LLMs (though it is not a given and personally that is not much of a requirement nor something i have anywhere high in my interest when it comes to using LLMs - i'd rather have them produce better code slower, than dummy/pointless/repetitive code faster).
However one thing i never thought about was how "great" it is that they generate a lot of lines of code per whatever time interval. If anything i'd prefer it if they generated less line of code and i'd consider an LLM (or any other AI-ish system) "smarter" if they could figure out how to do that without needing hand holding from me. Because of this, i just can't see LoCs as anything but a very bad metric - which is the same as when the code is written by humans.
>this tech is here to stay
How can you say that when all these models are externally sourced by companies that actively make a loss per token? When they finally need to make a profit, how can we be sure these models as well as their owners will remain as reliable and not enshittified? Anthropic has been blacklisted in the last 24 hours so its a turbulent industry to say the least
This whole reply, and every other "anecdote" reply is more worthless than the pixels its printed on, without a link to your "actually did a good job" password manager.
(wow funny how these vibe code apps always are copies of something theres many open source versions of already)