One of the most wild parts of the "Is AI useful" arguments is how different people are getting so astoundingly different results from it. One guy will claim that he can one-shot "Port this unmaintained app from 2017 to modern Android," go do the dishes, and come back to a working application. And the next guy will say, "I can't get it to do anything groundbreaking and have to micro-babysit it every step of the way to the point where it is more work." Assuming both people are truthful and operating in good faith, how can this be explained? How can simply writing a prompt in English be a "skill issue?"

I'm personally much more towards the "I struggle to get it to work well" side, but there have been so many anecdotes from people who are earnestly saying it does all their work for them and that they hit enter, go to bed, and wake up to working software. I'm trying to imagine a hammer or band saw that produces such wildly different results in the hands of different users.

that’s perhaps a bit harsh, and I’m late (intentionally) to all this as I’ve worried about personal atrophy, security, and feeding the shoggoth

but also with my instructions it will churn on prompts for hours. It makes use of my local rags by domain (which it also helped me build and maintain). Most recent example: it’s been on my list to set up headscale for like 5 years now. I know how to do it, I’ve read the docs and kept putting it off, partially cuz Tailscale works and has a generous free tier, and partially because I didn’t want to strand assets that I’d somewhat shortsightedly hardened to only accept ssh from my Tailnet.

claude did it all in 3 hours, while I worked on other stuff, prompting for `op` access when it needed keys, and logging each usage, with minimal involvement from me. All with more documentation, roll back points, and “learnings” propagated to the appropriate rags than I could have created in a weekend session. It made mistakes (some that I likely would have made, like region settings) then reasoned about it, corrected and tested, all transparently, all git backed, all posted about on my Mattermost server as it went, so I can easily search it the way I would of an extremely detailed teammate did it.

I don’t know what to make of it all, but letting it churn through my backlog has been eye-opening.

"it must be some sort of weird badge of honor to be so profoundly incommunicative that you can't get the ai that desperately wants to sycophantically pump you up about all your brilliant ideas to be that useless. that is genuinely harder to do than simply using the technology even passably."

Definitely incomprehensible that someone might have a different take than you about the 'goodness' of your prompt outcome...

I mostly view AI as raising me up one more level of abstraction in coding. It still requires babysitting and verification if you want anything vaguely maintainable that's not pulling in a ton of irrelevant libraries, but now I am able to give a clear spec for a ~50-100 line task and it'll get it 90% right, and it's a small enough chunk I can validate it well.

Claude though is still slow enough that I think it works out to maybe a 50% improvement at my time to do a task, but sacrificing deep knowledge and learning. I worry that I'll never develop the same skills and intuition that senior engineers developed a decade ago because I'm outsourcing a lot of the planning and thinking.

Here's the thing. I've been an AI denier and skeptic since the beginning. I've softened. This is my reality on the ground:

I am tech lead on a front end project that is typescript with react and everything else as bog standard as possible. In the last year the backend devs have been pressured to be full stack, so have been submitting AI PRs to my repo with Claude. The PRs mostly 'work', but invariably have major problems that wouldn't make it through a code review in the before times.

Front end JS on a CRUD app is the literal best scenario for AI. Professional senior devs are using it, and it can kind of work, but still not really.

I've given it the old college try myself recently, and it is at the same time astonishing and disappointing. Yeah it can do a lot, but there's always something wrong with it. The 'something' is not always a bug; it can be a design issue, performance issue, unused code, duplicate code, etc, thus many people don't notice the problems. But I think for those of us who prided themselves on quality, the stuff doesn't work well enough to use. For those who leaned more towards speed, they're overawed. I am concerned for my career, yes.