I completely agree. While the best outcome may obviously involve lots of thinking, from what I've seen an experienced, the AI tools we have today put people on a slope to becoming "meat proxies". Some of the brighter, stronger engineers may have a less steep slope, but I'm convinced that with time they too will be meat proxies. AI is attacking on too many fronts for it to be otherwise:

  - AI generates too much code that overwhelms humans
  - AI deskills users concerning the gritty details of coding by having them not do it themselves
  - Having most of one's co-workers become meat proxy will put pressure for others to convert
  - Lack of effort or expertise invested into codebase leads people to care less; caring less -> not caring about optimal outcome -> not expending brain power on problems we don't care about.

We are clearly getting to the point, or past the point, where humans will not understand the code base very well or at all. I fear we are building a tower of "it kind of works this way" (maybe it does or maybe it doesn't), in a couple years people won't know what they can change, they won't get much of the architecture.

If one can't understand the monstrosity that was built up over time, where the code base is enormous, millions and millions of lines and components, all with a few buggy corners that the tests didn't touch, and a human didn't have the insight to write a test that understands and verifies the corner cases, it will be like a magic machine that we can't change because we don't understand it and we can't afford years of effort. Some may hope their LLM will be able to understand it.

We'll have things that are buggy in ways that become very difficult to understand or fix, and no one can afford the effort. Remember when we had all those huge old programs from the 60s or early 70s, before there were tests maybe, or unit tests. Code written at a lot of human effort, but before the ideas of software engineering made it more tractable to build large programs reliably, testable.

We are creating a new problem for the people that take over our new code, just like the "vast for the time" 1960s huge cobol program can't be changed.

Some applications (e.g. a kernel) need to be carefully engineered with every possible edge case considered. Some apps (e.g. a web game) don't. This is a way for careful engineers who need to build dependable software to integrate AI without compromising quality.