> I think where people get worked up about this, is that it's not easy to tell what understanding other folks are taking away from their AI usage.

Alongside understanding, you also loose taste and perception. You can still say whether some things is good or bad, especially if you interact with it, but you can no longer articulate why. And with perception loss, you can no longer preempt a bad situation as you lose your awareness of things.

So for you a car may be a vehicle that you drive around. But for a mechanics it's a much complex system. When something doesn't work, you will lack the vocabulary to express what's wrong. And even when everything works, you may not be able to explain how.

The thing is, LLM are very buggy translators. Agents in software development help because they provide a way for LLMs to verify their output. But correctness is very hard to express. OK for a simpler task, but not really for a complex systems. Just like fluency in a foreign language is not about knowing the rules of grammar and a lots of words, but also knowing the idioms and other figure of speech.