It's a good point but in most of the places i've worked an average person could leverage any 2026 llm to improve things 50% easily. And I'm more anti-llm than not somehow.

    - fixing regular computers issues (confusing browser or os behavior)
    - finding more possibilities with your current programs (around 2020 someone once asked me if I knew a way to capitalize a word on a computer, when she realized there was a dedicated shortcut she became ecstatic)
    - having a rubberduck to discuss your daily work dead-ends
    - thus decoupling from team friction as you need to ask less questions to people that might hurt you.

On the other hand, if humans could apparently do that work without automated verification, then perhaps so can agents.

> no way of knowing if the AI is bullshitting them

Back testing, hold-out sets, cross validation.

This has all happened before.

> they have no way of knowing if the AI is bullshitting them

That hasn’t stopped Tesla. Their self-driving cars work by connecting the car’s controls directly to a neural network.

There are many tasks humans perform which have no or slow feedback loop. Even if they can audit/validate AI output, they don't know if it will be good when it reaches other humans.

test suites cover nowhere near enough to assure reliability

and if they did for your system, then it's so simple that whether or not you used AI doesn't even really matter

right cause test suites never have issues