Yeah, anyone who has had to manage developers in a big company will be able to tell you that it is a constant struggle between giving them freedom to work independently and get things done vs. limiting their access to prevent potential bad stuff from happening. In practice, the more bad stuff happens (and the more consultants you allow in), the more hurdles are introduced. Eventually even trivial tasks become a corporate nightmare of approvals. You can't have the conveniences of both work philosophies at the same time. Even the smartest AI agents won't change that fact. It simply limits their work speed to the speed of the approval process.
Well, there's definitively a pareto frontier of freedom and avoiding bad stuff. And many companies ain't on it.
(In practice, I found Google to be better at having (internal) security controls that are reliably but also don't get in the way than eg banks I worked at.)
the difference being, that if you build those kind of approval processes up with AI agents, the system will function a bit faster. you could have a web reading agent. a web writing agent. a coding agent. a credentials agent. etc. and have everyone make these formal requests.
it will catch quite some prompt injections, as the prompt injection has to promptinject a secondary system in order to be successful.
> the more bad stuff happens (and the more consultants you allow in)
In your parenthetical, you just said the same thing using different words
Humans are vulnerable to scams, cons, phishing and social engineering. This is a vulnerability whose root cause we not know how to fix. We can only alleviate its effects by re-engineering the environment around those vulnerable humans.
LLMs are just as gullible, we just tend to call their vulnerabilities "prompt injection", even though it's basically the same class of attack under the hood. Unlike humans, though, LLMs are a lot less diverse (you and I are using the same Opus 5.5, but we have different brains). This makes attacks on LLMs much more predictable and easier to pull off, just as a phishing campaign that targets one specific person is easier than an attack that scales to millions of diverse victims.
In the age of agents, phishing will continue to be the biggest attack vector of all, it's just the victims that will change.