It feels like a fundamental problem. The more you restrict the agents, the less useful they become. The less you restrict them, the more dangerous they become.
Human coders and sys admins routinely make catastrophic mistakes, too, so it's not like we're facing a novel problem.
It makes sense, but a good system where you can say "give this VM access to repo X, branch Y, and server Z for the next 4 hours" without leaking any secrets into the VM would be a major improvement already.