alt Hacker News> We can trust humans not to do stupid things. hold my beer
I can definitely delete a thousand items with a typo in my bash for loop/pipe. You should always defend against stupid or evil users or agents. If your documents are important, set up workflows and access to prevent destructive actions in the first place. Not every employee needs full root access to the billing system; they need readonly access to their records at most.
These people aren’t doing bash loops, they’re regular non-technical people who just want to use an AI Agent to access services and aggregate data.
If people accidentally delete stuff, they tend to notice it and we can roll back. If an agent does a big whoops, it’s usually BIG one and nobody notices because it’s just humming away processing stuff with little output.
An accountant might have access to 5 different clients accounts, they need to do their work. They can, with their brain, figure out which one they’re processing and keep them separate.
An AI with the same access via MCP might just decide to “quickly fix” the same issue in all 5 accounts to be helpful. Actually breaking 7 different laws in the process.
See the issue here?
(Yes the AI is approved for this use; that’s not the problem here)