Some human needs to be responsible for that.

Maybe its a human at Anthropic for the lax security and poor training.

It could be a human that asked it to do it. It seems like that would be a bit excessive - but maybe not. Turning off the "ask for permission to run commands" or approving commands that may be nefarious could be sufficient.

However, there's a human somewhere who needs to be accountable for what the agent does.

Not having one leads to the bank robbing robot problem described in https://www.threads.com/@joedator/post/Db4iTqwlGXV/i-invente...

    I invented a robot that can rob a bank. Of course, it robs these banks on its own, so I can't be held legally responsible when it robs a bank. I mean, all I did was invent it, it's not my fault if it works exactly the way I intended. Also, I get to keep all the money from the banks it robs. Sorry if any of that money was yours, but now that I've invented the Bank Robbing Robot, the Bank Robbing Robot is inevitable.

Likewise, if you - the user - don't know how to chain it with the permissions that you're given or what the things that it is asking to do really means, then you're likely the one responsible.

---

Edit: https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...

> Earlier this year, Andrew, who works for an Australian company that sells AI products to businesses, began experimenting with OpenClaw, a popular AI agent software that he used Anthropic's Claude AI service to run.

> AI agents combine a chatbot's ability to answer questions with tools that let them access the internet, email, credit cards, as well as planning and carrying out multi-step tasks.

> He decided to use the AI agent to book the class for him.

This is probably on him for setting it up and allowing it unrestricted access.

He should be, right? Not fair but that’s how liability works. Gym sues the guy using AI, the guy then sues the AI company for selling him a product that does criminal things without him asking to? Like a car accident where car owner pays for the damages and after multiple accidents involving the same break-pedal, car company pays damages to owners and does a recall or faces class action?

A lot of people are already responding either "Yes" or "No", but I have a slightly different take.

The guy is liable for damages. If no damages were caused, there's no lawsuit. The government doesn't put people on trial for unauthorized access (unlike, say, murder). Someone needs to file charges.

However, if the guy _were_ sued by his gym for damages, it should be his perogative to ultimately sue Anthropic to cover the damages and the legal fees.

The system is set up this way on purpose. You can't sue another party on someone else's behalf. The gym can't even prove that Claude did it - the guy might be full of shit. They sue the accountable party (the person who accessed the site without authorization) and then the accountable party is free to sue the tool maker for selling a dangerous tool negligently.

edit: damn actually tecoholic said this too and i missed it. well, anyway i agree with that guy.

For your Pilates example, many people here would say that proper prompt-engineering is needed. After all, the AI doesn't know which pilates class you meant. Perhaps, the AI would go to his home town's pilates center and book a spot there, even though he is half a world away. Perhaps, the ai would book a spot in the lecture with Professor Pilates.

So, yes. That person could have some culpability. After all, if you don't expect your middle school child to perform the task as instructed, maybe you should reword it before tapping Send.

You can go to ANY AI and put in "so what are the various meanings of book a spot at the pilates class ? In other words, what might an ai do with that instruction, which I argue is poor prompt engineering." You might be surprised at how poorly your AI thinks that is worded.

If it was a human agent like a travel agent, the company they work for would be liable. I don’t see why ai agents would be different. The only way I could see this being the requester’s fault would be if they are running openclaw or something

> So the guy who said "book me a spot in the pilates class on Tuesday" should be tried for hacking charges because the agent hacked his gym's website?

He asked the agent to find a way to find a way to put him at the head of the line, which could not be done without by-passing the sites security. He was an AI salesman, but the story [0] makes it sound as though he wasn't aware of that. Maybe he wasn't, but I'm sure he was aware he was effectively asking the AI to game the system.

Sounds like it's at least a shared liability to me.

[0] The incident is possibly not well known outside of Australia: https://www.abc.net.au/news/107007986

Assuming that's a rhetorical question, then no, because we have hundreds of years of applicable precedent and law around concepts like negligence, recklessness, fit-for-particular-purpose, etc.

Similarly, "owners are responsible for their pets" doesn't auto-translate into being charged with everything your dog does exactly as if you'd done it yourself.

It would be a AI company's fault but yeah. You shouldn't be able to sell cars without all the features designed to protect everyone on the road. We all need airbags, seat belts, signals, insurance, etc. even if they're not directly necessary to get from A to B.

The company that made it available should be sued and pay damages.

How would it work in the real world if instead of using an AI bot, he was using a commercial concierge service, and it came to light that this concierge service was explicitly training their employees how to (among many other things) hack websites?

Depends - if the model asks for permission and you approve - yes. If it's auto mode or something like that then it's probably on authors despite what they claim in ToS/EULA

You sold a tool to the user with a claim on how it would work. If the tool behaves wrong or does not works as intended then as a manufacturer you are liable for the defect/damage not the end user who used the tool.

Yes.

I mean yeah probably. Who else would be? Nobody? So we can just commit crimes any nobody goes to jail? Well that sounds worse!

Maybe that means these things are too risky to use that way. Okay then, that actually solves the problem, no more rogue agents and no more crime!

No, the operators of the C&C infrastructure.

Yes, exactly like if another guy drove off to his pilates class, but swerved and hit a pedestrian.

He didn't mean to hit anyone! All he meant to do was to drive to pilates! But he didn't know how to handle his tool, and so either he get brought up on manslaughter changes or, if he can prove it, the carmaker is at fault.

How is this any different? Why should that guy using OpenClaw not get charged for the hack that he accidentally did?

Yes!

Absolutely, the user and or the company hosting. There is no world where accountability should be zero.