alt Hacker NewsMy comment wasn't out of left field. People are responsible for what their agents do. If enforcing that responsibility destroys "99%" of the economic value of these things, then that's what will have to happen. Personally I highly doubt that 99% is the right number, but yes, if enforcing laws about what people are allowed to do with very powerful software eats 99% of the value of that software, then it is what it is. We aren't going to end up in a situation where people are allowed to have their software do whatever it wants because of this loophole that "it wasn't me, it was a rogue agent!". It's silly and naive to think this would be the equilibrium that is reached here.
If you, like me, want to keep most or all of that 99% of economic value, then you should be interested in the ongoing development of security for agents. But just "this is how it's going to be, so oh well!" is not a serious stance, IMO.
Maybe my comment wasn't clear as to who has to make this decision.
You're imagining that both the AI labs and the hospital victims are on the same team or somehow coordinating. They're not.
Hospitals can (and should) put up defenses, sure. But hospitals are not AI labs. AI labs are AI labs, and accordingly they are much more capable attackers than hospitals are defenders.
Hospitals cannot spend arbitrary amounts of money defending themselves from proliferating nation-state level attackers. AI labs need to prevent their tools from being used in this way. However, the only way to do that is for AI labs to neuter their tools and destroy the vast majority of the economic value. Achieving this will certainly not be easy.
I'm not proposing that we just give up or don't try to solve this. I'm saying it will not be easy.