Context overload can cause strange results, yes. Hence why a HUMAN needs to be held responsible for the output of their tools.
EVERY breakout that's hit mainstream news has been because of a single 'Security Firm', Irregular. Maybe I'm unaware of some less-headline-grabbing ones, but they all seem to stem from being 'unaware the environment wasn't sandboxed'
Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?
You keep repeating the "stupid users keep causing the problems so we punish them argument"
This doesn't work worth a shit. It especially doesn't work with things that seem safe and become wildly dangerous. In fact most governments control this by ensuring their population doesn't get to touch those dangerous things at all. The open source AI people get really mad when that's said, but it is inevitable.
Worse, the law does not apply to sovereign nations with nukes. They can and will make more and more advanced digital weapons until one causes some big ass problems.