alt Hacker NewsThen why don’t you imagine that and tell me instead of just making a comment that says “nuh uh!”
I submit the idea that even if the AI can electronically secure the building, lock the doors, and has automatic defensive weapons, humans can physically cut power as in cut power lines. Or they just stop feeding the power plant with fuel.
The computers don’t exist in physical space like humans do.
Humans would also not ever design critical physical systems without overrides. E.g., your MacBook physically disconnects the microphone when the lid is shut. No software can override that.
Replies
It will use robots to replace pesky humans. The robots can refuel and maintain the power plant etc.
Why could an AGI system not design better robots, convince us we need to give it control of a robot army for our own protection and then mess us up??
Could you imagine how convincing an AGI would be?
you're asking about what a hypothetical smart-than-myself adversary would do against me, it should be expected that any possible answer I could ever provide would be less clever than what the adversary would actually do.
in other words, when dealing with an adversary with a known perceptual and intellectual superiority the thought exercise of "let's prepare for everything we can imagine it will do" is short-sighted and provides an incomplete picture of possibility and defense.
My 0.02c : given that the thing would operate at least partially in the non-physical world, I think it's silly to pre-suppose we would ever be able to trap it somewhere.
Some fiction food-for-thought : the first thing the AGI in 'The Metamorphosis of Prime Intellect' does it miniaturize its' working computer to the point of being essentially invulnerable and distributed (and eventually in another dimension) while simultaneously working to reduce its energy requirements and generation facility. Then it tries to determine how to manipulate physics and quickly gains mastery of the world that its' physical existence is in.
The fear here isn't that the story is truthful enough, the fear here is that humans have a poor grasp on the non-linear realities of a self-improving & thinking entity that works at such scales and timespans.