>Can you explain how exactly an AI would develop a novel planet-killing weapon without anyone noticing and pulling the plug? Like I seriously don’t understand this argument.

That's the danger with creating a super intelligence, that you won't understand it. My dog is pretty smart, has good instincts and I'm quite confident he can survive in nature all by himself. But if a human decided to harm him, he doesn't stand a chance.

Seems like bioweapons (assuming "planet killing" means killing all the people, not literally blowing up Earth) are one way. Convincing the creator it would achieve their aim and could be contained when in fact it could not.

AI could invent something like social media. Obviously not social media because that's already been invented, but the next thing that slowly destroys society while seeming to be good.

> probably destroy almost anyone who survived the initial blast and fallout.

The difference between killing almost everyone, and killing everyone, is similar to the difference between almost getting pregnant and getting pregnant. A hundred thousand years later, a world where almost everyone had been killed would be teeming with humans.

> Can you explain how exactly an AI would develop a novel planet-killing weapon

I already answered that question in my previous comment, and the answer is that I am not going to explain how to develop weapons, but the larger point here is that we already can't explain how exactly AIs do innumerable things we see them doing every day, which is why we're having this conversation in the first place.

> I seriously don’t understand this argument

That is a point that I think everyone can agree on: you do not.

I think they mean we don’t have a deathstar. We can make the atoms here shitty. We can’t annihilate them or scatter them out of their own gravity well.

Life in theory could restart with our weapons, but what’s the point with the other scenario.