logoalt Hacker News

Jtsummers • yesterday at 5:35 PM • 3 replies • view on HN

> I don't care if ASI progress stalls after it's already killed all biological life

What's your basis for thinking ASI will kill all biological life, and how do you think it's going to happen?


Replies

mrob • yesterday at 6:05 PM

>What's your basis for thinking ASI will kill all biological life

I think it's likely to do that because any unbounded goal that doesn't explicitly protect biological life (and we have no idea how to actually define such a stipulation) is best solved by killing all biological life. This is an obvious consequence of unbounded goals consuming unbounded resources, conflicting with biological life needing resources to sustain itself.

>how do you think it's going to happen?

I can speculate (e.g. we're nowhere close to the maximum killing power of drones), but I don't know because I only have human intelligence. An ASI is by definition smarter than me and surely capable of coming up with better ideas. But I do know that it's not going to do anything that would make a good sci-fi plot, because those always give the humans a chance to win, which would be stupid. Everything will seem to be going great and then everybody suddenly and unexpectedly dies.

➕ show 2 replies
TeMPOraL • yesterday at 10:44 PM

In the eternal words of Eliezer, the AI neither hates you nor loves you, but you are made of atoms which it can use for something else.

root_axis • yesterday at 6:49 PM

Crazy that people imagine ASI as being capable of wiping out all life on the planet but simultaneously having zero understanding of human ethics, morals, or suffering. Very revealing definition of "super intelligence"

➕ show 3 replies