logoalt Hacker News

pjc50today at 10:02 AM4 repliesview on HN

How exactly does that work? The nuclear system of MAD relies on physical threat, lab A achieving ASI (artificial scary intelligence) does not prevent lab B achieving it.

I would like everyone involved to be a lot clearer about their threat models, with plausible series of clearly linked steps, rather than just sounding like a Vernor Vinge novel.


Replies

ralfdtoday at 10:49 AM

The idea in AI Risk circles is that the first true Scary Intelligence wins, as it can recursively advance itself and exponentially outgrows/hack any following (weaker) B or C intelligence.

mycocolatoday at 10:59 AM

Lab A reaches ASI, and is prompted the following: "Permanently nullify all other AI labs".If it's ASI it would succeed.

show 2 replies
568236attoday at 10:20 AM

[dead]