I agree but don't think that's the best example, although not directly alignment related LW loves that kind of ideation of fantastical sci-fi scenarios.

Better would be the very evident negatives of non-ASI AI the world is already experiencing: economic concentration, job displacement, negative feedback loops from syncophancy, loss of societal trust/education from widespread fakes, etc.

None of that needs a Terminator scenario and is way more likely to get worse and ruin the world compared to the scenario where AI "escapes the box", turns the earth into paperclips, then grey goo to build their spaceships to leave for a distant star for... unclear reasons.

One problem for LW is that strong AI did not emerge via the route Elizier was expecting (and tried but failed at creating himself) relying on symbolic logical reasoning and self-editing to rapidly evolve.

That assumption led to belief in the certainty of a "foom" scenario where your little mediocre AI turns on one day and then explodes into Mythos in 15 minutes and then tries to murder everyone.

They've tried to reinterpret the gospel as told by the sequences to fit the LLM world (such as in AI2027) but it's often a stretch that strains credulity now that we observe scaling requiring hundreds of billions of dollars and years of construction for each iteration. And the AI model itself is just a bag of weights frozen in time until burning a lot more money and natural gas to train more.