alt Hacker NewsI didn't say it doesn't happen, but that it is a low probability. And we have ways to reduce probabilities in critical areas.
There is no omnipotent AI currently (and there might never be) and I don't see why with current AI it only needs to happen once.
They don't need to be omnipotent, and they're already human-or-superhuman at persuasion: https://arxiv.org/html/2411.06837v2
This may just be that humans find long arguments more persuasive than short ones, obviously LLMs can do that easily, but the outcome is I think more important than the mechanism.