> LLMs are already more persuasive than most human writers.

And writers are well known to be the true holders of power in the world... In any event, you can see that even among humans, the most persuasive aren't the most intelligent.

> the algorithmic feeds which people now find addictive in the sense that they struggle to stop themselves from wasting time on them, got this way in part due to AI constantly optimising for just that.

There are many addictive things, most of which have little to do with intelligence.

> because we can just build for hurricanes as if they're Poisson distributed

Hurricanes are not particularly important components of power in society. The economy on the other hand is very relevant. Even to the extent it can be controlled, which is a fairly large extent, there's unlikely to be a consensus on the desired outcome.

Anyway, people will have to choose to listen to a machine that claims it's very intelligent even though it will make plenty of mistakes (as intelligence often does). Is that how you know people to behave?

> Humanity? All of it? Name one.

Some viruses. Probably some bacteria. Possibly some insects. I don't necessarily mean one single species, as they're all less intelligent than people.