It's not the ~50 billionares I'm worried about. They are human. You can appeal to Bill Gate's desire to leave a legacy he's proud of. You can appeal to Elon Musk's desire for technological progress.
What worries me are the god knows how many "billionaire equivelent" entities and institutions that swing their dicks around in the same manner but do so with the sociopathic ruthlessness of corporations and governments.
Peter Thiel may be evil but he'll almost certainly never knowingly instigate genocidal war in a foreign land for marginal benefit to his interests whereas that's exactly the kind of thing that nonhuman entities do because no person is singularly responsible.