alt Hacker News>It's all so obscene. Instead, why don't we try to protect human dignity and move towards a more humane future?
I think the Darwinian logic of reality might make this hard. If society A and society B are both developing AI and one of them stops in order to protect humans, society B may continue to develop AI and then it might either outcompete society A economically to the point of reducing it to poverty (it is theoretically possible to take most of another society's market share in something by only slightly outcompeting it in price or quality), or it might even outright conquer society A.
A solution to the problem needs to address this issue somehow.
Replies
Yeah, it’s the same problem as nuclear proliferation.
We see what happened with Ukraine when they gave up their nukes in exchange for a promise of protection.
The same could be true of countries who forgo AI development directly in exchange for promises.
this is Amodei's position in a nutshell for AI development. We have to go as fast as possible because China. It's not the only frame though. If AI models and warfare (cyber, biological) becomes easily accessible and dangerous enough, there is a strong incentive for the world's leaders to cooperate towards something akin to nuclear non-proliferation.
In fact, there's strong incentives now to slow down AI progress for multiple reasons: de-escalate tension over Taiwan and lessen China's desire to build their own advanced fabs, protect peoples livelihoods by smoothing the AI transition. Except the incentives to bring AI companies public (and maintain some twisted shred of American Hegemony) are greater.
Unless you change the balance of selection pressures…
"Darwinian logic of reality"
What does outcompete economically mean and why would it matter? Or do you mean society A dominates in some form society B? This has already happened in history and is the essence of capitalism. If you want to overcome this situation you need to replace capitalism globally.
Maybe a stupid question but aren't the Amish essentially this thought process put into practice?
They aren't able to outcompete their neighbouring societies economically yet are in no way impoverished, and in many cases actually come out on top by many QOL/health metrics.