Nuclear disarmament is governed by treaty, which is largely adhered to.

An AI treaty could quite easily be offered by the US because it has multiple frontier labs. Most countries would quite happily limit the activity of their larger companies in exchange for their companies not being eaten alive by AI generally. It is pretty great fodder for international cooperation, and governments can move fast when it is timely to do so.