The problem is that, in this arms race, unaligned AI has a competitive advantage over aligned AI.

The second problem -- already seen in Ukraine v. Russia -- is that in a high-stakes situation, humans will take every safeguard off.