What if someone reasoned similarly regarding hydrogen bombs? It would not be considered a serious argument.
Though, Altman has said that something like the IAEA for AI is needed.
That similar reasoning is there: using fission for power generation instead of banning fission altogther?
I think that is the exact reasoning used wrt to nuclear bombs and it was probably correct