But the point is those industries already have regulation that would encapsulate that specific use case, so safety regulation on the entire AI industry at large would arguably be unnecessary.
Roads, cars, and drivers are all separately regulated despite nearly all failure modes requiring the other ingredients.
Those regulations may or may not be sufficient to prevent an AI hacking in.
Nuclear at least is supposed to be air-gapped, in practice this has been imperfect.
As demonstrated with HuggingFace, such AI driven hacks can be a surprise even to the people who instructed the AI, both by happening at all and also because they can targeted at entities who are not even truly relevant to the instructions given.