You can read and edit binary code just fine, it's much easier with AI too

Indeed.

You can hardly re-create the binary from a suitable source, just like the weights just fell from the sky. But you can change the binary (patch the machine code) just like you can adjust the weights. The comparison is quite apt.

The whole reverse engineering and cracking scene is built upon this premise, you know.

Yes?