The SV (silicon valley) motto of fail fast and break things accelerated evolution of software design systems. Can you imagine what would happen if people were hesitant to use Git works because a merge could break something? Lol.

But this kind of precaution is quite necessary in the development of AI systems. Current regulations have turned a blind eye to accelerate software and AI progress. Now we have security bugs and misalignments, and we will have AI misalignments as well.

Knowing how computing landscape has changed in SV, I can't imagine modern day SV helping fix FAA systems. Alternatively we could go with multiple incarnations of the same system, being battle tested and proven with independent concurrences, alongside current FAA system. Yeah, it is a boatload of money, and probably wasting 90 percent of it in redundancy.