> The central argument is that AIs, living inside datacenters, won't be able to evaluate the "coherence" of software "beyond the human ceiling"

Actually, the article states that AI will easily be able to handle increases beyond the human ceiling as long as those increases follow a linear curve, not an exponential curve.

So your whole critique dissolves. Sorry!