> The vast majority in models up to trillions of parameters are legible, and their circuits beautiful.

That's very encouraging. For quite a while, neural nets were considered an incomprehensible black box. That paper shows they're not. Real progress.