The article's perspective reminds me of math class when I was a kid. The teacher didn't want the right answer alone, she wanted to see how I got there. In an absolute sense, the right answer is the right answer with or without my understanding, but that's not how the class was graded. We're approaching a point where AI is to science as my old calculator is to elementary school math -- it's capable of leapfrogging to answers faster and more accurately than we can.
Thing is, though, verification is knowledge itself. And if we can get to a point where the important questions have answers, we can surely have the verifiers explain those to us in ways we can understand them.
Meaning requires integration. But I assume AI can make meaning of it, now or in the future. I assume the same with people. However I don’t see the point of knowledge without meaning.
I don't think we're there yet for science. Perhaps for math proofs, but that's a very different domain ultimately since we know the rules of the game in a way we don't with science.
> we can surely have the verifiers explain those
This is not obvious to me at all. LLMs are capable of amazing feats of reverse engineering but are terrible at explaining and abstracting things well.
I think this is an artefact of how we train them, and they could be trained to be better at this if we focused on it, but at the same time, it shows its possible for these things to be able to do things they cant explain to us (granted, they can't explain them well - you can bash your way to understanding with them anyway but they aren't good at it)