alt Hacker NewsMath that humans don't understand but nonetheless allows AI systems to develop breakthroughs in various fields of science, technology, physics, engineering, medicine, etc., would have great value to humanity even if it doesn't help humans understand abstract truth at all.
Imagine if humans couldn't understand multivariable calculus, but we had access to an AI system that developed it, it initially seemed useless, then another AI system found a predictive model of electromagnetism using it.
Replies
I think you can also think about this without going as far as AI systems going and developing things on their own. Centaur math, with a human still guiding, but not able to fully comprehend results.
Imagine if humans could almost understand multivariable calculus. If someone had an idea that it is possible, and prompted it into existence, but did not understand it.
Actually when I put it that way, it seems easy to imagine that a lot of engineering can happen without the engineers fully understanding why their tools work, just that they do. If a human gives you those tools or an AI or a centaur, does it matter?
I think it already works that way with math a lot of the time (and definitely works that way with everything else.)
But at that point you have full AGI and its not just today's models. Today's models still need humans to understand things since it builds upon human knowledge.
When you have full AGI of course you no longer need humans to understand math.
> Imagine if humans couldn't understand multivariable calculus, but we had access to an AI system that developed it
Developing multivariable calculus requires much more than just solving problems though, it requires defining an entirely new system and space. That is not the situation mathematicians face today, modern AI cannot do that.
When talking about mathematicians and AI don't use fictive examples, we can look at what AI can do today and extrapolate that they can do more of that tomorrow, that is what we have to work with.
In the case you posit where AGI exists there is no reason to even discuss what is left for humans to do, since AGI is defined as when humans are no longer needed for anything, the AGI can do every bit of thinking humans can.