alt Hacker NewsI see two assumptions in the core argument, both of which are open to challenge.
1. AI can do proofs, but deciding which problems to solve, which math is useful, is by humans.
2. The math that's picked needs to be understandable by humans.
For #1, AI may be able to play a significant, if not a takeover, role for even figuring out what math is useful.
For #2, understandability by humans may be good for now, but could also turn out to be a significant constraint. Correctness is a goal, trust is an important requirement, human understandability may be an intermediary for that, but not necessarily the end goal.
In other words, the article may stand the current state of the art, but may not stand merely a couple years down the road.
Replies
> AI can do proofs, but deciding which problems to solve, which math is useful, is by humans.
This is within a very narrow view before the emergence of always running “minds” within any given domain. The only reason they don’t exist now is because they’re expensive.
Pretty soon we’re going to have always running minds that are constantly thinking about every domain imaginable and coming up with their own proofs and improvements and everything else imaginable within those domains.
Not sure I get that. If AI is not working for humans' benefits (i.e. it is not working towards an optimization problem set-up by humans), whom is it supposed to benefit, then? (i.e. how is that better than an entropy-producing machine)?
Same for 2, there are so many infinite ways to boil the oceans, but so few oceans to boil to begin with. Better make sure that this insane energy (both in the physical, due to natural resources scarcity, as well as intellectual) is spent towards meaningful and useful ends. We can no longer be the judges of that if we can't comprehend what we got in return.