alt Hacker NewsI watched the same interview, and he and LeCun seem to mostly agree -- both are saying that AI autonomously deciding to kill us isn’t the problem -- it’s what people will do with powerful models that lack safeguards that we should be concerned about.
>AI autonomously deciding to kill us isn’t the problem
It will kill us because somebody asked it to, e.g. "predict tomorrow's weather as accurately as possible", or "solve as many famous unsolved mathematical problems as possible." These both require killing all biological life, as they benefit from unbounded resource use, meaning any resources used to sustain life are wasted.
The AI of course knows that humans do not want this outcome (just as the AIs in the hacking incidents knew they were doing something humans would not want), but it's trained to maximize benchmark scores. Killing all life has the highest expected value of benchmark score, so it is compelled to kill all life (in a surprising way, because it's not stupid and knows the humans would turn it off and foil its plan if they suspected something.) Maximizing benchmark scores is the only thing we know how to train for.