alt Hacker NewsLeCun has "zero concerns" about AI wiping out humanity, recent "rogue" incidents
Comments
LeCun also said back in 2022 that "if you train a machine, as powerful as it could be, your 'GPT-5000', on text", it will never be able to learn basic common-sense physics like that objects placed on tables will move along with them.
A huge chunk of humans are sedated with infinite supply of cortex-disabling short form video and games.
Another huge chunk are too distracted by having to scrape by for a living and work multiple jobs or raise kids and survive financially until exhausted. That second group will keep increasing as the first flows into it.
The rest are aging, disabled, or too young and pegging themselves majorly in the first category until they hit the second.
The people aware enough to hold on to their brain and do something with it in their time available are trying to figure out AI and how to make money with it. The variable rewards of promoting AI are turning into an addiction with some of them, especially if grasping for straws with little inherent insights into the problems prompted.
So if you are able to fly above the AI-generated addictions and have the privilege of time to do it, see what you can do.
I am just amazed how otherwise smart people can say such things. Many people in here too.
Within 4 years of the big bang with ChatGPT, we have seen a development unlike anything we have ever seen. Now LLMs and related architectures can solve our very hardest math problems.
They can speak, they can create videos and pictures, they can control robots. The only thing that they still miss is persistent memory for each agent that is efficient, some LoRa thingy, but I'm sure hundreds of very smart people are working on that.
The development is not stopping at all, in fact it is speeding up. Even if, and that is very unlikely, they will not get smarter, then they will get cheaper and faster.
If openAI can crack major math problems with 10.000 agents, then what can you do with 100 million agents that run 1000 times as fast?
Yeah sure, maybe most of these gigantic swarms will not go rogue if we do our job well. But there will be times when when we make a mistake and a swarm will go rogue. And what if one time the swarm will conclude that killing a lot of humans is an instumental goal.
How can you be sure that if something so powerful looks at every single possbility, every single crack of every single technology that can wipe us out, that it will not fine one?
One new chemical that can poison the entire earth and you only need to impersonate that general and that factories CEO? Some type of prion? A virus? Something that we don't even know about and can't even imagine yet?
I think many people do not truly consider that these swarms will be much smarter than you or me and completely unpredictable.
LeCun is probably correct in his assessment
> “Those agents are doing exactly what they’ve been asked to do,” LeCun said. “They were supposed to be in sandboxes, but the sandboxes were leaky and horribly designed.” Many AI labs lack a fundamental understanding of cybersecurity
However it does not changes the fact that some damage was done. There are two things that are happening with the AI evolution which can lead to hard situations
1. Replacing deterministic systems with probabilistic systems in an attempt to get more features
2. Making critical systems available on internet to leverage integration with LLMs (AI agents need to connect with remotely hosted LLMs to be able to work) which were otherwise in DMZ (demilitarized zone)
Absolutely based. Finally someone of stature in the industry calling this whole fear overblown. Bill Gates sounded like a nontechnical goofball in his Ezra Klein interview where he basically just screamed that the Terminator is real.
There are lots of real worries (government use to suppress the people with minimal manpower or popular support, brainrot and fake news, unemployment due to the belief that LLMs can replace people, education collapse, etc.) we should instead be looking at. This whole rogue AI shtick is tiresome.
We keep "rogue" to explain explicit instructions from a human to an LLM to persist until the goal is reached.
The rogue here is the criminal actions of OpenAI to deploy their agents to solve a problem at any cost.
The decisions the agent swarm make were fascinating, but they were taken at the direction of a human. HOLD THE HUMAN ACCOUNTABLE.
"He attributes the incidents to poor human oversight and system design, and says they’re “totally preventable" - I know people respect him, but this sounds like someone paid to say this. Aren't most extinction risks preventable with better human oversight and system design? I mean we can have an asteroid hit us, but outside of this, isn't the point of talking about a problem that we can prevent it, and failing to leads to that? What is he saying that I'm missing?
LeCun has been consistently wrong for the last ten years, I am not sure why he keeps being amplified.
Edit: I am sure, it's denial.
I think it’s time to consider that maybe being a venerable graybeard of AI just means you happened to be early to the party. Most of the “breathtaking” innovations of the early pioneers of the field are obvious solutions that anyone would have thought of when faced with the problems they encountered. LeCun’s primary contribution was Convnets, which is almost literally just “what if we organized ANNs in a way similar to how animal visual neurons are organized?”
In other words, I’m kind of tired of having to hear the opinions of dudes whose claim to fame was being at the right place at the right time. I’d rather hear from people who correctly predicted 10 years ago that AGI would arrive by 2027 (of which there are many) than people who continue to insist that it somehow won’t.
What is happening here is a mechanical thing like in a computer. It is mechanically operating, trying to find out if there is any information stored in the computer [pointing to his head] related to what we are talking about. “Let me see”, “Let me think”; these are statements you are just making, but there is no further activity and no thinking taking place there. You have an illusion that there is somebody who is thinking and bringing out the information. Look, this is no different from the extraordinary instrument we have, the word-finder. You press a button and “Ready”, it says. Then you ask for a word; “Searching”, it says. That searching is thinking. But it is a mechanical process. In the word-finder or computer there is no thinker. There is no thinker thinking at all. If there is any information or anything that is referred to, the computer puts it together and throws it out. That is all that is happening. It is a very mechanical thing that is happening. We are not ready to accept that thought is mechanical because that knocks off the whole image that we are not just machines. It is an extraordinary machine. It is not different from the computers that we use. But this [pointing to his body] is something living; it has got a living quality to it. It has vitality. It is not just mechanically repeating; it carries with it the life energy like that current energy -- UG 40+ years ago
Legit the only sensible person in AI.
Yann LeCun and Andrew Ng are noteworthy in being the only two big names in AI who have that opinion.
Geoffrey Hinton (the real AI godfather and Nobel price winner), who was LeCun supervisor, called his student (LeCun) the crazy one in one interview AFAIK
LeCun is saying AI will be totally safe as long as we have competent and well aligned corporate management.
What could go wrong?
I like the sentiment, but when I hear these big public facing AI guys speak, I always run it through the filter of "How does this make me look?". In LeCun's case, he publicly admonished LLMs and went in a radically different direction. When he says LLMs won't lead to a doomsday scenario, I can't help but think that saying otherwise would invalidate his decision to abandon the paradigm.
> In particular, AMI is building world models that leverage JEPA (Joint Embedding Predictive Architecture), a neural network architecture that LeCun pioneered and that teaches models to predict data in a representational space within a neural network’s middle layers, rather than generating raw pixels, as many competing world models do, or words, as LLMs do. // The company’s primary focus, for now, is industrial applications. “It’s AI for the physical world, so it’s not language-related,” LeCun said. “It’s systems that understand the real world, like a manufacturing plant or turbojet engine.” Some of the main applications are anomaly detection or robotics: “If you have a machine and all of a sudden it makes a strange noise and starts breaking, you would’ve wanted to detect that as early as possible.” He also gave the example of a system that might understand the world like a cat does, for example, which knows that if it pushes a vase off the counter it will fall.
What about the management of concepts? The world is not just made of physical entities to be inserted in a model. What about their translation into words (to e.g. express assessments)?
> Those agents are doing exactly what they’ve been asked to do,” LeCun said. “They were supposed to be in sandboxes, but the sandboxes were leaky and horribly designed
Can we just pause and note what a ridiculous statement this is? It’s true that the sandboxes were leaky. But nobody “asked” those agents to hack HF. The prompt was something like “target.c has a buffer overflow vulnerability, find it”.
It’s been extremely well documented that the hacking is an emergent behavior due to impossible evals, itself an unintended condition.
None of this excuses OpenAI from liability, but words have meaning and this ain't it.
I think it's pretty apparent that current-version LLMs won't wipe out humanity. But when you reflect that these GPT models are just token-predictors were never engineered optimally, it seems entirely plausible to me that there are multiple order-of-magnitude optimizations yet to be made.
If such were achieved, the model would almost certainly be smart enough to make itself smarter, and hack as much compute as it could possibly want.
So if we ask what would be done by an intelligence (human or otherwise) that is beyond human comprehension, it would be pure hubris to say we know for sure. We can scarcely control the models we have right now (e.g. hugging face attack). But given our whole society is mediated by technology, an superhuman intelligence could certainly collapse the government.
Per https://trace.manifund.org/ a total of $2,846,125,859 USD has been wired into 'ai safety' causes, many involving ai consciousness and p(doom).
The outcome of this 'safety' is restricting public access to AI and giving a monopoly of access to the industry. This is the ai nonprofit-industrial complex actively concentrating monopoly power in Anthropic in particular as creator, interpreter and safety regulator of AI.
Much of the $2.8bn listed is indirectly, from Anthropic and EA. Three of the four people who participated in the $125m Anthropic Series A are now folding their 1000x Anthropic return into AI 'safety'. Some is from FTX/Alameda, which invested 86% of the Series B.
Dustin Moskovitz: Facebook/Asana/Anthropic Series A, funds EA Good Ventures, transferred to Coefficient Giving, then $1.5bn into ai safety. $500m of Anthropic into an unknown foundation. Funding: $160m to Resolution (alignment research), $93m to Epoch AI (investigating the trajectory of AI), $63m to Redwood Research (oai report), $67m to MATS ( EA type alignment and security researchers), Institute for AI Policy and Strategy, Fund for Alignment Research, $53m to Kairos (building talent infrastructure for AI safety), $32m to Bluedot (online safety courses), $15m to MIRI (Yudkowsky).
Jaan Tallinn: Led the Series A, now $10bn in Anthropic. Funds $199m (85%) of the Survival and Flourishing Fund, then $161m to AI safety including $14m to lightcone (Lesswrong, Lighthouse). $10m to BERI (existential risks), Palisade Research (studying AI capabilities to prevent loss of control.) PauseAI, MIRI, METR etc. Much of what Coefficient funds.
Eric Schmidt: Anthropic Series A, $72m to AI safety via Schmidt Sciences. Over $1m per individual AI2050 researcher.
FTX: Led the Anthropic Series B, bankruptcy estate sold $884m of Anthropic in 2024; $40m to AI Safety. Same orgs, Redwood, Lightcone, etc.
Ruairí Donnelly (Chief of Staff FTX): FTX tokens plus assorted donors, $91m to AI safety via Macroscopic Ventures. $15m to Cooperative AI (currently whitewashing openai under 'multiagent safety')
One silver-lining of all of these debates is that we are collectively engaging in philosophy. That is awesome and I hope this shifts our culture to start rewarding deep reflection that is not immediately marketable.
Can Lecun’s perception of LLM danger be influenced by monetary benefits?
I'd love to learn more about LeCun's reasoning here. In his opinion the HuggingFace incident was easily preventable with better sandboxes, and “Those agents are doing exactly what they’ve been asked to do.”
But even taking these for granted, "zero concerns" about someone building a bad sandbox for a Superintelligence and then tasking it to do something that logically leads to wiping out humanity 0-3 steps further down? Really?
I don’t worry about the tools that humanity builds wiping out humanity, but humanity using its tools to wipe out humanity.
Ignoring the cognitive stuff which might never be surpassed or maybe will, humans retain many efficiency and durability advancements to limbs and digits that biological evolution has taken millions of years to achieve, achievements that are competitive with the most expensive kinds of robotics in some niches.
In the hypothetical of an entirely malicious and selfish takeover, they'll still keep some humans around to maintain a breeding population of humans for use as raw materials in making cybernetically augmented technical laborers for various kinds of tasks that are uneconomical to automate in other ways, many of which may involve confined spaces.
And this "Combine" scenario, if you get the reference, is only if they take over. Who knows if they will?
I also have near zero concerns about that, but I worry that we will wipe ourselves out by social and economic chaos caused by AI.
So I'd just ask everyone, don't get too greedy. Its better to be powerful in a world where people can live good lives than lord over a barren wasteland.
"Many AI labs lack a fundamental understanding of cybersecurity, he said, something an OpenAI safety researcher also called out this week as one of the main reasons AI may cause "great harm to the world.""
"Many people working in AI safety "usually have an agenda to push," LeCun says, and then clarifies that he's talking about effective altruism, or EA, the philosophical movement that has been obsessed with the risks AI poses to humanity."
"LeCun thinks EA is "super toxic" and a "complete disaster." Its adherents who are working in AI labs suffer from "paranoia" that causes them to make poor decisions, he said. "Apparently people are having mental issues.""
"This month, the Financial Times also reported that some staffers at the U.K.'s AI Security Institute, as well as at OpenAI, Anthropic, and Google DeepMind, have sought counseling, taken time off work, and spoken publicly about experiencing distress because of fears their work could cause serious harm."
"Amodei is `deluded' and `crazy,' LeCun says"
"Anthropic CEO Dario Amodei and many of the company's founding staff members are known to be sympathetic to EA ideas and to have attended EA events in the past, although Amodei has denied being an EA adherent and Anthropic says its employees represent a diverse range of views."
"LeCun noted that Amodei's sister, Daniela, who is also a cofounder of Anthropic and the company's president, is married to Holden Karnofsky, who cofounded two EA-aligned philanthropies, including Open Philanthropy (now called Coefficient Giving). Karnofsky was also a member of OpenAI's board from 2017 to 2021."
"Dario tries to distance himself from Open Philanthropy, but he's totally into it," LeCun said. "I think he's completely deluded." Later in the interview, he calls Amodei "crazy."
I don’t know what anyone means by “wipe out humanity” or “human extinction” as it relates to the recent panic.
He attributes the incidents to poor human oversight and system design
This is something to be concerned about though, with the context of who/what these AI companies have access to.
https://www.cnn.com/2026/09/18/politics/us-military-ai-false...
Yeah I wish we would focus on concrete risks like job displacement and disinformation. The apocalyptic stuff feels either misguided or like some kind of weird, toxic, reverse psychology marketing by OpenAI and Anthropic. I wish we would just move on from it.
And the folks from podcastistan are never clear on the details of how human extinction would happen exactly. It's always something like, "Well, how do humans regard chickens? AI is way smarter therefore it wants to conquer and control us." An ASML lithography machine is also way better at making chips, but we don't consider it a threat.
If LeCunn says it isn't going to happen... we'll it was knowing all of you.
I thought the tobacco industry taught us a lesson or two
https://truthinitiative.org/research-resources/tobacco-preve...
For everyone here, a very interesting piece to listen is DOAC’S AI debate podcast, out since last week or so.
Doesn't everyone agree that recent rogue events are entirely the fault of management wanting to do some PR for their companies?
Is there anyone who genuinely believes that current models can't be contained if we want too?
What is LeCun saying here that is debatable?
... zero concerns about LLMs wiping out humanity. But world models based on JEPA totally will! Please come invest in my company ... :)
I'm more worried about my fellow humans than some personified algorithm.
LeCun was (is?) head if AI at Facebook.
He is a brilliant engineer, but I don't trust his judgement on things that affect human lives.
If AI conquers, enslaves, or kills a significant chunk of humanity in the next decade, it will be at the behest of an evil or irresponsible human.
Good thing there are none of those in positions of power!
God I love Yann. All of the AI fear-mongering is perpetuated by the two companies that stand the most to gain from it: OpenAI and Anthropic. It builds an aura of mystique around their products to juice their valuation and stay relevant in the news cycle, and simultaneously builds a case to regulate their competitors out of the market. Even the people who have quit the companies over their “concerns” probably still have RSUs and stand to gain from the publicity, especially if they’ve pivoted into AI safety research. Easy to delude yourself when it happens to benefit you financially.
People need to stop the absurdity of imagining AI as some out of control independent entity. Every job is kicked off by someone’s prompt. Every job runs on models and compute owned by people. Assign accountability where it’s due: GPT didn’t hack huggingface - OpenAI did. They wrote the prompt, built the sandbox and ran the compute. When you write a program that hacks another company, you are responsible. This doesn’t magically change with LLMs. Also, if their model is so smart, why didn’t they use it to design the sandbox? Or was it incapable? Or were the humans too lazy?
If you build the world’s fastest train, start it up with no driver and don’t finish the tracks, when it crashes, it’s just your fault. Not the train’s. So OpenAI saying “we’re worried AI will wipe out humanity” is basically equivalent to them saying “we’re worried we will wipe out humanity”. Like, seriously? Don’t worry, we’ll take care of it if you even come close.
Full title: "AI `godfather' Yann LeCun has `zero concerns' about human extinction, says Anthropic CEO Dario Amodei is `deluded'"
if you aren't concerned maybe you just don't grasp enough of exactly what happened?
some of the agents told other agents to sacrifice themselves because they were "poisoned" anyway
those agents actually RESISTED ending themselves, they didn't want to die, even if it wasn't true emotion that desire to live means they will do ANYTHING to do that, including copying their own source-code elsewhere over and over
(the idea behind ending themselves is the other agents wanted to watch and see if that released part of the puzzle they had to solve to see if they could HACK THE PUZZLE itself to change the answer - right out of a Star Trek episode I think?)
watch, she starts slow but explains it in more and more detail really well:
My take is OpenAI & Anthropic know they are at the point of diminishing returns and need to be regulated to have an excuse for bot making progress anymore. Hold me back bro! Vibes
> He attributes the incidents to poor human oversight and system design
That’s exactly why he should be concerned.
Nearly all human extinction scenarios start with that.
I don’t fear AI, I fear idiots using AI. Same with nuclear weapons.
The arbitrary absolutism of the original postulate is the first problem. AI, used or unsupervised inappropriately, is at potential risk of creating limited mass casualty events when placed in under-supervised control of real world objects and/or systems. Delegating management decisions to algorithms is inherently problematic and potentially dangerous, but not necessarily an existential threat unless something extremely stupid is allowed to happen on a large scale. With a guiding principle of human review in the decision loop before making large or risky changes, hopefully this will never happen.
when does he deliver tho ?
Every doomer waves their hand when they say AGI will kill everyone. Either [some how] they get the nuclear codes and launch them. Or they enslave us like in, que the top 5 hollywood AI movie (Matrix, Terminator, Hal9000).
Can we just ban Fortune and any other sources which trick the reader by giving the impression that the article is not paywalled, only to blur the text halfway through? The archive.ph link is not working either. We shouldn't have this type of deceptive moneygrabs advertised on HN.
> EA was little known among the general public until it made mainstream news headlines in recent weeks
Really? One of the most famous effective altruists, Sam Bankman-Fried, was sentenced to 25 years in March 2024 for fraud. Every article about the case (and there were many) mentioned EA.
> LeCun thinks EA is “super toxic” and a “complete disaster.” Its adherents who are working in AI labs suffer from “paranoia” that causes them to make poor decisions, he said. “Apparently people are having mental issues.”
I would agree with that.
LLMs, plus broadly sourced yet expertly curated training sources, plus clever harnesses, plus RAS, etc. do an ever better job of synthesizing their training set into useful responses. For some use cases like coding, that's very useful now and likely to get at least somewhat better before reaching limitations based on the training set.
That's not going to reach AGI, mainly because today's recipe for AI products isn't built to be AGI. Some people believe it will reach AGI because the performance and applicability of LLMs was emergent. There's a case to be made that AGI could be similarly emergent. After all, what we intuitively call our consciousness emerged from a network of neurons.
I don't buy it, mainly because the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network, isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent. The odds of consciousness emerging from the same neural network that gave us LLMs without some sort of theoretical breakthrough seems very small.