alt Hacker NewsAGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent.
You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do. The best LLM "memory" is a search engine and document summarizer stuffed into a context window (which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you). To learn it would need RL (which requires specific novel inputs) and retraining (so that it can retain and compute answers with the learned input). This would all take too much time and careful input/engineering along with novel techniques. So AGI is too expensive, time consuming, and difficult for us to achieve without radically different designs and a whole lot more effort.
Not only are LLMs not AGI, they're still not even that great at being LLMs. Sure, they can do a lot of cool things, like write working code and tests. But tell one "don't delete files in X/", and after a while, it will delete all the files in "X/", whereas a human would likely remember it's not supposed to delete some files, and go check first. It also does fun stuff like follow arbitrary instructions from an attacker found in random documents, which most humans also wouldn't do. If they had a real memory and RL in real-time, they wouldn't have these problems. But we're a long way away from that.
LLMs are fine. They aren't AGI.
Replies
> AGI isn't going to happen within the next 30 years so this is moot. The actual researchers have said so many times. It's only the business people and laypeople whooping about AGI always being imminent.
The statements of what "actual researchers" are you relying upon for your "next 30 years" estimate? How do you reconcile them with the sub-10- or even sub-5-years timelines of other AI researchers, like Daniel Kokotajlo[1] or Andrej Karpathy[2]? For that matter, what about polls of AI researchers, which usually obtain a median much shorter than 30 years [3]?
[1] https://x.com/DKokotajlo/status/1991564542103662729
[2] https://x.com/karpathy/status/1980669343479509025
[3] https://80000hours.org/2025/03/when-do-experts-expect-agi-to...
>You cannot get real, actual AGI (the same ability to perform tasks as a human) without a continuous cycle of learning and deep memory, which LLMs cannot do
I disagree that this prerequisite is more necessary than e.g. having legs to move over the ground. But besides that, current LLMs are literally a result of the continuous cycle of learning and deep memory. It's pretty crude compared to what evolution and human process had to do, but that's precisely how the iterative model development cycle with the hierarchical bootstrap looks like. It's not fully autonomous though (engineer-driven/humans in the loop). Moreover, the distillation process you describe is precisely what "learning" is.
Agree with your overall point. Curious what you’re basing the “not in the next 30 years” claim on, if you’d care to expand.
I think you're somehow right and wrong at the same.
All those "it's like ..." are faulty – "post-it notes" are not 3k pages of text that can be recalled instantly in one go, copied in fraction of a second to branch off, quickly rewritten, put into hierarchy describing virtually infinite amount of information (outside of 3k pages of text limit), generated on the fly in minutes on any topic pulling all information available from computer etc.
Poor man's RL on test time context (skills and friends) is something that shouldn't be discarded, we're at 1M tokens and growing and pogressive disclosure (without anything fancy, just bunch of markdowns in directories) means you can already stuff-in more information than human can remember during whole lifetime into always-on agents/swarms.
Currently latest models use more compute on RL than pre-training and this upward trend continues (from orders of magnitude smaller than pre-training to larger that pre-training). In that sense some form of continous RL is already happening, it's just quantified on new model releases, not realtime.
With LoRA and friends it's also already possible to do continuous training that directly affects weights, it's just that economy of it is not that great – you get much better value/cost ratio with above instead.
For some definitions of AGI it already happened ie. "someboy's computer use based work" even though "it can't actually flip burgers, can it?" is true, just not relevant.
ps. I should also mention that I don't believe in "programmers loosing jobs", on the contrary, we will have to ramp up on computational thinking large numbers of people and those who are already verse with it will keep reaping benefits – regardless if somebody agrees or not that AGI is already here, it arrives through computational doors speaking computational language first and imho this property will be here to stay as it's an expression of rationality etc
The post-it note analogy is good, but as a psychiatrist, I'd frame it differently: LLMs are essentially patients with anterograde amnesia.
They can reason brilliantly within a single conversation — just like an amnesic patient can hold an intelligent discussion — but the moment the session ends, everything is gone. No learning happened. No memory formed.
What's worse, even within a session, they degrade. Research shows that effective context utilization drops to <1% of the nominal window on some tasks (Paulsen 2025). Claude 3.5 Sonnet's 200K context has an effective window of ~4K on certain benchmarks. Du et al. (EMNLP 2025) found that context length alone causes 13-85% performance degradation — even when all irrelevant tokens are removed. Length itself is the poison.
This pattern is structurally identical to what I see in clinical practice every day. Anxiety fills working memory with background worry, hallucinations inject noise tokens, depressive rumination creates circular context that blocks updating. In every case, the treatment is the same: clear the context. Medication, sleep, or — for an LLM — a fresh session.
The industry keeps betting on bigger context windows, but that's expanding warehouse floor space while the desk stays the same size. The human brain solved this hundreds of millions of years ago: store everything in long-term memory, recall selectively when needed, consolidate during sleep, and actively forget what's no longer useful.
We can build the smartest single model in the world — the greatest genius humanity has ever seen — but a genius with no memory and no sleep is still just an amnesic savant. The ceiling isn't intelligence. It's architecture.
Given how many "fundamental" limitations of AI have been resolved within the past few years, I'm skeptical. Even if you're right, I am not sure that the limitations you identified matter all that much in practice. I think very few human engineers are working on problems which are so novel and unique that AIs cannot grasp them without additional reinforcement learning.
> it will delete all the files in "X/"
How many "I deleted the prod database" stories have you seen? Humans do this too.
> follow arbitrary instructions from an attacker found in random documents
This is just the AI equivalent of phishing - inability to distinguish authorized from unauthorized requests.
Whenever people start criticizing AI, they always seem to conveniently leave out all the stupid crap humans do and compare AI against an idealized human instead.
I don't agree. The recent emergent behavior displayed by LLMs and test-time scaling (10x YoY revenue for Anthropic) is worth some hype. Of course, you are correct that most people who rally behind AGI do not understand the fundamental limitations of next-token prediction.
I don't think humans learn any differently than post it notes TBH We call them text books though!
> tell one "don't delete files in X/", and after a while, it will delete all the files in "X/", whereas a human would likely remember it's not supposed to delete some files, and go check first.
Have you seriously never had someone to go do something you told them not to do?
> It also does fun stuff like follow arbitrary instructions from an attacker found in random documents, which most humans also wouldn't do.
I guess my coworker didn't actually fall for that "hey this is your CEO, please change my password" WhatsApp message then, phew.
I've seen people move the goalposts on what it means for AI to be intelligent, but this is the first time I've seen someone move the goalposts on what it means for humans to be intelligent.
I disagree. There is some argument to be had that they're already generally intelligent. They're already certainly better than me in basically anything I can ask them to do.
So that leads to the question of what qualifies as intelligent? And do we need sentience for intelligence? What about self-agency/-actuation? Is that needed for "generally intelligent"?
I don't know.
But I feel like we're not there yet, even for non-sentient intelligence. I personally think we need an "unlimited" context (as good as human memory context windows anyway, which some argue we've already surpassed) and genuine self-learning before we get close. I don't think we need it to be an infallible genius (i.e ASI) to qualify as generally intelligent ... or to put it another way "about as smart and reliable as the average human adult" which frankly is quite a low bar!
One thing for sure though, I think this will creep up on us and one day it will suddenly become apparent that it's already there and we just didn't appreciate/notice/comprehend. There won't be a big fireworks display the moment it happens, more of a creeping realisation I think.
I give it 5 years +/-2.
LLMs are AGI because they offer intelligence on any subject.
>the same ability to perform tasks as a human
The first chess AIs lost to chess grandmasters. AI does not need to be better than humans to be considered AI.
>without a continuous cycle of learning and deep memory, which LLMs cannot do.
But harnesses like Claude Code can with how they can store and read files along with building tools to work with them.
>which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you
This don't matter. You could say a chess AI is a bunch of different people who work together to explore distant paths of the search space. The idea you can split things into steps does not disqualify it from being AI.
>But tell one "don't delete files in X/", and after a while, it will delete all the files in "X/"
Humans make mistakes and mess up things too. LLMs are better at needle in a haystack tests than humans.
>It also does fun stuff like follow arbitrary instructions from an attacker
A ton of people get phished or social engineered by attackers. This is the number 1 way people get hacked. Do not underestimate people's willingness to follow instructions from strangers.
> which is like having someone take an entire physics course, writing down everything they learn on post-it notes, then you ask a different person a physics question, and that different person has to skim all the post-it notes, and then write a new post-it note to answer you
This is the best summary of an LLM that I’ve ever seen (for laypeople to “get it”) and is the first that accurately describes my experience. I will say, usually the notes passed to the second person are very impressive quality for the topic. But the “2nd person” still rarely has a deep understanding of it.