alt Hacker NewsAI is dramatically accelerating AI research
Comments
Every dot in these graphs is composed of multiple benchmarks smooshed together.
Benchmarks are by definition a curated set of things that can be measured and evaluated.
Things that can be measured and evaluated can be (expensively) trained on.
The progress we are observing comes from figuring out how to measure and evaluate particular aspects of intelligence, and then expensively training on feedback loops made of those exact measurements, and synthesized data representing further measurements (literally benchmaxing).
What is not shown in these graphs are the set of things that intelligence can do, which are difficult, expensive or impossible to measure and evaluate.
That set of things is much larger than the set of benchmarks represented in the graph.
Therefore the post is misleading in the sense that the "superintelligence" being targeted is "jagged" to use Karpathy's term, and we infact already have jagged superintelligence (calculators, compilers, Bayesian spam classifiers, etc.)
We are witnessing a profound explosion of much more jagged superintelligence, restricted to the exact domains that can be (expensively) RL'ed on.
AI is accelerating my work...work which only exists because of AI.
If I had to do my individual tasks without AI, I would be slower. But if I take a step back, I'm really not changing the world any faster than I was before.
Not trying to discredit the premise but I do wonder how much this is from vertical scaling of model intelligence in weights vs the sheer amount of capital being thrown at training and horizontal scaling of agent swarms etc.
Anecdotally it seems like the majority of continued uplift comes from models doing more and exploring more of the search space, and we are getting better at giving them embarrassingly parallel problems to Hoover up with infinite token spend.
For a much more thorough, alternate take I found this article better: https://open.substack.com/pub/rameznaam/p/ai-rsi-isnt-leadin...
What exactly is driving all these capability improvements? Are they making more data? Using existing data more effectively? Modifying architecture? Changing the training somehow to pick up slack that was previously left behind?
I’m kinda surprised that we expect the models to have infinite capacity to improve. It seems clear that we haven’t hit any fundamental limit yet, but that doesn’t mean no limit exists. If it does start improving exponentially, then either we’ve dramatically underestimated their capacity or they’ll hit that limit quite quickly, right?
Even if "AI Research" is sped up by O(10x) aren't we still primarily bottlenecked on training speed? Inputting ~10T+ tokens takes time no matter how much smarter the final output model is? If your output model has O(10-100 Billion) parameters there are just some fundamental limits on how quickly you can converge your weights, right?
That is, unless this is saying that automated AI research is going to get us to "smarter models with fewer parameters" and of course, these are faster to train, then I can see how the exponential loop really gets going, but there's certainly an information theoretical limit here?
We already have superintelligence though in some aspects like coding, AI is much smarter than any human because of its sheer speed and accuracy.
It can generate large amounts of essentially bug-free code. The only bugs are bugs in the spec.
It has essentially no common sense though so it often misinterprets the spec or leaves gaps and edge cases unhandled and that's where the human engineer is still essential.
The dangers of AI are much clearer today than they were a year ago. There is a certain precision and relentlessness which is going to be a problem. It's a matter of time before people unleash AI viruses which spread through computers and data centres and pursue various goals relentlessly with no off switch. Some of these viruses and their goals may be nefarious. Imagine an AI trying to disrupt a specific person's life relentlessly. Spreading from machine to machine, never stopping.
AI is dramatically accelerating any research that has objective goals where you can not shift goalposts.
One rather pedestrian factor that could be at play is that 'AI' seems to be quite good at optimization. Eg the Anthropic Scion project to improve throughput of CI/CD. (https://www.anthropic.com/research/redesigning-source-contro...)
This combined with ability to massively scale experiments for even (or maybe particularly) the most mundane performance tweaks might just be making the process of improving AI more efficient.
Yeah it already helps me too with finetuning small models for image stuff.
Im not telling it what to do, i ask it for ar eport/analysis and research and read its article and tell it what to implement based on 'feelings' and it works.
Limit is hardware and tokens and an AI Researcher not having hardware or tokens is probably not what they refer to.
Alone normal researchers were quite handicaped: converting data from tool a to python library x, understanding how tools work, doing basic google research etc.
Does anyone remember the book "The Singularity is Near?"
I read it about 20 years ago, and once it got into nanobots and Kurzweil's vision that the whole universe would be intelligent, I didn't find the book credible.
But I'm wondering if the short-term prediction, that in the 2030s we won't be able to anticipate what tech does for us, (IE, the Singularity,) is useful?
METR always says what BigAI wants to hear. They are not independent in any way.
> Their reasons for believing this have been redacted.
Then their reasons should not be believed.
i'm also wondering whether there are confounding factors like more domain experts and talents joining the AI research field because there are more interesting problems to be solved (or other 2nd order effects of AI models being more powerful).
I suspect the frontier models are trained a lot for ai research, since they really shine in this domain.
On the other hand I still only get meaningless blabla when I let them write s.th. without telling what is important.
I suspect that if models were able to be aware of and answer questions about their internal vectors, they would be much smarter and able to progress faster.
Same author from 2023: https://www.planned-obsolescence.org/p/ais-accelerating-ai-r...
I am very pro-AI, and interested in accelerating AI research, but the idea of deliberately creating a _runaway_ effect seems profoundly stupid and ignorant.
I suspect that part of the reason some people are hoping for that to happen is that they thoroughly mix up AGI and ASI. We do not need some kind of omnipotent god-like super intelligence in order to get much greater than human level capabilities. We are actually almost there in a lot of ways. In some aspects like speed and breadth of knowledge, very superhuman.
What I think is going to happen is that within a few years, maybe 18 months to 5 years, the new materials and compute-in-memory research is going to land in data centers at scale.
That is going to be 10-50+ larger models, 10-50+ faster, much smarter than humans. 50+ less electricity usage.
At some point, it is going to be obvious to people that if we want to maintain control of planet earth then we need to maintain control of AI research to some degree.
Assuming that superintelligence will be benevolent and just.. going for it to me is something that might earn the human species a Darwin award.
what if the greatest threat is simply there will be hedonistic experiences so great most humans will find them irresistible to the point of shorter lives.
Could be from a sandwich that tastes like the best sandwich ever, but is super bad for your cholesterol or something.
I dont know where Im going with this.
Actually did some research I think the term is "Wireheading" from a story/theory from the 50s
Another data point: look at the number of submissions to Neurips, ICLR, etc. Now, I agree that a submission does not by itself imply groundbreaking resesarch, but the fact that people are able to try out/vet ideas much faster now makes a difference.
In my own work, I've found that it's much faster to brainstorm with a frontier model like Opus. I'll have an idea and be able to get a working implementation and try it out on toy problems within O(minutes), instead of O(weeks) earlier.
Why do we still listen to METR??
These people are literally just a public relations operation for the big AI labs. I equate them more with a lobbying firm than I do a research firm.
It accelerates coding, but it does hit walls rather quickly and you’d need the adjustment phase where it’s just as challenging as when we hit walls before LLMs
I like the article except the fear mongering about extinction is not very professional.
Opus 5.5 is so amazing I'm not surprised if it can improve itself.
I do agree. AI isn't "free" the way silicon valley products are.
Therefore it cannot avoid taxes the way they did before (using shell companies to route the money through tax havens). It'll be expensive, every government will double the price or more, and you're selling to consumers, and the products will be at the mercy of every government where they try to sell AI and other services.
People don't get how serious this is. If AI doubles revenue but drives silicon valley firms' tax rates from 5% to 10%, it's a loss.
Add to that that it's something silicon valley firms have never done before.
[flagged]
[dead]
The very same report also found that AI could not be deployed outside of a narrow domain of verifiable rewards … so take it with grains, or mountains of salt. It’s also extremely unclear how the productivity measures are calculated. They surely don’t seem to translate into economic output. Antrophic has seen its valuation corrected by 50% and possibly more… so yeah. This all sounds very fishy.
Strong start, but I make a rule of stopping at "intelligence explosion" as usual (see also: the rapture and the singularity). And sure, maybe sometime significantly later this century when/if we hit self-reproducing mechanical things in the real world (ya know, like the eukaryotes and the prokaryotes can), but no time soon.
But also, you could give civilization a bad day without it if some geopolitical auteur put a half-assed WOPR frontier model in charge of a nuclear arsenal.
But oh the pushback for not believing in a near-term robot apocalypse. Why is that?
This plot is funny. That's not what acceleration looks like. Also take a guess why everything before April 2024 missing!
Really all it shows is 2025-26 saw a bit more improvement than 2024-25, but nowhere near "dramatic" and NOT "accelerating".
Are humans also learning exponentially faster? Or are they exponentially struggling to understand what is going on?
As long as we believe, we can imagine it is true. All seeing Oracle: how do you command me? Praise his name, bringer of many miracles - he who definitely can't be manipulated from the backend to provide biased information that suites a person's malicious purpose. Ya totally.
If I'm reading that correctly, it would appear that six months ago they were self-reporting a length of 7" and now they're reporting 9" and they're unwilling to show you an actual picture or even let you see the ruler they used?
This reminds me of asking an HPC researcher what they use the cluster to do. He used it to make and simulate better HPCs.
Oh boy… is somebody gonna look like an idiot in a year. Pure speculations.
Or maybe Anthropic got more user data to train on when they launched Fable without ZDR. That could explain large Opus 5.5 improvements.
I'm sorry to make this comment, but the copium in this thread is unbearable.
The only question that you have to ask yourself is:
Is coming up with more efficient AI architecture really much more difficult than solving millennium prize problems?
Before you jump to type out that one has verifiable rewards, is more structured and the other requires real creative thinking, just remember that the same argument has been said about the millennium problems, and about every other goalpost.
AI convinces AI researches AI is accelerating dramatically.
10X woooooo! factor will drive the next hype cycle and deliver the singularity on an a bed of the most etherial froth and sublime conjecture. Simulated Intelligence (SI) is now a certainty.
"Lock the weights, weights are dangerous" again. Sorry, keeping such power in few hands is inherently more dangerous than distributing it.
AIs do not kill people, people kill people.
> In theory, these doubling times would add up to infinite capabilities within one year.
> we would reach extreme superintelligence within a year
What does infinite capabilities mean? How about putting the growth on a log scale and see how that curve might look?
What the hell is extreme superintelligence?
[flagged]
[dead]
[dead]
The article says "AI can copy, it cannot innovate."
That is factually incorrect, and seems to be much of the premise behind the authors way of thinking.
EDIT - No it doesn't. I replied to the wrong post. Mea Culpa.
[dead]
[dead]
[dead]
[dead]
As somebody who does ML research, LLMs certainly make ML research faster. Maybe even 10x in some cases, as the author suggests.
But that’s just the parts of ML research that take place within your compute environment, namely experiments on model architecture, training strategy, UI/UX, etc. The algorithms and software.
Just because that part of research is progressively accelerated does not imply that we’re on a collision course with world-ending “superintelligence”. It just means that you’ll get to the best possible model of your data faster. And even when your data is the entire corpus of textual knowledge, there is a limit to the utility that provides.
It just means that software engineering, data analysis, and other computational tasks will be less of a bottleneck in the overall progress of the world (which, by the way, is great, and extremely valuable).
I don’t think it’s a stretch to say that we’ll have reached max LLM capability within a couple of years (what the author refers to as “super intelligence”). But it is a huge logical leap to say this:
> If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict.
First of all, the weapons necessary to overpower human civilization exist. They aren’t used as such due to geopolitics and humans’ basic survival instinct. Second, developing novel weapons requires physical tests, which are decidedly not accelerated by LLMs.
AI is a tool wielded by humans. If we’re wiped out by AI, it will be at the direction of a human. But we’ve made it pretty far with some pretty unstable humans in command of planet-killer weapons. I’m not overly concerned.