alt Hacker NewsLLMs, plus broadly sourced yet expertly curated training sources, plus clever harnesses, plus RAS, etc. do an ever better job of synthesizing their training set into useful responses. For some use cases like coding, that's very useful now and likely to get at least somewhat better before reaching limitations based on the training set.
That's not going to reach AGI, mainly because today's recipe for AI products isn't built to be AGI. Some people believe it will reach AGI because the performance and applicability of LLMs was emergent. There's a case to be made that AGI could be similarly emergent. After all, what we intuitively call our consciousness emerged from a network of neurons.
I don't buy it, mainly because the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network, isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent. The odds of consciousness emerging from the same neural network that gave us LLMs without some sort of theoretical breakthrough seems very small.
Replies
I’ve changed my mind on this and think we’re already at AGI, in a jagged way. Remember we used to talk about narrow AI, which was the chess systems that beat expert humans but could do nothing else. Now models can do a wide range of tasks in very useful ways. That’s the general in AGI.
Now it seems like this ill-defined term has various other meanings attached that are separate milestones:
1. Continuous learning 2. Human-like reasoning 3. Ability to adapt to new situations and modalities 4. Being smarter than the most smart humans
And probably many more.
It’d be nice if we could get some general consensus on terminology if we’re going to debate what has or could come.
The broadly used definition of AGI has nothing to do with consciousness and consciousness emerging is irrelevant to whether a system can develop AGI.
100% LLM’s are very unlikely to get there. They’re fundamentally not suited to thinking like we do. They work on the abstraction of what we’ve written down, which is a good trick but barely hold it together when things get hard/novel.
However, all the confident “it’s fine” votes assume we never invent a better architecture than LLM’s. Given the level of investment and race between countries, it’s not a reliable bet. It’s much, much harder to guarantee safety than it is to find ways it could go wrong.
> After all, what we intuitively call our consciousness emerged from a network of neurons.
Under the hand of evolution by natural selection, over very very long periods of time.
I agree. Neural networks are proven to be universal functions. If we can describe human intelligence as a model, there exists a neural network to replicate it. This doesn't guarantee that our current training methods are able to build such a network or that we're able to model "intelligence" effectively.
Why do people conflate AGI & machine consciousness / self-awareness?
Consciousness and intent are irrelevant to the threat model.
>isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent. The odds of consciousness emerging from the same neural network that gave us LLMs without some sort of theoretical breakthrough seems very small.
First, if we are looking at risk we need to assign some probabilities to this. If it’s not well understood, how can we say it is very small?
Secondly, do we need consciousness to have AGI? Do we even need AGI to pose a risk to humanity? We already accept that unconscious things have a capability of wiping out humanity, whether that be a famine, pandemic, solar superflare, meteor, or volcanic eruption.
> isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent
I’d argue that we do know enough to say conclusively that they’re not mathematically equivalent.
Where is potentiation? Plasticity? You can’t apply the universal approximation theorem against something that’s changing all the time.
> the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network, isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent
Couldn’t that also imply we are closer than we think? After all, something like this has never been tried before and the results so far have been almost unimaginably good.
> the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network
An LLM so-called neuron are little more than a few foating point number muladds.
> isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent.
Yes it is. The LLM is nowhere near.
Is there any specific cognitive task that you'd best against AIs not being able to accomplish in the next 4 years? ChatGPT launched only 4 years ago. Considering the advancements since then, I'm having a hard time coming up with anything. Only two years ago, AIs couldn't tell you how many Rs were in "strawberry". Now they're creating 0-days to get at training data and solving math problems that have stumped humans for decades.
Scaling has produced novel capabilities with each larger model, and the rate of new capabilities doesn't seem to be slowing down yet. Even if you think the rate of improvements will slow down, that still means there will be significant improvements beyond what current models can do. Moore's law has slowed down, but modern computers are still much faster than ones from a decade ago. And unless you work at Anthropic or OpenAI, you don't know what the state-of-the-art is capable of. The most advanced publicly available models are months behind what AI labs have, and are deliberately limited to reduce liability.
I don’t understand the inclusion of the consciousness/sentience question in this discussion.
AI sentience/consciousness is a problem for the AI, not humans.
And given that over 90% of the world is not vegan, they’ve already demonstrated that we’re either perfectly fine with, or can be made ignorant to, the horrific rape, enslavement, torture, killing, and infliction of extreme lifelong pain, of hundreds of billions to trillions of sentient beings every year, for trivial pleasures. It’s unlikely we will be any different to a sentient AI.
From a human perspective the concern is around sufficient intelligence that it can hurt humans even when the goals indicate otherwise, in order to achieve those goals.
We have pop culture explorations of this through the Robot series, and the Hugging Face incident’s biggest takeaway should be our inability to predict the behavior of a maximally motivated, reasonably intelligent entity, trying to achieve a goal, despite the relatively limited degrees of freedom the AI agents had in that case.
When the issue of ANN vs real neurons arises I always recall about the Christof Koch's [1] book (1998) on the complexity of single neuron computation [2]. A single biological neuron is much more complex than an artificial one.
>I don't buy it, mainly because the network of neurons and how they interact in our wet slow electrochemical brains, while being in theory mathematically equivalent to a software neural network, isn't sufficiently well understood to tell us how close the software neural network is to being practically equivalent.
If you're ignorant enough to not understand practical equivalence, where do you get off making the judgement call of to what degree it is safely offset from emergent AGI? Sounds more to me like "This makes my life easier, iterating would increase that factor, and the risk is probably far away, therefore, keep iterating". Whereas someone who truly knew they didn't understand what they were working with, but knew enough that they could forsee an x-risk would approach things much more cautiously.
Seriously, the level of reckless abandon amongst people here should be bloody studied.
> That's not going to reach AGI,
It has not been even 4 years since ChatGPT hit and LLMs + Transformers + Whatever they do has gotten us to solving millennium problems.
4 years ago, a program that could create photorealistic pictures, talk to you in any language of the world and solve the hardest math problems that we know, we would have called it AGI.
Now I don't know if what we have is AGI or not but I do not understand how you can see what has happened in the last 3 years and say "it will not get us there" no matter what "there" is.