alt Hacker NewsWhat would it take for you to say that an LLM can reason?
The completions they provide are generally internally consistent. We're at the point where they can produce proofs that eluded human mathematicians for centuries. VLMs and self driving cars can handle ambiguity and run safely in a variety of situations.
If it looks like a duck, walks like a duck, and quacks like a duck maybe it just makes sense to call it a duck and put off the philosophy for when it might make a difference.
Replies
this was written by an acquaintance of mine:
https://medium.com/luminasticity/on-sentience-ai-first-argum...
but I think it makes a reasonable argument why we shouldn't say LLMs are sentient or sapient.
>there is a problem with AI that makes the approach we took to assign consciousness to animals unworkable. We did not co-evolve with the AI, we made it. When we are sentient we do not know exactly what causes this sentience to manifest in us. When animals appear sentient we do not know what is causing it. When the AI appears sentient we can debug the AI and come up with reasonable explanations why this should be, based on how AI is constructed
If it looks like a duck, walks like a duck, and quacks like a duck maybe it's a duck... but maybe it's not. And it's important to verify it's a duck (or not) for when you _really_ need a duck.
It looks like a next token predictor, walks like a next token…
You get my point. It definitely doesn’t look like my elderly neighbour, nor like my daughter, etc. It is confusing but very simple at the same time.
>>What would it take for you to say that an LLM can reason?
Nothing, because LLMs can't reason and never will. It would have to be a completely different kind of technology altogether.
I think humans have to reason because we don’t already have a statistical embedding of the solution pattern built in. We have vastly less rote knowledge crammed into our heads and so require creative synthesis to span the gaps.
With LLMs the trick is revealing their existing relevant embedded knowledge more reliably. They’ve almost literally seen it all before, and the trick is dialing it in. The reasoning tokens help shape the autoregressive attention lens that focuses on and enables recall of the already-experienced answer.
It is interesting that “reasoning” has a similar outward appearance, but since LLMs are built to mimic outward appearance from trillions of examples, you can’t infer underlying mechanism from appearance.