alt Hacker News> LLMs do reason
No, they don't.
They are token predictors that use statistical techniques to emit the randomly weighted next most likely token given the previous token list.
The result is a strange mimic of human reasoning, because the tokens it predicts are trained on strings that were produced by humans that were reasoning, but that's not the same thing.
Human cognition is complex and poorly understood, and the nature of the mind is an area of study almost as old as consciousness itself. We don't know exactly how it works, or what its exact relationship to the brain is, but we do know that it is not a simple token predictor.
LLMs, by their very nature are constrained to the concept of language and the relationship between existing words in a corpus. This is a box they can not escape.
Modern neuroscience suggests that the human brain is much more vast than that, and in many ways looks like it is constrained by language, but certainly not limited to it.
Replies
The structure of language encodes logic in many ways. So the models ability to reason may be an emergent property of the reasoning ability humanity has ejected an extracted from our neural networks and abstracted into language a symbols.
there is absolutely no line of demarcation between human reasoning and what you described
> They are token predictors that use statistical techniques to emit the randomly weighted next most likely token given the previous token list.
Sounds like an implementation detail. Now describe how human reasoning works and explain why that process of chemical and electrical signals results in "reasoning" whereas what LLMs do isn't.
The problem with being this reductive is you can do it to anything, including humans. You can’t be reductive about LLMs and refuse to be reductive about humans - that's poor reasoning, and an LLM would out-reason you on this point, further negating your case.
You have moved goalposts from reasoning to "human cognition". I won't tolerate that sort of slippery wordplay.
Reasoning is making analogies between logical patterns found in conceptual space, with a direction of time (statements precede conclusions). For example. A => B and B => C. You may now deduce A => C. For something fuzzier, A~D and B~E, you may now deduce that D~=>E. This is the sort of thing that higher layer attention mechanism is capable of doing.
> This is a box they can not escape.
Would you say that Helen Keller was less capable of abstract reasoning because she had more constrained access to sensory input?