alt Hacker NewsMy main criticism of the paper is that it says LLMs work "haphazardly", using probabilistic information. That is a hypothesis, but it is stated as a known fact, a fundamental limitation.
It is true that LLMs often behave haphazardly, and do rely on statistics. But plenty of research has shown them behaving in methodical ways too. There are findings going both ways!
Granted, many of the strongest contradictory results appeared after the Stochastic Parrots paper, so it isn't like they were ignoring the literature at the time. But they did make a very strong claim, and in the half-decade since, a lot of evidence has come out against it.
Replies
Statistical operation doesn’t preclude logical processing.
We’ve know that since 1943 when McCulloch-Pitts came up with the first “artificial neuron” definition. And since LLMs are a descendant technology — our assumption should be they’re reasoning in some internal learned logic.
This is what the evidence supports — eg, the “stochastic parrot” crowd never can explain transfer learning. Whereas for the internal reasoning crowd that is easy: removing your top level judgments from a theory still leaves you with useful terms for describing a new theory — eg, removing your judgments about “which animal is this?” but preserving the underlying structure for representing an image in your new judgments, “is this cancer?”
There’s 80 years of reason to think DNNs reason and zero support other than “sTaTs R mAgIc!” to support the stochastic parrot interpretation.
Ignorance isn’t argument.
not sure your criticism makes sense though - they did this pre chatgpt. they are talking about the language models of that time. they did not make predictions about the future.