alt Hacker News> It argued that large language models (LLMs) generate text by statistically predicting likely sequences of words rather than understanding what they are saying—a process the authors captured with the metaphor of a “stochastic parrot,” a system that repeats patterns without comprehension.
I don't understand what we're setting the record straight on. This is the core point of dispute, and the author just blazes past it to focus on other things. I'm glad to hear "stochastic parrot" isn't intended as an insult, and I agree that it's not right to think of LLMs as a box with a little homunculus inside replying to you. But to me it seems obvious that LLMs are not repeating patterns without comprehension and do understand what they are saying; otherwise they would not be capable of doing things they routinely do.
Replies
> But to me it seems obvious that LLMs are not repeating patterns without comprehension and do understand what they are saying; otherwise they would not be capable of doing things they routinely do.
So this seems obvious to you, and yet to many others, it is equally obvious that LLMs can/could do the things they routinely do without any meaningful sense of "understanding".
> But to me it seems obvious that LLMs are not repeating patterns without comprehension and do understand what they are saying; otherwise they would not be capable of doing things they routinely do.
Is it possible you're making the following error described in the article?
> The fact that these systems are designed to mimic the way we use language makes it very easy for people to mistake them for other people.
Clearly you don't believe it's actually a person ("it's not right to think of LLMs as a box with a little homunculus inside replying to you"), but you do believe it's doing something a little bit magical. Is it possible that because the interface is linguistic, and every other thing in your world that communicates with language is intelligent, that you're projecting something that just isn't there onto the situation?
I'm sorry if this line of questioning is a little invasive. But this is literally the "danger" the original paper talks about, and it seems an awful lot like you've fallen for it.
But it shouldn't even be contentious like that. It's not a fundamental mystery how these things work. It is for the most part not a valid target for the kind of speculation you seem to want to do about it.
It's not like you can be agnostic, or measured about this. It's like someone explaining a car to you, saying, "look here is where you put the fuel, here is where it ignites, where the axels are turned..." And you, trying to be measured, are like "hm well yes of course that all is clearly important, but there is clearly just a bit of magic here somewhere, between all the different 'parts'."
This is a facile point. Lisp expert systems transparently don't understand the meaning of any symbols they process, yet with enough developer elbow grease they can do all the same things an LLM can do, with much higher reliability. The fact that LLMs are less transparent than Lisp expert systems (and easier to program) is extremely bad evidence that they understand language. Especially given that AFAICT Opus does not properly understand concepts like "four."
I think it’s pretty clear that they are repeating without “comprehension” - both mechanistically (as in there is no facility for comprehension in their formulation) and in the ways they fail. The standard rs in strawberry, should I walk or drive to the car wash, etc all play on the fact that they don’t have any real world model or thoughts against which they can judge their output, as do many of the jailbreaks which basically play on the fact that the model has memorized patterns.
There are people who argue semantics, that we can call the pattern matching that LLMs do “understanding”, or the moronic “how do we know that’s isn’t all we do” but for the normal use of comprehension, LLMs at a fundamental level don’t.