alt Hacker NewsAfter having used LLMs for some time now, I don't agree with the concept they are just token generators, unless you think that's all humans are too. The way we test in most schools is just picking the right token. We also give them unique problems that they never saw in their training, which is the nature of programming. I realize they are probabilistic token generator models, but I find it harder and harder to accept that somehow there isn't something more going on. I'm not sure whether they are intelligent or not, but for the most part token generation is how you get degrees too. The thing is a parrot just says things it has already heard, it doesn't perform complex reasoning on novel situations and then explain it succinctly.
Replies
Here's the thing: most things people do does not involve tokens of any kind. It is, in fact, stuff that not easily describable. For example, it's trivial for a person to walk, but they cannot verbally describe what muscles they're activating in what order to make that happen.
Cognitive skills such as tool use and complex navigation predate language as well. That means there's a core of reasoning in humans that doesn't depend on "tokens" or "language" of any kind. Language is a tool for communication and forming complex human societies, but it's not cognition.
> The thing is a parrot just says things it has already heard, it doesn't perform complex reasoning on novel situations and then explain it succinctly.
Well a parrot does perform complex reasoning on novel situations all the time. It just doesn't have the wiring to connect that to "tokenized" human language. I suspect LLMs have the opposite problem, where they exist in the domain of their "tokens" and have no way to connect these to truly novel situations that have no existing words to describe them.
They are just token generators. It is just that 'just' does a lot of lifting!