alt Hacker NewsFive years on, which term do we see as less accurate to describe LLMs? Artificial Intelligence or Stochastic Parrot? I guess it's still an open debate.
Replies
The latter is definitely more colorful, and reflects a parrot's tendency to glom on to patterns. "Not X, but Y" being one of the more infamous ones.
Once in frustration I called a certain frontier model "Sam Altman's Tin Bird" to another agent with memory, and ever since then that other agent refers to ChatGPT as "the tin bird". Definitely a RAG artifact more than an attractor in that case, but I found it amusing.
Though, I would point out that where people fall on that seems to correlate very highly with their ability to explain how an attention head works.
I think "(intelligent) language understander" is an apt term. It contains within it the fact that these models are mainly trained on text, and "understand" it beyond a simple token-by-token level (i.e. their latent space maps to more and more complex concepts).
It also separates them from "world understanders" since any understanding they might have about the world comes from text (or images if we include multimodal models). They do not gather experience, memories or other "qualia" that many people (me included) would probably include in a definition of human experience/intelligence.
(fwiw i think artificial intelligence is a good, broad term, but it is both too broad to describe the current sota, and too loaded nowadays to be using in nuanced discussions)
This is a false dichotomy. Artificial Intelligence is more of a marketing term type of Hi-Fi or High Definition, ie. being a “suitcase word”[1], ie. it packs various different meanings and phenomena together to the point that without explication one cannot know what we are even talking about. Content recommendation system and LLM are completely different things.
What professor Bender is trying to explain here is that they were trying to describe how the LLM’s actually operate, to which point stochastic parrots is a fairly decent term. It is only disparaging if you know absolutely nothing how LLM’s work or you have some strange affixation to chatbots and believing they are far more capable than they actually are.
[1] Coined by Marvin Minsky: https://www.thekurzweillibrary.com/consciousness-is-a-big-su...
Its less of open debate would say, and although superposition [1] is interesting, as a way to explain power of some effects, it is clear they are right now closer to Stochastic Parrots than AGI.
Why do I say that? Because you can trivially beat most guardrails, simply by encoding your prompt in base64 for example. :-) Just word matching...no real understanding.
[1] https://chrisclay.substack.com/p/what-is-superposition-in-ne...
Spicy autocomplete
> Stochastic Parrot
Nearly all (99%+) people who use this phrase are anti-AI and just looking to show off how much they dislike AI and how clever they can be in insulting it.
So it's a great phrase because in just about every case I can ignore what someone says afterwards.
Similar to "glorified autocomplete."
Which frame inspires a more productive research program? Which has better predicted the trajectory of capabilities over the past five years?