alt Hacker NewsNorvig (and Chomsky) are talking at cross purposes.
Norvig confuses engineering for science. If an engineer wants something that works (and guarantees that it will work) then you use the most effective tool available to achieve that goal. No problem there. But there's absolutely no reason to think that tells you anything scientific about the phenomenon you're modeling. For example, aeronautical engineering says very little about how birds fly. And - currently relevant - LLMs learn to use and understand language, but they use millions of times more training data, and millions of times more computing power, than a child does. So an LLM may be a great technology, but it's probably not a good model for how humans learn and use language.
Chomsky OTOH doesn't see any use for statistics in science, at least not in linguistics. Fortunately for the field of linguistics, Chomsky's influence in linguistics is waning and linguists are beginning to use statistical tools to do science.
Replies
Humans decidedly take much more data than LLMs. Like orders of magnitude. We happen to feed only distilled data to LLMs though. That's why they appear so efficient.
> LLM may be a great technology, but it's probably not a good model for how humans learn and use language.
but from all models, it's the closest, "all models are wrong, but some are useful"
i agree. but it's uncanny that stochastic generation of language yields a form of recognizable intelligence.
in college, i was somewhat involved in cogsci wrt semantic mapping of linguistic concepts based on human surveys (this was well before LLM/AI-word2vec), and it surprises me today how the mathematical properties of relatedness/distance seem now to drive machine intelligence, something which we did not consider at the time at all.
maybe i'm getting in my metaphysical sentimental stage, a turn a lot of adults make after middle age, but i wouldn't fully dismiss the fact that there may be an equivalence.