logoalt Hacker News

bunderbunder • yesterday at 11:29 PM • 1 reply • view on HN

But benchmark performance is not the same thing as intelligence.

Which is a really important distinction. One measures specific points on the jagged boundary. The other is a word that blithely paints it all with a single brush, and sweeps the jagged boundary problem under the rug in the process.

It really is important to keep track of that. Whenever we talk about “intelligence” as if it were a well-defined (let alone quantifiable) thing, we fall into the kind of wooly thinking that lets anyone say pretty much anything. We abandon the realm of grounded statements in favor of pure vibes.


Replies

rcxdude • today at 12:03 AM

Eh, I get what you mean. AI especially seems to have performance that is much more difficult to reason about than human intelligence (where most aspects of mental capability are surprisingly strongly correlated). But in this context, if you broadly focus on intelligence as the ability to complete problem-solving tasks, models have been improving pretty uniformly at the same time as getting cheaper.