The 20W number includes EVERYTHING else the brain does. The chips/models are literally only producing tokens. Let's see an LLM drive a robot harness and have the robot produce speech, as well as move through 3D space, keep track of metabolic needs, etc. etc. etc. before we compare efficiencies. That is even assuming the tokens are of equal quality. This comparison is currently Apples and Oranges.

I couldn't source the parameters from the screenshot or the nearby graphs, but from the nearby graphs you can see that at concurrency C=1, tokens/Joule (vertical axis) has totally plummeted, and obviously concurrent inference is much more efficient by batching. Divide the memory by the bandwidth and thats how long it takes to dump the full RAM contents through the chip. Do you want to do this once per token for a single conversation, or do you want to progress multiple conversations if you're going through all the weights anyway? The peak in the graphs is easily 22x more efficient than the low bottom right part on the graphs. So in batched mode its already more efficient than human speech.

> Humans are still 22x more efficient, which is not that far considering the rate of progress in this area.

Based on a human output rate of 3.3 tok/s, which seems questionable as a means of comparison

At just inference! Which both a human and a model can not do without training, but while training rounds to zero for the model, for humans it scales linearly.

I am relatively certain we have already squarely been beaten in net efficiency at scale.

Probably not when you consider the training cost and upkeep expenses, not to mention the depreciation…

Fairly amazing when you think about it, like human intellect can run on a bowl of rice and a chicken yakitori skewer.

I wonder how that stacks up if you consider all the time you have to keep the body alive when it’s not actively producing “tokens”.

You’re missing the factor for intelligence/token.

I mean, surely when quality is accounted for the difference is significantly higher