alt Hacker NewsHumans decidedly take much more data than LLMs. Like orders of magnitude. We happen to feed only distilled data to LLMs though. That's why they appear so efficient.
Replies
suddenlybananas • today at 4:26 PM
The BabyLM challenge has shown that multimodal models don't seem to learn in a more data-efficient way with respect to tokens. Adding an unlimited amount of images doesn't help you learn syntactic islands faster.
If you are counting by bytes and including all sensory input, sure. In terms of amount of text, it's not close in the other direction. It's very difficult to make an apples to apples comparison here.