alt Hacker NewsI don’t have any kind of ML background but I have always thought of sample efficiency as the great unsolved problem of AI. We humans have unbelievably good sample efficiency; often we can durably learn something on just a single example or two. This is the main area in which LLMs are vastly, vastly behind us.
Replies
10xDev • yesterday at 5:30 PM
No, the unsolved problem of AI is continuous learning. We never stop learning, we don't have a "training phase". You are always updating your world model even when you sleep. Also more quality training data does lead to greater learning efficiency as you have more priors to work with.
➕ show 2 replies
the caveat is that we are not learning those small number of samples from scratch, since we're coming in with a large amount of training already, much of it from before we were even born