alt Hacker NewsInteresting comment: https://news.ycombinator.com/item?id=49264602
> IMO the much more likely explanation is that the brain uses contrastive learning somewhere. [...] Instead of using calculus and outer products to calculate derivatives, you feed real data and false data forward through the network and each neuron tracks and trains on the difference.
Replies
Kinrany • today at 2:42 AM
I'm curious if there is an explanation for it not being more common to sleep like dolphins do, small parts of the brain at a time.
➕ show 1 reply
Separately, do I understand the idea correctly:
- when learning on the real examples, just propagate forward
- when learning on the fake examples, first generate them by propagating white noise backward