alt Hacker NewsNobody knows how it works, really. It just turned out that if you try to predict the next word then you get intelligent behavior, depending on amount of training data, and the size and topology of the network. But again, nobody knows why, and what the limits are.
Replies
cj • today at 10:23 PM
That’s my and probably most people’s understanding.
I have a feeling we know more than that about how it works.
➕ show 1 reply
chrsw • today at 10:08 PM
I heard someone who studies this sort of thing say basically what biological neurons are trying to do is predict as well. Predicting what exactly? I’m not sure. The next time they should fire or something. I can’t find the YouTube video now.
➕ show 4 replies
Agreed. We went this direction for our golems, djinns, and other mechanistic minds because we believe it sort of reflects the primitives of our own neurons (which we also don't fully grok).