alt Hacker NewsI think "clean" means something like each neuron encodes exactly one concept and it's possible to discover what the concept is.
Replies
ky3 • today at 4:55 PM
Doesn't "orthogonal" convey that meaning more crisply and precisely?
➕ show 2 replies
I think "clean" means something like each neuron encodes exactly one concept and it's possible to discover what the concept is.
Doesn't "orthogonal" convey that meaning more crisply and precisely?
But they don't. That's the whole reason they work in the first place.
Every neuron in the layer encodes fractionally something about the concepts embedded in the whole thing. It's more like a hologram than a series of connected facts.