alt Hacker NewsYep. It's nearly identical to the neural nets we were using in the 90s. Back then even a supercomputer wasn't big enough or fast enough to do what we do today.
I have to wonder though. Is this all a human brain is? A similar thing to an LLM just scaled exponentially larger. I mean a brain is not just neurons with simple connections to each other. The neurons, axons, dendrites, <insert_unexplained_thing>, etc in a brain are all holding and processing information in different ways and doing it nearly 100% in parallel. That's a really big model.
The biological discoveries show how complex a biological brain actually is. Even the tiny brains in a bee or spider are able to solve puzzles and use tools. That's crazy.
Replies
In the 90s you didn't have norm layers, residuals, attention, and some more.
So you're missing a lot of the building blocks that make LLMs. It's not a matter of just having the compute.
Attention layers were not used in the 90s.
LLMs are semiotic infrastructure. You won’t find a better analogy. The cognitive frame won’t hold.
> I mean a brain is not just neurons with simple connections to each other.
No, it's not. There are many animals that have extremely complex and even learned behaviour that have literally zero neurons.
Clearly "neurons" is an oversimplification just-so story, not a scientific theory.
Probably better to not simply reduce it by just saying X is Y then if it has all that extra complexity and capacity.
No, it’s definitely not what a human brain is. That makes very little sense. The ways we interact with language (and thus conceptual memory) is completely and fundamentally different.