Ultimately, the brain is just a bunch of neurons activating in a specific pattern. This observation does not really tell us anything though. It doesn't acknowledge the difference between a 2500 Neuron fruit fly brains and a human brain.
Likewise, the fact that LLMs are a stochastic autoregressive process (which is a class of systems every bit as rich as the ODEs used to model neurons) tells us nothing a priori.
One is an observation the other is not, it's a description of what it is; one is a posteriori, the other is a priori (contrary to what you say).
They're not comparable.
Absolutely. If someone makes the weights do continuous learning etc then perhaps an llm can internalise morals. Of course, just like a human, it will be possible to talk it out of those morals. Another recent thread about this is https://news.ycombinator.com/item?id=49744420