> Given the right resources and environments, many small models can function together in an emergent way.
What is this based on? Every researcher I've heard talk about this says it's exactly not true, as an uncontested rule, because the larger models will more effectively contain the smaller models, and use them together in ways that the connections between the smaller models can't. Remember, even MOE is to save compute/memory, not to help performance/parameter.
Does that mean an African elephant 2.57×10^11 (neurons) is smarter than a human 8.6×10^10 (neurons)?
That is my understanding as well. Thousands of monkeys do not equal or surpass a man, intellectually, even if working together. There is some intrinsic super linear scaling in intelligence.