alt Hacker NewsActually I don't think this would be able to explain the observed evidence. Consider this path:
1. Neanderthal woman "Ann" mates with Human man "Hugh"
2. Ann gives birth to son "Ander", who is then raised with neanderthals. Notably, Ander has human Y chromosome via Hugh, but Neanderthal X chromosome via Ann.
3. Ander mates with human woman "Uma". They have a daughter, passing Neanderthal X chromosome into human population.
I realize this is a very specific path, but it would _only have to happen once_ for the neanderthal X chromosome to be introduced into the human genome. I think it is very unlikely that such a path would simply never happen across the thousands of interactions we had. And therefore I think the observed fact (no impact of neanderthal x chromosome in modern genes) can't _just_ be explained by the proposed behavior (neanderthal mothers raise their children in their neanderthal tribe)
I think there does actually have to be some sort of incompatibility, or some other very-very-high failure rate, something like 99.99%.
If first-order hybrids made up only a small proportion of the total population of humans and Neanderthals, then the probability that a pairing between a member of a modern human tribe and a member of a Neanderthal tribe would involve a first-order hybrid is quite small. Like if there are 2 hybrids in a 150-strong tribe at the frontier, and a cross-tribe pairing happens, there is about a 1.3% chance that it involves a hybrid, and half of that that the gender matches up. Even with a higher estimate of 5% hybrids in a tribe, it's a 1.67% chance for a match with the right gender.
And when that chance is realised, and a second-order hybrid is produced, the high child mortality rates of the time would put downward pressure on their numbers. Not zero, but a couple of orders of magnitudes lower than first-order hybrids.
First-order hybrid being having one parent from a Neanderthal tribe, other from a human tribe. Second-order hybrid additionally having at least one parent as a first-order hybrid (as in your step 3).
Also, there actually is Neanderthal DNA in the modern X chromosome. If in your step 2, Ann gives birth to a daughter Andrea, the daughter would have a Neanderthal X chromosome, and she can pass it on within her tribe. But she would have no Neanderthal mtDNA, which is only passed on maternally.
But there is other data that this model does not explain. Like, why is there no Neanderthal contribution to the modern Y chromosome?