That's a common misconception that turned out to not be true.

https://www.science.org/content/article/africans-carry-surpr...

It also doesn't hold for the African Diaspora, especially in the Americas (and probably flowing back into Africa as we speak). It's also worth considering that many of the actual traits that Neanderthal-associated genes codes for probably have analogues in the much-wider African genome.