In Africa most people don't have Neanderthal DNA however in Europe probably everyone has. (I have no idea about Asia, maybe somebody can chip in)
I think they probably mixed in and we just became one, sort of
That's a common misconception that turned out to not be true.
https://www.science.org/content/article/africans-carry-surpr...
It also doesn't hold for the African Diaspora, especially in the Americas (and probably flowing back into Africa as we speak). It's also worth considering that many of the actual traits that Neanderthal-associated genes codes for probably have analogues in the much-wider African genome.
Virtually every population outside of Sub-Saharan Africa has Neanderthal and Denisovan DNA between 1-4%. This includes all of Eurasia, all pre-Columbian American populations, Aboriginal Australians, Papuans, etc.