You're right that "wipeouts" and "total population turnover" basically doesn't exist in our genetic history, but there's one critical exception to that rule of thumb: massive epidemics really do wipe out entire local populations, and given that early farming was involved (with its higher pathogen burden) this cannot be ignored as a factor here.