alt Hacker NewsI wouldn't single out the concern new diseases if the population is small. Most diseases co-evolve intra-population. The lethal ones are the ones that suffer a mutation and are suddenly able to be passed to a different 'species'. So, if they already survived on a 'knife's edge', immune variety is of comparatively low concern (but still existential) on the list of things that can end your species (climate change, competition, demographics - 2-3 infertile females in a group of 20, say bye bye to tribe).
Isolated populations are all going to be creating isolated variants of any disease doing well enough to stick around.
And the impact of events where any individuals die to a new variant, is amplified for a small population. The risks of highly correlated vulnerabilities are on top of that.
Variants of the flu continue to quietly emerge and kill people today. Despite all our regular exposures to their constant churn and weather shielded environments.
But you are certainly right that the cross-overs are incomparably worse. And diversity becomes species extinction protection at that level.