The answer that seems to be emerging from several different lines of research is that a) they always had fairly low fertility and b) they didn't really go extinct as such, they just intermixed with Homo Sapiens Sapiens and because the later had much higher fertility, Neanderthal genes got diluted down to the present ~2% in the Eurasian population.