I think it’s important to remember that the process of selection acts directly on human traits. For example being exposed to high summer heat temperatures may eliminate some people who have unproductive sweat glands, or needing to run down your food may eliminate people who have a muscles that easily tire. Selection (largely) does not act on far removed traits like egg cell characteristics as a proxy of human traits like muscle performance because the genes that are used by egg cells are quite different than those used by muscle cells. So if you worry about some kind of human trait decline you should be much more worried that people have access to air conditioning and grocery stores.