Populations can grow, even rapidly, in the short run even with collapsed fertility. This is because in part it's a lagging metric, but more importantly (and less intuitively) because fertility also determines the age ratios within a society. Imagine a society has a common fertility rate of 1 then that means each successive generation is half as large as the one prior. So if there's 1 new born, then there's 2 20-40 year olds, 4 40-60 year olds, and so on. High fertility rates result in societies with far more younger than elderly populations, and vice versa for low fertility societies.
In the past most societies had high fertility rates, which means there are relatively few elderly, relative to younger people, today. So the global death rate is relatively low as the the number of people annually exiting the stage is relatively low. But when low fertility generations start to die off, this gets turned upside down. You'll have large numbers of elderly dying off, and they'll make up the majority of the population. But this only happens ~80 years after the birth of the low-fertility generation.
This is why Japan looks so catastrophic today even though their fertility rate is much higher than places like South Korea. They've had a mediocre fertility rate since the 60s which then went into collapse in the 70s. By contrast South Korea had a fertility rate of 6 in the 60s, and didn't fall below replacement until the 80s. So Japan has had a solid decade or two head start. The fact we still aren't at the life expectancy of the first below replacement generation in Japan also means today is still going to be seen as 'the good ole days' for them. And this is where most countries are headed.
This something people don't understand. Fertility rate only affects the current generation. You cannot retroactively increase the fertility rate to fix the problem.
If the current generation has a 1.0 fertility rate, that means the next generation will be half as large. If said generation now doubles it's fertility rate to 2.0 it will only maintain the size of that generation, all the old people will age out and be gone all at once.
Now you might say, fine, let's fix it, 4.0 fertility rate, that will only double the size of the next generation, meaning it will only fix one generation of 1.0 fertility. You have stabilized the population, not increased it.
Even if you do this, you now run into a huge bottleneck: parents need to take care of their old parents and their large family all at once. You fixed the biological problem but you're now running into economic collapse.