Birth control was illegal in the USA until the late 1960s (other than condoms, which is what sailors used with prostitutes). It takes a while for changes to propagate through society.
Also, consumer credit was illegal until the same time period, and only legalized for people with vaginas in the mid-1970s. That alone might have made all the difference with marriage and fertility (which after all are only mildly related).
Imagine how your choices would expand if unlike your mother, you did not need to become Mrs. John Doe to be able to move out of your father's house.
So you're saying that the median marriage was forced back then? I don't see why the same reasoning wouldn't apply to men as well or who was forcing people into marriages besides some vague idea about "the patriarchy or something".
I was responding to OP's point about how our bodies are better suited to have children when we're younger. I think that's scientifically supported. My main belief would be that having children when you have peak fertility probably goes hand in hand. Birth control is a good hypothesis, you could say it broke this link.