> Build a new city from the ground up with a bunch of cheap modern housing, walkable and all modern goodies and ... nobody will move.
If you build a city full of empty buildings in a place with minimal existing population like the Alaskan wilderness then obviously. But what if you double the housing stock in all the existing cities where people want to live?
> Ground space is limited, so one solution is to build more vertical, but it is costly and has its own limitation.
Extremely tall buildings can get pretty expensive, but moderate height buildings (e.g. five stories) have similar per-unit construction costs to single family homes. Meanwhile now count the number of places you can draw a 100 mile radius where the median height of the existing buildings is even that tall. It might literally be nowhere in the US.
Notice also the extent to which density can thwart the scarcity of land. You put a five story building with four units per story on a plot of land instead of a single family home and the contribution per unit of the cost of land has gone down by 95%.
> Meanwhile now count the number of places you can draw a 100 mile radius where the median height of the existing buildings is even that tall.
100 miles? My dude, that is a circle from Poughkeepsie to the tip of Long Island. Forget one-story single-family housing, there is farmland in that circle. There are hundreds of square miles of state parks in that circle.
If people were willing and able to commute that distance you could easily quintuple the housing stock in that area building nothing but one story SFH.
Manhattan itself has a median height of 4-5 stories; the outer boroughs bring that down to 2-3, because commuting from Far Rockaway to midtown is already a schlep.