alt Hacker NewsRail is great when you have a lot of people or things moving on the same path.
Highways are great when everyone has a different path.
Japan has most (but not all) of its large destinations on the pacific coast, which works great for rail.
I'm sure passenger rail networks used to have more routing options than amtrak does now, but it's hard to get between a lot of places by rail without going through Chicago. In the western US, you can go north/south in the pacific states or near the missisipi. Sure mountains are hard to cross, but there's no north/south in the plains either... Or Atlanta to Florida, etc.
Air is exactly the same but there are connections to everywhere from Kansas or Montana or wherever. I really think the rail situation in the US is primarily lack of investment. And it sucks; even here in Chicago where the connections are plentiful, the schedules and frequency are awful.