A general reason: they take up a lot of space. Not a problem in most of America, but that'd be a deal killer in Japan. If you're not familiar with Japanese neighborhoods, I'd recommend spending some time on street view - it's actually kinda cool. Make note of the parabolic mirrors at the intersections.

For America specifically: they're not a great fit for places where everything is spread out and the road system is sensibly designed. I can drive from one side of Tulsa to the other (on streets, not freeways) in a fraction of the time I could drive across a similarly-sized European city. That's because the city itself is designed for cars. It has straight major streets with 40-45 mph speed limits that form a grid. Neighborhoods sit inside the grid and the streets in them curve with the landscape. In most of the city you have maybe 3-5 traffic lights per mile on the major streets, so unless it's rush hour you get minimal slowdowns. Sometimes I can drive several miles without hitting a red light.

The ideal situation is to have a straight road with no traffic directly between you and where you want to go. Obviously, that's not possible. So you have to compromise. Roundabouts suck, but they're better than almost any other option in places that have twisty narrow streets and lots of pedestrians. Many (most?) American cities aren't like that (at least to the extent European cities are), so roundabouts don't make as much sense here.