A lot of the low hanging fruit is still dams that were made for non-hydroelectric, non-drinking, even non agriculural purposes. In the Northeast there are 19th century dams that originally powered mills before steam power. The barriers in some of these cases are just the cost of removing the dam, which can be substantial if it borders private property. Some of the flood control functionality is no longer necessary, because it can be migated and in some cases the land can just be bought out.

Most of these dams were either in disrepair, had geotechnical problems, or were sufficiently silted up they didn't generate much power anyway. A few dozen rooftop solar installations probably generate more power.