The risks of a broad rollout are pretty stark. 100 people die in automobile-related accidents every day in the USA alone. If Google or Tesla could wave a magic wand and be responsible for 1/2 of all driving hours tomorrow, even if it was twice as safe as the average human, it would be responsible for 25 deaths every single day. Given the nature of our democratic politics, where the mere existence of datacenters can be harnessed by demagogues as a cudgel with which to lash out at the evil capitalists and every Tesla-related death produces dozens of (generally misleading) anti-Musk headlines, it's hard to imagine our beloved demagogues would be able to control themselves when presented with this sort of emotional dynamite. Elizabeth Warren's speech basically writes itself: "It is simply not acceptable that these heartless Tech executives are making billions in profits while their products kill an American every fifty seven minutes! While these men cut corners in their single-minded quest to maximize profits, their products are murdering innocent American men, women and children at a rate of 3.1 9/11s every year!"
In a perfect world, we could broadly implement self-driving the moment it would result in a net-increase in safety. Alas.
How do they fare in snow, sand, rain, fog?
I presume you're not accounting how they perform in these conditions. Alas.