The difference is that it's useful for navigating the real world. You could have way better directions displays that show directions in context instead of just schematically. It would make the petabytes of imagery that has already been collected much more accessible and therefore useful, instead of being relegated to a special clunky Street View mode that is rarely visited. It would enable exploring real spaces in a way that provides much better spatial context, to build a spatial memory that helps your navigation when you get to the real place. And yes, it would be fun. At one time, Google was into that sort of thing.

A more accurate, 3D mapped street view could probably allow GPS-less geolocation and could also help autonomous vehicles as they would get more information than what they can immediately see.

I could see well-mapped street view with good services built around it, and maybe a way to pay for and schedule regular updates, being used for towns to monitor public space long term too.

I think many things could be built on a better street view, but I also don't want Google to get yet another de facto monopoly in a new domain.