I do this sometimes too but the ux has always been lacking. Arrow key directions sometimes don't work if the system thinks the forward image is e.g. not actually forward. You try and resort to clicking, you might miss the click target and end up 600 yards down the road instead of to the next frame. Or you merely zoom in and don't move at all.

It has always been like this ever since they released streetview. I'm surprised they haven't improved it, but then again, maybe I am not surprised. Probably actually a good thing google hasn't felt compelled to touch it recently.

In unrelated news, I got a notice that they are killing google earth pro in January, so get your downloads in now before they stop sharing the packages.