I use Vimium [0] for this very reason -- highly recommend it! Although admittedly this only solves for a fully keyboard-driven _browser_ experience.

[0] https://vimium.github.io