Made something very similar recently, minus the monitor input switching:

https://news.ycombinator.com/item?id=49823733

In this case the key thing was to make this work with an old MX Master, which isn’t properly supported for one-button multi-device easy switching.