I do something similar with my ThinkPad and Steam Deck connected to different ports, and I change inputs and plug in a controller when I want to play a game then.