I got a similar idea recently. I was becoming annoyed with the coding agent, running on my laptop, being unable to use sudo (with my supervision) when ssh-ing to my remote server. I wanted to build this as a PAM module. I started building this but after a while I googled (well… asked an agent) a bit more and found out that it already exists: https://github.com/z4yx/pam_rssh

I’m a happy user!