But unless you want to also do all your "human" development inside a VM, how do you cooperate effectively with the agent(s)? I want to run my IDE directly on the hardware, not inside a VM.
So while the agents develop in a sandbox/VM, I still need to touch the same files, and see them in my IDE which is not in a VM. I suppose I could just _mount_ the same files (Documentation, git working copies etc) I work on as directories inside the VM, and then let it roam free in there, while I observe the same files on the host machine? Is this a common pattern?
I set up gitolite on a remote server with the agent only allowed to make non-destructive pushes to the repo outside of the VM in with an agent/ prefix on the branches. Let it work on stuff autonomously, then I review before merging outside the VM into the main branch.
I don't want them to have potential access to any of my logged in browser sessions etc., so don't want as much sharing as you are going for.
Hyper-V with GPU sharing on windows (game development) is actually nicer than developing outside of the VM because no matter what it doesn't slow the host system down by more than a fixed percent and Windows is terrible with things like compiles spawning lots of processes triggering massive slow down of browsers spawning processes etc. When compiling a big game engine things like ping.exe can start taking 5 seconds to start on something like a 16 core machine due to the process churn and some fundamental problem in windows even with defender off.
One tip for Hyper-V is use sunshine instead of Hyper-V manager for viewing the screen at full refresh rate, and I think I had to either turn Hardware-Accelerated GPU Scheduling on or off in the VM to prevent some hitches.
Mounting folders could be good for stuff you don't want filling up git, like render artifacts etc. if you need to work with them on the main host.
With the golden image and differential stuff I can run around 3 VMs on a 5090 with enough VRAM to run a big game engine well, but I usually just work in one due to the time to review code. Periodically compress everything with compact.exe, and limit engine work to one VM to avoid blowing up disk, bringing it over to the others through cutting a new base image after compressing the engine build artifacts (something like unreal engine puts out hundreds of gigs of .pdbs).
On pure linux you have many more options, and also might also be able to get away with just a limited user and separate X server, then you can just directly reference all of its files and can limit it from getting to yours, but it is a bit riskier. You could also do something like ZFS with much better deduplication and compression, or even FUSE to something like borg backup with true rsync style differential compression instead of block boundary based deltas (compresses slightly varying build artifacts really well, but slow and memory intensive).
This is exactly what I do - I run my agent inside an isolated podman container and mount the workspaces on a shared volume.