This sort of works.

The problem is that you'll end up wanting to run 2 or 3 (or 20, 100, 10,000) agents at once and that gets very hard with a single VM.

There's also an argument that you should be using a separate sandbox for each code operation a LLM performs (or at least each set of related operations). That's even harder to do with conventional VMs.