Modern 3D APIs are built around passing around a few buffers per frame and long command queues. Should be doable with IPC.