I used a similar methodology for https://github.com/emberian/don feel free to dig thru anything in there I think all the notes are in the repo itself... of course it helps having a game that was explicitly designed around networked deterministic lockstep. forces the agent to learn a lot of lessons from reality instead of my typing :)