The majority of inference actually does happen in cpu.