> I don't see why it makes a difference for this purpose that you're replaying network packets or controller input

Building a simulation that has perfect determinism is incredibly time consuming. Incredibly. Especially one that is identical across platforms, chipsets, and architectures.

Deterministic simulation replay also breaks anytime you change the simulation. Which is kind of obvious. But quite meaningful.

In any case, I’ve shipped games that use both solutions. And let me tell you, deterministic simulation from input is an order of magnitude more effort to build, test, and maintain!