They don't only reference a frozen training reality, they also reference the live rollout. Otherwise they wouldn't work at all.

They do continuously update from experience, but that only lasts for as long as the current rollout lasts.