Yes, but they're not randomly distributed across the entire number space.
For example, receiving traffic from a given address is a pretty good indicator that there's somebody there possibly worth port scanning.
And where there has once been somebody, there or in the same neighborhood (subnet) might be somebody else, now or in the future.
Then it isn't random noise. It is determined by your own actions.