Statistically you'd expect some "random" samplings of the population to show significant deviations from the mean just by chance.

Roll enough different sets of dice and you'd expect some to end up being all sixes - that doesn't mean that set is rigged. Yeah, they're the ones you'd do further tests on, but it's not evidence in itself.