"A few centuries aren't long enough for such marked selective pressure on a polygenic trait."

Are you sure? In extremis, if blue-eyed people (a polygenic trait) are considered absolutely unfuckable, I would expect them to disappear from the population in 10-15 generations, or at least become very, very rare.