The affair part probably isn't as damaging to the average relationship of a CEO and their wife as the public embarrassment is.