I was told (by my partner, who works with birds in many capacities) that they consider a "direct" or head-on look to be something of a threat, as a predator would do. Consider looking at them sideways, with only one eye -as they would to each other- and see if it makes a difference.