How do you know those people aren't staring at you because you started staring at them in an unpleasant way in response to them randomly glancing a look vaguely in your direction?
I don't stare at people. They don't stare at me either. Problem solved?
I check if people are looking at me. It’s useful and sometimes they do. Then we are locked in staring at each other like two cats.