How do people around you react to your face blindness?

Do they consider not being remembered rude? Do you get incredulous reactions like people with aphantasia?