I think some of the reason that the whole aloof thing has crept in is for self-protection. If you proclaim loudly you love something and others ridicule you for it then you quickly adopt the aloof persona yourself as a defensive mechanism against getting hurt.
Funny thing, looking at the handful of friends I care about - every single one of them, without exception has something they’re deeply passionately into. Photography, or Tour de France, or Design… None of these are interests I share, but the type of personality that can get deeply obsessive about something I find very appealing.