That's an interesting thing to feel.

I think if you knew more musicians personally, you would feel different. What you're imagining and the day-to-day lived experience of most real musicians don't exactly overlap that much.

So in a sense you're ruining the music for yourself based on nothing but you're imagination.

I think a lot of modern life suffers from this problem. Most of us live such walled in lives now, experiencing everything second hand, from a distance. The absence of information lets us fill the gap with anything we want, which ends up being more a reflection of our own psychology than reality.