Disrupting someone's federally protected right to assemble and worship is a civil rights issue, just like forcing people to do that would be. If you stop and think about this a bit it's really obvious how "disrupt what's happening in a house of worship I don't like" would play out if it was allowed.

In fact there's history of black churches, synagogues, mosques and Sikh temples have been attacked, bombed, disrupted, and otherwise interfered with. You don't have the right to disrupt a girl's baseball team practice on a campus either. The Klan can't march through your living room.

This isn't hard to understand.