Who is everyone else that has a problem here? Since tourists almost exclusively deal with people who work with tourism. Villagers annoyed that American tourists are swimming in their lake? Street scum irritated that American tourists are taking up space on the sidewalk? When I was young I used to hear anti-tourism sentiment sometimes, usually from the most base people. Everybody else was busy working or living their life, so they didn't have time to loiter in touristic places at touristic times. Unless they worked there.

As for your run-ins with card hostile businesses and people, you have the option to make your purchases with businesses who accept cards. Most customers choose that options, because cards offer the best protection and convenience for the customer. To the tune of the endless teeth grinding of some small business owners who think that their low profits are to blame on a tiny merchant fee.