Yes, the hardest problem for me is the social aspect of opting out of consumerism. It makes most people feel really uncomfortable, like being sober in a group of social alcoholics. Clothes don’t look cool enough. Local vacations are seen as boring. Not going to restaurants is perceived as a lack of social aptitude.
Another problem is housing and living environment. Although it is very much possible to live in a smaller space, nice neighborhoods (quiet, clean, green) are expensive.
But basic necessities are almost free in rich societies, if you have time.