The public was well-aware. They had stickers on shops. Your Jewish neighbors were paraded through the streets for deportation. Once they were gone, people took the furniture, the businesses, or simply moved into their apartments. On the country side, there were various land reforms where people who joined the NSDAP party were given fields from famers who were either simply deported as being Jewish or political opposition.

Of course people always had the feel-good lie "oh they're just being relocated to XYZ" but in those times you'd never leave your furniture and other valuables behind when moving if you were not forced to. For German people it was a win-win situation: More work for everyone (either as a party soldier or in the construction), steal some valuables from your neighbors who just got taken away, and feel good about your noble aryan genes.

Sorry for rambling on this topic but there are books for every mid-size Germany city which detail the unfathomable amount of looting, stealing and "M&A business" that was done by everyday "normal" German citizens during these times.

And most of these crimes were not prosecuted because of political decisions after the war.