I think the demonization comes from people resenting the use of authority to restrict liberty, and from conflating the resentment of authority with bigotry. In this case, let private people discriminate freely.

"The trouble with fighting for human freedom is that one spends most of one's time defending scoundrels. For it is against scoundrels that oppressive laws are first aimed, and oppression must be stopped at the beginning if it is to be stopped at all." -H. L. Mencken

I think people should be allowed to make their own choices in terms of whom to hire/associate with, with absolutely no outside intervention. That doesn't make me a bigot.