It’s the trump admin. They don’t care about the decorum you’ve described. They would have no qualms about looking racist. Have you not seen what ICE has been doing? Racism is a badge of honor, and so is flipping off the courts and public opinion. No I believe this is simply paranoia and racism driven by Miller and his cronies.
There have been cases of British, Bulgarian, Canadian, German and Irish nationals also gotten in their claws. Seems pretty race agnostic to me.
It's not about "looking racist"; or at least, it's not about public opinion. A racially targeted measure would violate specific laws and would be challenged in court, likely successfully.