I think the least of your worries will be immigrants when an imperialistic nation with superior military power realizes that the annexation of your land is necessary for its survival.
Why would the US annex canada? It is already in the american hegemony.
I actually am moderately concerned with that to be honest. If the US doesn't have free and fair midterms I don't think it's really that paranoid to be worried about it.