Here is the 1870 census of the USA: https://en.wikipedia.org/wiki/1870_United_States_census
Note, indigenous people were less than 1%. New York City had the largest proportion of immigrants at 45%. Your claims of "only natives were there" and "everyone else immigrated from somewhere" are demonstrably false.
How is it false? “Natives: 25,731, total population: 38,925,598“