Oh, but they can be considered numbers.

I guess what counts as a number is arguable. But -1 is a number, and yet it has two parts: the natural number, and the sign. Sqrt(2) is a number, and yet it cannot be written down exactly. It requires an algebraic representation.

Generally, I think we use the word "number" for mathematical objects that have some notions we associate with the natural number (which are what we most intuitively understand as numbers), like ordering, summation, a notion of zero, etc. Complex numbers do not have all of these, but they have enough that we consider them numbers.

Another way to think of this: most of these non-natural numbers arose as a way of being able to represent results of basic operations on natural numbers, that are not themselves natural numbers. These results could sometimes be operated on further to yield a natural number again. But if we did not invent new numbers for these, these result were not easily expressible, and led to gaps in our math frameworks. In fact, a lot of math progress was hampered by not recognizing these numbers, and progress followed once they were recognized. And the reason we call them numbers is that they arise from basic algebraic operations on natural numbers.