I think it would have been clearer if they weren't called "numbers".
For example, look at matrices [1]: no one asks themselves whether they are "real" or not. We know that we have defined operations on arrays of numbers which are useful to some degree.
If complex numbers were just called "complexes", a lot of confusion would just wither away.
Also, interestingly, quaternions and octonions aren't called "numbers".
[1] Complex numbers may, of course, be seen as special kinds of matrices, but this is beside my point here.
They’re definitely numbers though -in many ways they’re nicer numbers than the Reals.
Complexes are overloaded as is - cellular, simplicial, chain etc.
We should call them "diternions".
Oh, but they can be considered numbers.
I guess what counts as a number is arguable. But -1 is a number, and yet it has two parts: the natural number, and the sign. Sqrt(2) is a number, and yet it cannot be written down exactly. It requires an algebraic representation.
Generally, I think we use the word "number" for mathematical objects that have some notions we associate with the natural number (which are what we most intuitively understand as numbers), like ordering, summation, a notion of zero, etc. Complex numbers do not have all of these, but they have enough that we consider them numbers.
Another way to think of this: most of these non-natural numbers arose as a way of being able to represent results of basic operations on natural numbers, that are not themselves natural numbers. These results could sometimes be operated on further to yield a natural number again. But if we did not invent new numbers for these, these result were not easily expressible, and led to gaps in our math frameworks. In fact, a lot of math progress was hampered by not recognizing these numbers, and progress followed once they were recognized. And the reason we call them numbers is that they arise from basic algebraic operations on natural numbers.