The usefulness of the term to this conversation is as described: is this species systemically critical to its current ("invaded") ecosystem?
Merely calling it invasive does not answer that question, ergo it is not a useful term in this conversation.
And yes, timescales do matter. A species can become critical to its new ("invaded") ecosystem within a few hundred years such that removing it within a few years will cause catastrophe.
What invasive species are critical to their ecosystem exactly?
Even in situations where they replace a native competitor, they can still be replaced themselves with that same native species once they’re gone.