alt Hacker NewsHistorically, no.
The simplest way to build autocorrect is to have a dictionary, look it up for each word, and suggest the "closest" dictionary word if a typed word is absent from the dictionary.
It has existed way before transformers and on platforms that couldn't dream of running even the smallest transformer.
What do you mean by "look it up" if not measure the closest dictionary word by edit distance?
I think you'll find the broader idea of word vectorization to be remarkably similar to what text transformers do.
https://en.wikipedia.org/wiki/Edit_distance