alt Hacker NewsFor the umpteenth time a transformer is an arbitrary turning machine due to the fact that it has a theoretically limitless tape (the context).
A single transformer step is simply O(1) (for a particular parameter size), but it applied a potentially infinite number of times.
Thus while a single transformer cannot solve any problem, a transformer system applied in the normal way can solve any computable problem.
The context acting as a "theoretically limitless tape" while simultaneously the step being O(1) are contradictory. You cannot have it both.