>The original universal transformers is called "universal" because if you allow for per-token looping decisions, it can theoretically be Turing complete without needing CoT (some nuance here about levels of precision used).

Looping the transformer is just as turing complete as CoT. It doesn't fundamentally grant it any new theoretical capabilities. You could just scale the model into infinity with infinite context window.

Turing completeness doesn't care about the efficiency of the underlying implementation, which is fine in theoretical computer science, but if you have a model with a finite computational budget, you do actually care about the differences between write only tape vs read-write tape and single tape vs two tape. Having a fixed number of registers like a CPU also helps with reducing the number of redundant operations.

We see none of that with looped transformers, maybe we do see a fixed number of registers.