The lack of dedicated registers meant a lot of memory access, which slowed things down considerably.
It gets worse because the TI99 only has 256 bytes of RAM directly addressable on its 16-bit bus. All the other memory in the system is video RAM and is accessed 8 bits at a time through the video display processor. Oh, and you can only do this when the VDP is not accessing the memory. This is incredibly slow and severely hobbles the potential performance of the CPU.
Was that part of some kind of cost optimization? Why would anyone design such an obscurity?