What is inefficient is thinking in 5000 floats, and then quantizing all that computation into a 17 bit token, just to immediately turn the token back into a vector.

> Binary is more efficient than text for many system, including language.

It is not necessarily primarily about efficiency, but more the fact that a lot of thinking doesn't use language at all.