They can estimate confidence based on distances in that state space.

But yes, it gets tricky.