But how is that any different than people being misled by real humans saying words that reflect real thinking, but which are actually dead wrong?

The fallacy here is "thinking == correct", not "tokens == thinking"