alt Hacker NewsI strongly suspect that if you dug deeply enough into how a human brain does reasoning, the answer would boil down to "it tries a bunch of pathways and picks the one that seems right". Pathways weighted by experience, but ultimately still guess-and-check. What would the alternative be?
Maybe our ability to reason is not as mystical and special as our ego might hope it to be, and discomfort over LLM's bringing that to light is the root of some people's urge to continuously downplay and discredit them.
(As far as I can tell as a non-neuroscientist, the literature on how our default mode network and prefrontal cortex interact agrees with my assessment - free association and logical verification respectively)
You have introspection, you can see a part of your thoughts, you know how that introspective part works since its what we call consciousness, you are conscious about it.
That introspection isn't an illusion, what your consciousness see of your thoughts obviously are tings that has been calculated and aggregated by the brain, so we know the brain calculates and aggregates those thoughts and feelings to produce its results. And we know LLM doesn't do that, it doesn't have a side system that does that sort of introspection.