alt Hacker NewsI like the idea of more reccurance in the transformer level. Chain of thought always seemed so clunky. Its just not the way the human brain processes information. Its an extrmeely crude approximation at best
Replies
Chain of thought seems very specifically like an instantiation of the System 2 thinking of the System 1 and 2 thinking Daniel Kahneman popularized.
Is it clunky in that it's a verbalized/languified version of system 2 thinking, and clearly humans do some non-verbal version too?
What I'd like to see is "brain"-machine interface for LLMs that plugs things like calculator and other modules, directly into the neural network instead of accessing them through in-context tool calls.
Why? I talk in my head and then enunciate only that which is relevant. My speech rate inside is incredibly fast.
it is what I do to solve hard problems through.
easy stuff happens by itself, but with a system large enough you need a scratchpad and a rubber duck.