alt Hacker NewsSome concrete facts about LLMs are explained by their next token predictor nature. Every time it says "wait, that's wrong." instead of generating the correct thing the first time.
Some concrete facts about LLMs are explained by their next token predictor nature. Every time it says "wait, that's wrong." instead of generating the correct thing the first time.
I think that's relatively emergent too though! BERT never really did that (at least to my recollection), presumably because its training was never sufficient for it to develop corrective reasoning in a chain of thought.