alt Hacker NewsThat chess analogy deeply confused me. Chess engines don’t compute win probabilities and choose the highest move.
I don’t think a chess engine is an apt analogy at all. In a chess engine, there is a concrete search tree and although it emits one move at a time, it’s actually picking the entire branch (of course, with iterative deepening as the game progresses).
There is no obvious place in transformer models where the entire trace was already computed prior to a single token being chosen. It’s possible, maybe even likely, that the whole trace exists internally as activations. Multi token prediction and diffusion adapters point to that being the case. But to my knowledge no explanation has been given for where in the model the future plan is stored.
Replies
Imagine a checkers engine then.
Yes I understand the analogy was a bit loose. I'm comparing what happens at "inference time" in chess engines to what happens at train time in LLMs. In hindsight AlphaGo Zero was the perfect analogy, but I missed that opportunity.
The analogy with chess still works, but there's an extra step to think about. In both cases there is some kind of search over possible future trajectories. A chess engine explicitly searches branches of the game tree and evaluates which moves lead to good outcomes. In RL for an LLM, you sample rollouts, evaluate the resulting trajectories, and use those evaluations to update the policy.
The extra step with the LLM is that you don't keep doing that whole search at inference time. You use the rollouts to update the weights, so in some sense the useful information from that search gets compressed into the model.
But if you accept that the model is, in some loose sense, storing what it learned from those rollouts in its weights, then at inference time they are doing a similar job: taking some input state (prior tokens or a board position) and choosing the next action.