alt Hacker NewsThe model doesn't "know" how to generate tokens any more than it knows how to stop generating tokens. The sampler simply stops pulling values when it outputs a "stop token", which is a token the same way every other token is.
That is to say, it stops when it's statistically the most likely to.
OK thanks, so the neural net (that no one can explain fully) generates a "stop" signal at a certain point.