>There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.

You're mixing up two claims here, and only one of these is kind of true. Yes LLMs do internally plan ahead in a way that is emergent rather than strictly part of their architecture, so that part of your claim is true. The way you word it by saying they are "coalescing the probabilities of a range of tokens at a time" is poetic sounding jibberish though. What's actually happening is one distribution output for the next token computed from a hidden state that implicitly encodes where the text headed.

Your claim that if an LLM does happen to pick a token "th" instead of "tw", then the LLM isn't stuck with that decision is entirely false for autoregressive LLMs which is what all of the frontier models are. Whatever an LLM picks as its output token is final, it has no ability to undo that token selection and it must continue on the basis of that choice. It can't go back on that decision and revise the output.

If you're interested in this, Anthropic has a summary of a very technical paper on this topic that mostly deals with this issue with respect to poetry:

https://www.anthropic.com/research/natural-language-autoenco...

No, they really do one at a time. You're incorrect on that.

Mathematically, a long chain of conditional probabilities is equivalent to a single probability over the whole range. But computationally, for that to work out, the computation for the first probability needs to somehow consider all the downstream probabilities depending on it, which obviously isn't how autoregressive language models work. They can pack in as much downstream computation as their neural architecture allows for, which is quite a lot.

Suppose in some context you have three equally plausible conpletions after "Be": "tween a rock and a hard place", "twixed he stood there" and "lieve he can fly". To model this probability distribution of the whole sentence, the next token "tw" needs to appear at 2/3 probability and "lie" at 1/3. After "tw" would be a 1/2 chance of "ix" and a 1/2 chance of "een"; after "lie" would be a 100% chance of "ve " and in any case the rest of the sentence after that would be 100%.

The model needs to somehow "think ahead" to know those are the possible completions. For example if "lieve he can swim like a dolphin" was another equally plausible completion, that first token would need to be 50/50 instead of 67/33. So the computation of the first token somehow needs to encode the fact that the guy thinks he can fly but not swim, even though it doesn't become relevant in the output until several tokens later.

In practice this probably happens to some degree but definitely doesn't happen perfectly. To perfectly model the first token's probability distribution, it would have to include knowledge of the entire distribution of all possible outputs, which is just not happening. So it approximates. Surprisingly, the approximation is good enough to produce language.

You can see this breaking down in the seahorse emoji incident from last year. When you ask the model if there's a seahorse emoji, it first completes "Yes," as if a few tokens later it's about to produce a seahorse emoji. But when it actually gets to the token that would produce a seahorse emoji, it can't because there isn't one. But it's already outputted "Yes, the seahorse emoji is" and can't just go back and change that to "No, there's no seahorse emoji." Some models would try a few times and then say there isn't one or a system error seems to be making them unable to produce one, other models (including then-current ChatGPT) would loop forever with ensuing hilarity.

But on some level there is uncertainty, right? Even if it’s not token-specific but at the word- or phrase-level? Otherwise what does the temperature setting do? Or has architecture changed significantly in the background?

everyone in this thread is saying something kind of imprecise and reductive and varies between models and even modifications among the models

There are diffusion-based models and transformer-based models (and many other "architectures"), so your comment does not make sense.