Even so, one might wonder why we don't try making systems that take different approaches. For example, after a traditional first pass of output, they could do sliding-window "optimizations" considering each token in the context of tokens both before and after, and possibly replace words or phrases in-place.

For example, I've noticed quite a few cases recently of LLMs outputting "but" where "and" would make more sense, or vice-versa. Surely that could be improved by such an approach?