Of course they do, error messages get tokenized and put into the context window just like anything else. This isn't a Markov chain.