It's more or less obvious that the LLM is a lossy-compressed version of the training data; it reproduces sequences of tokens that are the sort of thing that could plausibly occur in the training data, and avoids sequences that are implausible.

Because most of the training data has good grammar, the LLM is strongly trained on grammar; it will rarely predict ungrammatical gibberish. Even if there are grammar mistakes in the data, they are not systematic and so don't reinforce each other.