It's because LLMs' pretraining is basically a compression. Later post-training and fine tuning is something else, based on the compressed data.