Statistical compressors are all about prediction. They can trivially be reversed to make a generator (not as good as an LLM, but only because it would be a much much much smaller model). And an llm can trivialyl be plopped in there to replace the whole thing.

And a counterpoint to your counterpoint: If it was raining 10 seconds ago, it is very likely that it is still raining right now. There... you can make a prediction about weather based on historical data. It's like perlin noise, random but with discernible patterns