The record setting file compressors are all very generalizable. This comes about because you score compression by adding the program size (including any training data or dictionary) to the compressed data. If they didn't score it this way you could just ship a dictionary of the test data and your program just prints that. So of course they combine it and effectively measure the Kolmorogov complexity.

This means that training data isn't that helpful to top of the line compressors. eg. Fabrice Bellards nncp that's currently the record holding compressor for enwik9 is an LLM that learns on the fly. It's 628KB as shipped (LLM code complexity is high) and it wins on data sets gigabytes in size due to it's ability to create the training data on the fly. No pre-trained LLM comes close when you measure it as above (program size + compressed result).

So it's really not worth that much concern. There's a reason we all say AI is compression and we dismiss comments like the above. They don't pan out. No one's shipping significant dictionaries or pre-trained data. You want to win competitive compression? You'd better have raw code that learns on the fly and is naturally generalizable.