Models are not a collection of training data. The entire model is shaped by the training data it sees during training.

What we call knowledge is encoded as weights. Something we identify as knowledge is the sum of many weights across the model. However the same weight may be involved in very different topics. You can’t point to every weight and say that this one is only for a part of Wikipedia or Harry Potter that I don’t need. That weight could be active for a multitude of topics.

There have been a lot of attempts to reduce the size of mixture of experts models by selectively removing experts. The results are not good, but knowledge isn’t neatly separated within the model.