You're very right about KL divergence. I spent a couple days playing with the Gemma 4 models. That's 10 separate models (varying weights, MoE, QAT or not, etc...) with identical tokenizers. I treated 31B at BF16 as the gold standard, feeding Wikipedia snippets, and anthropomorphizing a bit:

Gemma 4 31B: "Um, if I really said all of that, I guess I'd say this next"

Gemma 4 26B: "Dude, I would've said completely different stuff" (large divergence)

Gemma 4 12B: "Umm, there's zero chance I would've said some of this" (INFINITE divergence)

Gemma 4 E4B and E2B: "Derp derp, I'm happy to say almost anything" (lowest divergence)

For models which are chat trained, they simply would not recite Wikipedia, so the divergence is almost meaningless. I thought about capturing a realistic coding session and trying to use that as the corpus, but you need to preserve the turn-based tokens and such, so I moved on to other things.