Well I doubt that :-) the image tokens can just be added as further tokens, so one wouldn't even need a dedicated output head. E.g. Emu architecture [1] does text, image, and videos all together. So definitely not new. That's not even a separate head, everything stays together.

Regarding whether that's standard to pre-train multi-modal LLM, apparently that is not the case (at least, according to ChatGPT's answer, but that's reasonable - or anyway it was only a wild guess on my side). Apparently it's more common to have a text-only and an image-only pretraining phases, followed by an alignment to make the two embedding spaces a common one.

[1] https://arxiv.org/abs/2409.18869