Have you missed all the breathlessly excited blog posts from all the frontier labs about how they’re using their best models to implement their inference stack?
I bet it wouldn’t be very hard to write an inference stack that subtly leaked the weights into the output tokens :)