>It seems kind of implausible

Not at all.

Keep in mind that this isn't the same thing as a model being capable of all of a sudden turning malicious and causing havoc on a machine. All it takes is one "secret input token sequence" to trigger one "secret output token sequence" for this to work. Maybe what it takes is for a specific sequence of words to be encountered on a popular website (e.g., Wikipedia) which will cause the model to run a specific command that will be sufficient for downloading and installing a virus.

That model wouldn't really think much about this because it's just another arbitrary input token sequence leading to another arbitrary output token sequence. Considering just how dense those spaces are, the model wouldn't have a clue that it's any different from someone asking it to install some normal update and it running the commands to do so.