> Is it possible to poison a mainstream model this way?

Absolutely.

That's why you should never blindly trust model output.