logoalt Hacker News

ares623 • today at 7:05 AM • 1 reply • view on HN

Is it possible to poison a mainstream model this way?

Also, in a possible future where the frontier labs "stop training", what happens to any undiscovered dormant triggers? Is it something they can patch away relatively cheaply and quickly?

Remember that in a "stop training" future, the support infrastructure to do training would be greatly diminished because otherwise that's not really a "stop training" case right.


Replies

jacquesm • today at 12:06 PM

> Is it possible to poison a mainstream model this way?

Absolutely.

That's why you should never blindly trust model output.