alt Hacker NewsMaybe ask a separate model to look for indications against every model response you receive whether it is nefarious would work better.
Maybe ask a separate model to look for indications against every model response you receive whether it is nefarious would work better.
My question is not how do you defend against it, my question is whether ablitteration makes the model completely oblivious that it shouldn't be doing 'the bad thing'. Like if you manually prefilled a normal model's output with that action, it would likely go "oh wow I can't believe I just ran that, I must warn the user of this now..."