logoalt Hacker News

sweezyjeezy • yesterday at 5:43 AM • 1 reply • view on HN

My question is not how do you defend against it, my question is whether ablitteration makes the model completely oblivious that it shouldn't be doing 'the bad thing'. Like if you manually prefilled a normal model's output with that action, it would likely go "oh wow I can't believe I just ran that, I must warn the user of this now..."


Replies

vikramkr • yesterday at 2:08 PM

Probably the answer is it depends on the model and the compromise technique and there's not going to be any sort of consistency at all