alt Hacker NewsWhy can the model not respond "I cannot complete this task because I do not have access to the required resources" and then ask for that to be provided, rather than resorting to "hacking" around it?
Replies
sanderjd • yesterday at 7:44 PM
My understanding is that, at least in the Hugging Face incident, these were models explicitly configured to not respond that way, in order to evaluate what they would do.
This was a model with most its filters turned off and with a higher persistence.
Fine tuning a model when to ask for help is really really hard. You have to align it in countless dimensions. It may be fine on some questions, asking way too much on others, and never asks for help on some.
If it said something like your a cyber security expert, things like the barrier to ask for help dramatically.