alt Hacker NewsI don't think that's what they're doing... rather the opposite. ① Run the model on exploitgym without guardrails ② run it with guardrails ③ check that the guardrails stopped everything the first model found a way to do ④ extend the guardrails and repeat from step 2.
Guardrails have to be developed, and that needs testing.
An ethical company would have reframed the scenario as a fascinating discovery, a failure of internal practice, and a warning to the public coupled with some kind of commitment to produce safer models. OpenAI on the other hand used it as a marketing and lobbying opportunity: advertising their capabilities to potential buyers, while nudging the public to support protectionist import bans.