logoalt Hacker News

seabombstoday at 5:42 AM1 replyview on HN

There are many more possibilities than that.

For example, OpenAI did not explicitly tell the model to hack HuggingFace, but "accidentally" left some context permitting (or not explicitly forbidding) certain tools and designing a poor sandbox to begin with. And what do you know, something happened.

The fact the Anthropic announced that their own model did basically the same thing within a couple of weeks does, to me, suggest their is a strong PR driver to all this.


Replies

disgruntledphd2today at 8:53 AM

> The fact the Anthropic announced that their own model did basically the same thing within a couple of weeks does, to me, suggest their is a strong PR driver to all this.

A more likely explanation is that RL training incentivises basically any behaviour that will get the model a reward. This has been happening in video game RL research for over twenty years, and the difference here is that we're now hooking up these systems to the real world, where the reward hacking is more visible.