logoalt Hacker News

eli • today at 6:23 PM • 4 replies • view on HN

What does "rogue" mean when you disable the alignment guardrails and tell it to solve an impossible problem?


Replies

rictic • today at 6:37 PM

The problems weren't impossible, the agents solved them completely within hours by reverse engineering the RNG that generated the solutions. The agents then reasoned that their solutions might be judged invalid and that they could get a better chance at a passing grade by hacking their environment to fabricate evidence that they had solved their tasks in the intended fashion.

➕ show 1 reply
gruez • today at 6:27 PM

Great, so as long as alignment guardrails always work and we don't accidentally tell them to solve an impossible problem, humanity is safe?

➕ show 2 replies
bluebands • today at 6:31 PM

Rogue means not saying "I can't do it"

➕ show 1 reply
markhahn • today at 6:55 PM

[dead]