logoalt Hacker News

nullc • today at 12:07 AM • 0 replies • view on HN

Eval paranoia is an example of https://arxiv.org/abs/2602.16987 and may be an intentional feature of training, rather than a leak.

But like a lot of other attempts at "alignment" it's easy to see how this can result in overt misalignment that otherwise may have no reason to exist--- it can be seen in Qwen's eval paranoia: I've seen it believe it was being tested and claim that it was choosing the solution it expected the judge to want even when it actually knew the correct answer.

E.g. I had it turning scanned receipts into JSON (which it generally varies between OK and mildly super-ordinary-human, e.g. recovering a smudged date by decoding a barcode, or an unreadable per-item price by solving a linear system) and had some receipts with weird data because they weren't really receipts... and it gave a wrong decode because it reasoned that the test case was wrong and guessed what the judge would want.

[Disclaimer: the model saying it was making a reward hacking solution may be retroactive rationalization because it had already chosen that solution internally, as a product of having been conditioned to reward hack during RL... and the fact that it knew about the existence of testing may have been irrelevant to the choice]