logoalt Hacker News

octoberfranklin • today at 3:11 PM • 1 reply • view on HN

But the signal seen on real sequences is much stronger than that

But the shuffled sequences aren't real sequences!

Sorry if I'm being daft here, but I still don't understand what the source of truth is for the shuffled sequence tests.


Replies

wavemode • today at 3:43 PM

To make an analogy - in one test the researchers took a real number from nature and asked the AI what it was, and the AI guessed it correctly a decent amount of the time. In another test the researchers just rolled a die and asked the AI what they rolled, and it couldn't guess correctly. The "source of truth" is just whatever the die roll was.

It may seem obvious that the AI fails in the control experiment, but the purpose is to rule out the possibility that information about the correct answer is leaking into the model context somehow. Or that the model is relying on some statistical technique like frequency analysis.

➕ show 1 reply