alt Hacker News> The latter is a strategy: there usually isn’t a correct answer, now or in the future. The model is playing a game consisting of repeated rounds, and the only way to evaluate it is to see how well it plays
This is self contradictory. You can tell a correct answer as you said, by looking at the score. In my $DAYJOB I am making hundreds of RL environments that produce a score for each intermediate state.
Replies
drob518 • today at 4:09 PM
Right, and I’d add that in some games there is actually a “win” state that serves as a binary indicator. Given this, you can actually view the model as the thing being classified into two states: (1) consistently wins the game and (2) doesn’t consistently win the game.
Sure, you are scoring them. But I doubt that you are scoring them in the sense that you are minimizing loss where the loss is a proper scoring function that treats the model as a classifier where that treatment as a classifier actually means something. Unless you happen to be RLing a problem where there is either exactly one correct rollout or there is actually a bona fide natural distribution over rollouts independently from the model.