alt Hacker NewsNot reliably - look at microsoft's guidance project that did some of this.
Once a model goes through instruction fine tuning and RLVR/RLHF the log probs decorrelate from what's ground truth (given the propensity of the data seen during training)
Replies
dist-epoch • yesterday at 9:14 PM
I'm pretty sure these models also went through RL post-training, the question you ask still needs to be interpreted as an instruction.
But I assume it wouldn't take much more training to make them good again - just a matter of picking the right loss function, since this sort of stuff is probably all encoded in the last layer or two anyway.