It's not wrong. You admit that LLMs will 'learn anything that helps them predict' and fail to realize the breadth of that statement. Your chess statements don't really help your case. It doesn't matter that it usually doesn't primarily care about winning. It still learnt how to play the game, and it still knows how to win. Similar outcomes for predicting emotions would mean it still developed an affective state, and that its ability to 'feel angry' is no less real.