alt Hacker NewsThis is pedantic, but, actually RL has improved the quality of sentence construction in LLMs quite dramatically... And once you do some RL on that model, it aint a next token prediction machine any longer.
This is pedantic, but, actually RL has improved the quality of sentence construction in LLMs quite dramatically... And once you do some RL on that model, it aint a next token prediction machine any longer.
That's just training, isn't it? (Even if the term is "post-training") It doesn't change how inference works.
My understanding is it usually strengthens the "thinking ahead" part of inference, but that part was already there, and it's still at the end of the day picking one token and then purging internal state in a way that can only partially be recovered from.