logoalt Hacker News

zahlmanyesterday at 10:39 PM2 repliesview on HN

> I'm not sure what you want me to do with that information

For example, you could cite specific things that you believe to be "AI tells" or "admissions".


Replies

Planktonneyesterday at 10:47 PM

It's a short article; you could read it. One example to get you started is the very first sentence:

> Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete.

The article is about how 'next-token predictor' is the wrong mental model; it opens with the admission that it is not the wrong mental model.

show 1 reply
angoragoatsyesterday at 10:50 PM

Not the person you’re replying to, but I read the whole article as an admission that it’s still a next-token predictor. More specifically: what does RLVR fundamentally change that somehow makes the whole process no longer a next-token predictor? The article makes no attempt to explain this. Additionally, I find its framing of the term “next-token predictor” as meaning “predicting the next token only based on raw training data” in common usage to be a bit dishonest.

To summarize: yes, RLVR and other synthetic training methods exist! It’s still a next-token predictor, and it does not “learn” or “think” or “reason” in the human sense, like so many people seem to believe.