alt Hacker News> diminishing these things as 'Next token predictors' seems absurdly reductive.
This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
Regardless, if all you want is a next-token selector, save your money and roll a die.
Replies
> which in no way challenge the established view that these bots are next-token predictors.
I mean, of course they are, that's literally what the inference loop does. You can look at the source of your favorite model runner and you'll see exactly that.
What I find misleading about this term is that it focuses attention on the "next token" part and glosses over the "prediction" part as some sort of unspecified "statistical algorithm" - even though this is where most of the work happens and where the interesting questions are.
> This shows a deep misunderstanding of the paper's claims, which in no way challenge the established view that these bots are next-token predictors.
No, this shows an appreciation of the symbolic richness behind that token 'prediction' which the paper leads on.
> Regardless, if all you want is a next-token selector, save your money and roll a die.
Tell me, where is the emergent symbology guiding that dice?