logoalt Hacker News

verdvermyesterday at 10:21 PM1 replyview on HN

actually, bar is true

but wait, the models constantly go back and forth on these things in their thinking traces, so it is unclear which self correcting is actually correct


Replies

maxbondyesterday at 11:29 PM

It doesn't really matter to this argument, if they are self correcting at all, then we can't assume all errors will permanently injure the trajectory. It's not like dead reckoning or a similar process where there is never an opportunity to reassess. It's more like a long division problem; it is possible to correct errors using in band information, without the external reference dead reckoning would require. (The incidence of false positive self correction does matter to the question of whether the model is actually of comparable quality after the quantization, of course.)