alt Hacker NewsTo this day frontier models think that A and not B means A and B when the sentence gets pushed far enough back in their context window. The context length that model can reason over without obvious errors is much smaller than the advertised context. Between a 1/4th to a 1/20th what is advertised on the tin.
Replies
Npovview • today at 4:56 PM
Do you also happen to remember what you ate last thrusday?
➕ show 2 replies
Critiques like this tend to focus very hard on what models can't do. It's true, they have limitations.
But they're also superhuman in so many other ways. It's valid to point out limitations, but that doesn't support the conclusion that models are not incredibly powerful and capable of the functional equivalent of reasoning at human or superhuman levels in many scenarios.