logoalt Hacker News

Kim_Bruning • yesterday at 7:04 AM • 0 replies • view on HN

Wait a second, none of it? How about formal reasoning? Regular IF-THEN-ELSE can do simple logic, and prolog can do inference already. So are you saying LLMs can't do stuff that computers have been doing for ages?

To test this for some of my own uses, I've had this quick benchmark with progressively harder reasoning needed to understand novel prose. Each generation of models I've tested can unravel more layers of deliberately misleading writing; while meanwhile I've seen humans give up on the first question.

So either the models are applying reasoning, or some form of magic is happening.