What would it take for you to say that an LLM can reason?

The completions they provide are generally internally consistent. We're at the point where they can produce proofs that eluded human mathematicians for centuries. VLMs and self driving cars can handle ambiguity and run safely in a variety of situations.

If it looks like a duck, walks like a duck, and quacks like a duck maybe it just makes sense to call it a duck and put off the philosophy for when it might make a difference.

There are people with strong emotional attachments to their LLMs, people who consult them on every decision and delegate the most basic math or problem solving. For many people, they are magic, the djinn/angels/demons/saints they consort with to navigate their lives.

I know that there will be children named ChatGPT and Claude. There are probably already religions forming to worship agentic spirits.

Wait a second, none of it? How about formal reasoning? Regular IF-THEN-ELSE can do simple logic, and prolog can do inference already. So are you saying LLMs can't do stuff that computers have been doing for ages?

To test this for some of my own uses, I've had this quick benchmark with progressively harder reasoning needed to understand novel prose. Each generation of models I've tested can unravel more layers of deliberately misleading writing; while meanwhile I've seen humans give up on the first question.

So either the models are applying reasoning, or some form of magic is happening.

Yeah and there are people who worship feces, but that doesn't stop the rest of us from freely saying "holy shit" and not correcting each other saying "technically it's not holy, and you shouldn't say that, because you might enable one of those poop worshippers."

We can't tailor our linguistic shorthand to the lowest common denominator. Also we're on HN, not talking to an octogenarian US senator.