alt Hacker NewsOf course it’s important to remember that the ability of an LLM to answer an obscure riddle like that has nothing to do with its reasoning abilities, but rather depends on whether the answer was included in its training dataset.
Of course it’s important to remember that the ability of an LLM to answer an obscure riddle like that has nothing to do with its reasoning abilities, but rather depends on whether the answer was included in its training dataset.
The word is in most online dictionaries for what it is worth. It's also used in Biblical texts, albeit only a handful of times. I do agree it's not a true assessment of an LLM's overall reasoning. No person I have ever asked that riddle to has gotten it correct. Then again, that is probably partly the point of the riddle.
I would like to reiterate that both Claude and GPT answered correctly. It was just bizarre how Claude got a initial, minor detail incorrect, but reasoned enough to get the more difficult answer correct.