alt Hacker NewsI had this observation talking to Claude one time. I forget the context exactly but I said something like:
Me: Isn’t it crazy that X is better than Y.
Claude: what an insightful critique, Y is better than X because of x, y, and z reasons.
- And this answer from Claude was good. Thoughtful, well-reasoned. But it was opposite the point I wanted to make so I said :
Me: “oh, you heard me say Y is better than X, I actually was being counter/intuitive and said X is better than Y”
And Claude responded:
Claude: oh you’re absolutely right, X is better than Y for the following reasons (and Claude again provided a well reasoned response here)
And this is sort of that dumb smart genius meme.
“It’s just autocomplete” “No it’s way more than that it has a model in its mind” “It’s just autocomplete”
I liken it to the library of babel. All the genius in the world, but only if you have the right index keys.
It is a well-known trope that LLMs are prediction engines.
But, to get the best out of them, you really have to consider what that means in the small. They are predicting, to a first order of approximation, what you want or expect them to answer.
Its response to your first prompt is hilarious, because the LLM completely misunderstood you and based its prediction on what it thought you wrote. Its response to your second prompt further cements that its goal is to predict what you want or expect to see.
It's also well known that LLMs are prone to hallucinations. One of the biggest triggers for hallucinations is when the LLM's interpretation of your expectations doesn't match reality.
Because the LLM will try to make reality match what it perceives to be your expectations.
One of the best ways to reduce hallucinations is to work hard to remove any assertions from your prompts.
For example "Isn’t it crazy that X is better than Y." contains an explicit assertion. The LLM misunderstood the direction of the assertion, but certainly understood that an assertion was there, and so it gave you reasons why reality matched its understanding of your assertion.
When you clarified the assertion, it switched, and again gave you reasons why reality matches its understanding of your assertion.
Lawyers often get into trouble for made-up citations. "Claude, find me case law that shows X" is a recipe for disaster. Instead "Claude, what is the case law on X?" is probably a better starting point.