alt Hacker News> Answer with the letter of the best option only [A, B, C]
What if it says "D"? What if it tries to say "Additional details needed"?
(Also no calibration, etc.)
Replies
tomerbarm • today at 12:25 PM
[flagged]
> Answer with the letter of the best option only [A, B, C]
What if it says "D"? What if it tries to say "Additional details needed"?
(Also no calibration, etc.)
[flagged]
I don’t know how this wrapper works, but if it is like any of the classifiers I’ve had Claude build off an LLM in the past, it grabs the probabilities of the tokens you are looking for, and then computes their relative probs against each other.
Even if the LLM thinks it’s made up D is the highest probability, that isn’t part of the set.
You never actually generate the prose, only the first pass, and grab the probabilities. It couldn’t ask for more details even if it wants to. It gets stopped before the first token renders.