alt Hacker NewsWhen they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct
Replies
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility)
Yeah but what stops it from producing confidently incorrect outputs...
I read "hallucinations" as "generates novel output with no grounding/source". i.e. "it just made something completely up".
I believe their "accuracy" metric (sonnet 5 level) is where "right/wrong" is measured.
What about the LLM calls though that are done midchain? In the Home Assistant video the multi-intent prompt gets split using what looks like a traditional llm model, which I'm assuming is vulnerable to classical hallucinations.
That's really funny when you consider that generative models also don't hallucinate if you check up on them on every token generated?
if it puts a high confidence value on a wrong answer, thats still hallucinating, no?
llm hallucinations are high probability tokens that are incorrect vs the real world