alt Hacker NewsI realize hallucination has no precise definition but this doesn’t sound at all like anything I’ve ever heard called hallucination. Hallucination is usually plausible wrong answers or made up info that ends up fitting the most likely response (like a manufactured citation) and comes from the way LLMs work at predicting tokens. This example demonstrates completely implausible output, it’s not something that fits with hallucination.
All that said, it doesn’t require cross session leakage, it could just be training data or like those nightingale (probably the wrong bird*) data generations where they just prompt an LLM with nothing and it starts spitting out conversations.
I see a bunch of downstream comments about caching, sounds like maybe there’s an error where it loads nothing instead of the cache and so starts spitting out random generations.
* edit: it’s magpie. Worth looking at the concept, I’m not sure people realize they LLMs generate random conversations when prompted with nothing, this seems at least as likely as sessions leaking: https://github.com/magpie-align/magpie
Replies
One of his tool results mentioned the word minecraft.py, and the response was about Minecraft.
It's a hallucination.
The word “hallucination” has become overloaded, but it general means an LLM producing some output that isn’t plausible or grounded. When you have a very long context session where the context includes “minecraft.py” it’s not hard to extrapolate that Minecraft may have ended up in one of the reasoning traces and that distraction snowballed until it appeared in the output.
These effects are becoming more rare as the SOTA models are improving so much. If you spent a lot of time with earlier LLMs or you experiment with smaller, quantized local LLM models this type of thing happens very frequently. When you see it happen so much on a model you’re running on your own hardware it becomes a reflex to chuckle and reset the session with a clean context. When it happens from a hosted provider it can be scarier because it’s not the type of failure mode most people are used to seeing.