alt Hacker News> But the answers it gives are not reliable. They sound plausible if you don't know anything about the subject, but they're not reliable.
Do not underestimate the utility of having a starting point overview on a topic you know absolutely nothing about. It may be immensely valuable even if some details are off. That's what made the XVIII's Encyclopedia such a valuable tool for civil society.
By the time you get to the point where those wrong details become relevant, you have gotten a basic understanding of what the overall topic is about, so you're prepared to get a second opinion from a different source - and this time you may know enough to start asking relevant questions, rather than starting from full ignorance.
Replies
> a starting point overview
Which an unreliable answer is not.
> even if some details are off
Hallucinations are not a matter of some "details" being off. They are a matter of plausible, confident-sounding claims that are just plain wrong. They don't help anyone to get a "basic understanding". All they "help" with is getting a wrong understanding, that the poor person who's asking can't tell is wrong, because it sounds plausible and is stated with such confidence.
When humans do this, we call them "bullshit artists", and we don't view them favorably. Why should AIs get a pass?
> Do not underestimate the utility of having a starting point overview on a topic you know absolutely nothing about.
Perhaps, but we already had that in the form of search-engines and primers and how-to guides and Wikipedia. The actionable questions already had answers.
Adding an obsequious device that dynamically hallucinate half of a conversation with not-necessarily-true dialog is (if not a detriment) only a marginal improvement.