> I don't want a model that says "I don't know", because I will verify the answer anyway.

Few people actually review answers or code. Because they have been sold the myth that these models can do it all. The main problem is that LLMs dont have causal models, and as a result, their reasoning is a high probability word salad and not a logically sound argument. Particularly on tricky corner cases which it hasnt encountered. I would still agree with you that sometimes hallucinations are actually useful as it provides a strawman, and having even a hallucinated answer to spar with is better than a "dont know".