alt Hacker News> AIs may downplay the seriousness of things. I was chatting with various models and most pushed me in the direction of “you don’t have cancer” for a while. Just something to note.
This is certainly another data point for "never trust AI chatbots".
Replies
Disagree. You don't take the evaluation at face value. You still push back and explore every thread. You don't abandon your own logical reasoning.
But you definitely want to scour the entire search space for solutions.
And the most efficient means for that is LLMs.
It's my experience that doctors are compartmentalized in their thinking, limited in their time, and definitely will not be extensively exploring your case.
E.g., I had 30 years of resistant hypertension. Multiple medications. Saw many cardiologists. Nephrologists. I was told to "just lose weight." BP was erratic over the years. ER visits. It was awful.
I googled. Pheochromocytoma. Hyperaldosteronism. Asked the cardiologist, "Could it be this?"
"NAAAAH, just lose weight."
But I insisted on a referral to an endocrinologist. And when I suggested it to them, given my extensive history, they were basically like, "Oh yes, we'll get you tested. I'm surprised you haven't been tested before?"
I had a left adrenalectomy 6 months later. BP has been controlled ever since.
They don't know everything. They can't.
This is the opposite of the experience people have when googling symptoms, though, where you often find articles suggesting that you’ve got some very serious condition when you do not. Interesting that the human bias when reading WebMD type articles is the opposite of the bias of LLMs.
Statistically, it makes sense- most people asking AI if they have cancer don't have cancer, so a model that is lacking detailed information should basically be returning based on the basal rate of cancer in the population with a threshold applied.