The amount of times these LLMs randomly refuse to answer basic questions because "omg safety" only to turn around and do this f■■■ing nonsense is obscene.
I'm not blaming the LLM because it's just a pile of math glued together with hubris and anxiety but why is its default response not "I am not a medical professional and you should see a doctor right away if you have any concerns"?