When you are worth hundreds of billions, people start falling over themselves running to file lawsuits against you. We're already seeing this happen.
So spending $50M to fund a team to weed out "food for crazies" becomes a no-brainer.
It's so shameful.
We let people buy kitchen knives. But because the kitchen knife companies don't have billions of dollars, we don't go after them.
We go after the LLM that might have given someone bad diet advice or made them feel sad.
Nevermind the huge marketing budget spent on making people feel inadequate, ugly, old, etc. That does way more harm than tricking an LLM into telling you you can cook with glue.
It is a no brainer. If a company of any size is putting out a product that caused cancer we wouldn't think twice about suing them. Why should mental health disorders be any different?