It's so shameful.
We let people buy kitchen knives. But because the kitchen knife companies don't have billions of dollars, we don't go after them.
We go after the LLM that might have given someone bad diet advice or made them feel sad.
Nevermind the huge marketing budget spent on making people feel inadequate, ugly, old, etc. That does way more harm than tricking an LLM into telling you you can cook with glue.
Ad companies prompt injecting consumers. LLM companies countering with guardrails.
I don’t feel like that’s a reasonable analogy. Kitchen knives don’t purport to give advice. But if a kitchen knife came with a label that said ‘ideal for murdering people’, I expect people would go after the manufacturer.