Welp, AI almost killing someone is definitely an "oh shit" moment.
The most interesting part is that there is no direct line between someone's accidental death and a chatbot giving life-threatening advice.
Imagine one of the models that has "accidental-deaths-via-bad-advice" just slightly turned up, with the model-provider's intent being to kill 5% more people per year.
Wonder how many AI deaths have occured that we dont know about(since they presumably died). With the adoption numbers we are seeing it much have happened already.