Every single person who utilizes a navigation application to traverse a place that they have no previous independently verified experience, is taking existential risk based on a computer telling them what to do
There are literally thousands of cases of people dying or being injured because they did what a computer navigation application told them to do
This is also literally what the Target stock scheduling system does for target employees for restocking shelves
The vast majority of peoples lives are run by someone else’s computer
That’s fundamentally different, and I think you know that.
It’s one thing to ask an algorithm how to build an A* driving map from point A to point B. It’s another to ask one how to be a better person and go to Heaven.
I’m not religious, and I’m not arguing this from a pro-religion POV. I happily work in AI, and I’m not arguing this from an anti-AI POV. I am highly technical. I love computers. I’m excited about the future. I rely on deterministic algorithms to make my days better. And yet, I do not want to trust the words of an LLM to counsel me on how to be a better husband or father. At this stage, the AI does not know me in the way a counselor or advisor, or even pastor or priest would. And yes, I think that’s a crucial difference.