alt Hacker NewsWell its not that simple. In the same way that throwing an LLM into a process will always have a risk of blowing up spectacularly.
In this case it failed open. It didnt recognize that it was in an edge case (which itself is an edge case). So what are you proposing to be the solution to that? If the car itself does not recognize that its in an abnormal situation that needs intervention then how do you intervene?
His point is that humans are prone to the same error. The flooded engine damage doesn't come from humans recognising the danger of a flooded road and choosing not to attempt it.