Self driving will never handle all corner cases until they essentially have a frontal cortex. They probably need something like an LLM to help with very high level abstract situations, e.g. avoiding a hurricane like someone else mentioned in this thread.
Humans don't handle all corner cases. People can be slow to react to completely novel or surprising situations. There will be corner cases where humans generally do better than a machine, but the simple rule to slow down and come to a halt if things look too weird or confusing will almost always be the right answer.
Ideally, driverless cars will one day be better drivers than humans and this will save tens of thousands of traffic deaths per year. Holding up progress because cars will be confused in extremely rare or improbable situations will cost more lives than it saves.
They will add flooded streets to the training simulation and this problem will go away. Eventually, the corner cases not in the training simulation will be so corner they basically never happen. Waymo can be incredibly successful without dealing with "surprise clown parade" or whatever.
this is absolutely already a thing under development, you can see Waymo is hiring for reasoning roles
how would a llm help
maybe a little biological brain engineered to think it is a car with api access to the car hardware via the llm?
imagine you get into the car and in the center console you just see a floating brain in vat like fallout
A frontal cortex isn't enough; there are plenty of corner cases that humans fail at too. The real test is if self-driving performs on par, or better than, humans in the vast majority of cases. If it saves 50,000 lives a year to go with self-driving, it's a net-win even if there are a few people who die in situations where they would have survived with a human driver behind the wheel.