There's no way to model what a "tricky situation" may be to an opaque and ever-changing piece of self-driving software. It may fail in random ways at random times — it's completely, 100% unpredictable.

Therefore, you have to be 100% ready at all times to react in case anything that's possible happens.

Sounds way more tiring than just driving yourself and only having to account for the known, relatively easy to model human failure modes.