On the other hand, LLMs can (and do) also find every mistake and fix it.

I'm not ashamed to admit that I occasionally ask my LLM to find and fix all the bugs in a file. And not ashamed to admit that it often does. Most usually algorithmic problems, not memory problems, fwiw, because I use modern C++.