Yes, evolution is not going to penalize everyone with a "100-year brain" when the rest of our body seems optimized for something more like a 30-40 year lifespan. Evolution is all about propagating your genes, and not too many 100-year olds are going to be doing that!

In any case, fuzzy associative recall is really the "design spec" for a brain, so semi-graceful degradation and fuzzy recall comes for free.

I'm not sure if old-age memory unreliability is totally random - I would not expect it to be. I'd expect these associative recall failures to happen when there are too many memories partially matching the same "key" (activation pattern), which presumably happens mostly when they do have something in common.