Yes... though I think Chesterton's fence definitely belongs in the "technically correct advice that actually does more harm than good" bucket, like "premature optimisation", "if it works don't fix it", the Unix philosophy and so on.

This doesn't apply to capitalisation, but generally especially in computing if there's something that looks useless you should remove it. If it breaks the fault lies with whomever left something useless there without a note to explain it.

The current project I'm working on has about 3 copies of every component because nobody bothers to clear up after themselves - dead code isn't doing any harm and it's better to leave it in case it's needed right?

Well sure, if you want me to work about 3x slower than I otherwise could. Not an exaggeration.