I think that's because you can usually smell when food is close to being rotten before you can see it...

EDIT: reading the WardsWiki reference from that Wikipedia page, there's also the point made by early users of the term that smells are something you have to check out, but don't always mean something needs fixing - e.g. a bad smell may be a gas leak, or it may just be a rubbish bin.