I'm surprised to have read to the end and found that they're still not performing any hardware monitoring and alerting. SMART may not always show up pre-failure warnings but when it does they can usually be trusted.
Wasn't it a conclusion of the Google hard drive reliability study that models based on SMART were not useful? I.e. drives with sector reallocations are much more likely to fail than those without, but their failure rate is still something like 15% per year, so what useful thing can you do with that signal?
Hard Disk Sentinel is really good for this type of thing. The developer is awesome and some years ago after I asked for some new features added code to better support my RAID adapter.