alt Hacker NewsI run an email sending service at scale (billions of messages per month, tens of millions of end users, thousands of customers). Most of our software development and operational effort revolves around abuse mitigation. That has been the case for 15 years. It's a cat-and-mouse game with two different mice: the senders, who are constantly trying to figure out how to get you to deliver their garbage; and the receivers, who are constantly trying to figure out how to block it. We're stuck in the middle.
It's hard to appreciate how difficult this battle is when running at scale.
Replies
Right, I won't disagree with any of that, but I'm not sure how it's related to what I wrote either. Maybe I should have been more specific that I'm talking about hosting your own email, not hosting emails for others, which brings out a lot of other types of problems.
What structural changes could we make to improve the situation?
> I run an email sending service at scale (billions of messages per month, tens of millions of end users, thousands of customers).
Giving you the benefit of the doubt and accepting your claim, doesn't that make you one of the people at least second-order responsible for the current state of affairs in email blocking? It would seem that your company, by dint of your volume, navigates roadblocks that the rest of us (ie. the 99.999% of Internet email servers and their admins), who aren't FAANG et al[1], have to deal with to get our users' legitimate email delivered.
If so, could you perhaps give us a brief explanation as to why an otherwise competent engineer can "follow all the best practices" with their server which has no known compromises[2], on an IP address they have controlled for, oh, let's say a full calendar year, and yet still can't get off those FAANG et al default-deny blocklists, but you can?[3]
A cynic might say that your service had a vested interest in paying for unimpeded access to those FAANG et al companies to get over the bar that the rest of us are unable to vault. A cynic might also say that those biggest of the big email services like it that way, because it drives more users to them at the expense of the rest of us 99.999%.
I'll try to remain open to the possibility that there are aspects of the industry I've not yet had any exposure to, and refrain from chimping out over having my users blocked through no fault of their own.
[1] Yes, I know, Facebook doesn't receive anywhere near as much email as they send, and Hotmail = Microsoft, etc. If I used an accurate acronym I could pat myself on the back for being Technically Correct, while nobody would know what the heck I was talking about.
[2] We shan't digress into a discussion of hardware/firmware/OS/application backdoors nor Snowden disclosures. It's not that hard to auto-install security updates and run a reasonably tight ship with no unnecessary attack surfaces.
[3] Or perhaps there aren't any default-deny blocklists at all, but in fact only much smaller default-allow whitelists? That would be cynical indeed.
> billions of messages per month
Just tangential, but maybe that is part of the problem, isn't it? Napkin math tells me that esch person in the world receives every month an e-mail from you, and you're obviously just one of hundres of providers, and only half of the population actially has e-mail... I think you get the point.
E-Mail got to the point where it's actually worse then physical mail to some degree. Physical mail at least has a hurdle for the sender, and it's easier to throw away without even looking at it. The amount of low-quality mail and annoying, unnecessary notifications I receive is just at a level where I really think of dropping e-mail except for absolute becessary services.