Email isn’t decentralized, and that’s fine

Email is the most accessible communication channel where no single entity has full control over whether messages get delivered or not. You could buy mydomain.com, set up a basic email server on your own computer, and send messages from [email protected] to anyone without relying on any for-profit software or services. Both fortunately and unfortunately, however, it’s incredibly unlikely that messages sent from a home server would ever be read by anyone.

Because SMTP is so open and accommodating that it does relatively little to stop spam, companies have stepped in to “save us from ourselves.” They decide what they think should and shouldn’t be delivered. SMTP may be decentralized on paper but out in the real world it’s dominated by vigilante justice. That sounds…bad.

The groups that police email deliverability mostly act in good faith to prevent unwanted and inappropriate messages from landing in our inboxes, though. If these cabals and cartels didn’t exist we would have to deal with exponentially more spam and unwanted messages than we do today. I just wish there was a broader selection of gatekeepers and influence over how they filter and evaluate my emails.

A few blocklists reject anything from huge swaths of the internet

Before a recipient’s email server will even scan an incoming message, it checks the sender’s IP and domain against at least one blocklist. In most cases that will be SpamHaus, which claims to protect 4.5 billion mailboxes. That includes all Microsoft email products, Yahoo Mail, Comcast, supposedly iCloud, and countless other email providers who all decide whether to accept or reject incoming mail based on SpamHaus’s lists. And it is not hard to end up on one of those lists.

When one user on Reddit described trying to set up an email server on a virtual private server and getting blocked, several of the comments pointed out that his household-name hosting provider was probably the issue. “Digital Ocean is the no no land of mailservers.” Another user wrote that “Digital Ocean is the number one abused IP that hits either of my instances.” And that isn’t the only one. Amazon Web Services IPs frequently end up on SpamHaus lists without ever doing anything wrong.

There are hundreds of blocklist providers. But according to email infrastructure engineer Braedon Holt, “Spamhaus SBL is the one that actually destroys your deliverability. If you're listed on obscure blacklists that nobody uses, don't panic — focus on Spamhaus, Barracuda, and SpamCop. Those are the ones ISPs check.”

In one article detailing the blocklist appeal process, SpamHaus’s support explained that a domain owner’s email server was “in an internet neighbourhood with ‘poor reputation’ that has shared (or inevitably will share) its negative reputation with your domain.” This means that even if you do your research and avoid hosting providers known to cause issues, you can still end up picking one that SpamHaus later decides is untrustworthy for reasons totally unrelated to your own sending behavior. We hate this just as much as you do. But at least you see an error when your email gets blocked! That’s better than a social media algorithm quietly burying your content without telling you.

Until there’s a more decentralized, less aggressive version of SpamHaus, play it safe by:

• Sticking with well-reviewed hosting providers built specifically for email.
• Periodically using tools like mxtoolbox to check if you’re on any blocklists.
• Only sending bulk emails (newsletter, marketing, or transactional) on a domain or subdomain specifically for email.
• Never cold emailing lists of people who didn’t sign up to receive your emails.

Then, start thinking about how you can build your reputation with the services that evaluate the content of your emails rather than just the source.

Most people use one of a few mailbox providers

The companies with the most control over email deliverability are the ones that hand out addresses on their domain, especially when they’re “free.” Gmail, Yahoo, and Outlook alone reportedly represent more than a third of inboxes and enforce behaviors and limits that go beyond what’s included in the SMTP standard.

While the group responsible for drafting email recommendations, the Internet Engineering Task Force, doesn’t technically require setting up SPF, DKIM, or DMARC to send email, for example, Gmail rejects all incoming messages without the first two measures enabled. On top of that, messages going to 5,000+ recipients won’t show up in any Gmail inboxes if they don’t have DMARC set up, along with one-click unsubscribe, PTR records, and spam rates below 0.30%. These are all very good requirements! Nonetheless, they were arrived at and are enforced by one of the largest and most powerful for-profit advertising companies on the web.

Even when you set everything up the way Google wants you to, it might still throttle or withhold your emails. When one software engineer left Gmail to set up his own domain, he found “You’ll probably find your sent emails ‘greylisted’ by your recipients’ email providers if your domain has been freshly registered...emails sent from my brand new domain were getting delayed by a few hours when sent to Gmail addresses.” How you avoid this is not always well documented or explained.

To send from a newly registered domain, you’re forced to keep the number of recipients low until you “warm up” your reputation enough to curry favor with Gmail. “Start with a low sending volume to engaged users, and slowly increase the volume over time,” is about as specific as Google’s documentation gets. Send too much too quickly and your messages get rejected outright. And that’s before even beginning to worry about the spam folder.

Your recipients’ spam filters are partly based on their email reading habits, partly on how their inbox provider thinks a well-behaving email should appear. Send something that you think is funny or sarcastic that Google reads as earnest and you’ll land in the Spam folder. Send one book launch email and potentially get moved to the Promotions tab forever. That wouldn’t be criminally unfair if weren’t for how dominant Gmail is. Read the Google tea leaves incorrectly and suddenly a quarter of your list now has to jump through extra hoops to read it.

Gmail et al are constantly changing the rules and deliverability is not an exact science. For now, the best you can do is:

• Slowly ramp up your sending frequency and volume.
• Limit the use of images and HTML.
• If you’re sending to a list, require double opt-in and avoid sending attachments at all costs.
• Keep your email under 5,000 words and 102kb to avoid Gmail truncating your content.
• Don’t use links from URL shorteners like bit.ly and tinyurl.com.

These are things that help keep you in the good graces of both inbox providers and blocklists, the entities that decide whether to let your emails *in. *If you’re sending in bulk (newsletters, transactional email, etc.), there’s an entirely separate group policing whether or not to let your emails out.

There are few options for sending one-to-many emails

You can technically send a single email to up to 500 people per day in Gmail. The reality, however, is that your email will almost certainly be throttled and likely sent to spam. For more reliable inbox placement at scale, you need specialized sending infrastructure.

First, there are backend services where you make API calls to send emails en masse, like Postmark, Sendgrid, and Mailgun. Then there are frontend services, where you build your emails and lists in more user-friendly interfaces, like Buttondown, Mailchimp, and Beehiiv. Unlike blocklists and inbox providers, you actually get to choose your sending infrastructure. And there’s a fair amount of competition! Still, acting as a proxy for thousands of accounts that each send thousands of emails means there are only so many players in the space.

Sending providers try to absorb as much of the deliverability burden as possible. You usually don’t have to worry about “warming up” your domain. And these platforms are generally less concerned with policing marketing content, leaving that filtering to recipients’ mail servers. Don’t send emails pitching illegal goods or services, gambling, financial schemes, or pharmaceuticals and you should be good. There are some grey areas, though.

Adult content isn’t always a black-and-white categorization, for example. Postmark and Sendgrid explicitly prohibit pornography while Mailgun only bans the worst of the worst adult content. Weirdly, the opposite is true of hate speech, with Mailgun calling it out by name in their prohibited content. Postmark and Sendgrid (and some frontend sending services) only allude to it with language that is more vague than their rules on emails containing multi-level marketing schemes. These are things you should know about your ESP.

What’s more, when you send something subscribers don’t like, even if it’s not technically spam, they’ll sometimes mark it as such anyway. And what happens when recipients click the spam button is an automated email to your sending provider that ticks your complaint rate up relative to the number of people who didn’t junk your message. When a complaint rate reaches a certain threshold, that account might be put under review or bounced off the platform. And there are only so many options to migrate to. There are a few things you can do to build rapport with your ESP:

• Avoid a flurry of activity when you first sign up. That includes things like importing huge lists right away, testing your setup with frequent subscribes and unsubscribes, and receiving complaints or low engagement on your first send.
• Use a purpose-build app like Buttondown or one of its competitors for email newsletters.
• Send emails from a domain you own.
• Make sure messages follow accessibility design best practices.

I’d love to self-host an email server, to use my own hardware and configurations to send messages without as few middlemen as possible. But, for now at least, keeping unwanted spam and harmful content out of my inbox is more important. And for that, email is still king.

Deliverability, an externality and a backstop

There is a for-profit arm of SpamHaus that sells its (usually free) data with enterprise add-ons. Gmail shows ads to its free users and charges Workspace users for ad-free email. Sending providers charge based on the size of their users’ lists. So, as much as it hurts to admit, deliverability gatekeepers aren’t completely out of alignment with the average user. If they screw things up too badly, if they overreach, people stop sending, email usage dries up, and everyone loses. Let’s hold ‘em to it.

“It is a well-established principle that an SMTP server may refuse to accept mail for any operational or technical reason that makes sense to the site providing the server,” Internet Hall of Famer John Klensin wrote in an SMTP revision from 2001. “However, cooperation among sites and installations makes the Internet possible.” And for that, I am grateful.