MTA, SPF, DKIM, PTR, WTF: a quick checklist on how to send e-mail from your domain

Table of Contents

Introduction

While I'm writer's-blocked on continuing with project Betfair (well, more like blocked on not being able to reproduce my old research results in order to fit the narrative I thought I had), here's a quick story from the development of Kimonote.

I got stuck following this guide on how to startup:

How to startup:

  1. Set up a landing page to collect e-mail addresses
  2. ??????
  3. ??????

and had figured out that the next step after collecting e-mail addresses was sending something to them. Except the issue was that I also wanted to make sure most e-mail providers didn't mark my mail as spam and it actually got delivered.

So here's a quick checklist on what to do in order to achieve that. This guide assumes that you have a dedicated IP address and a domain name whose records you can edit.

Checklist

Mail Transfer Agent: exim

Or Postfix. This is the server that will run on your machine and, when a client requests it to send mail, will do so.

The following commands are all executed as root, assuming a Debian-like system (say, Ubuntu). Install exim:

apt-get install exim4

I used the "Single configuration file" option during the installation: the only change I needed to make to it was disabling IPv6. To do that, add disable_ipv6=true under "Main configuration settings" in /etc/exim4/exim4.conf.template and reload the config with /etc/init.d/exim4 reload.

Try sending a test message:

echo "testing" | mail -s Testing (your personal email address)

You should get a message in your spam folder from root@(hostname, which is probably not your domain name).

Try again, this time setting the from address:

echo "testing" | mail -s Testing -aFrom:test@(your domain name) (your personal email address)

You should get an email from test@(domain name), still in your spam folder (if you're lucky). If you don't, check the exim logs at /var/log/exim4/mainlog.

SPF record

An SPF record is the first step on the route towards not getting your emails sent to the spam folder. In the world of email, anyone can pretend to be anything. I could right now use the exim instance running on my server to send an email from support@microsoft.com to (my best friend)@gmail.com claiming to be Microsoft. The only thing that's stopping me is that the recipient's email service will perform a DNS query to check the microsoft.com's SPF records in order to see who's authorised to send emails on behalf of microsoft.com, thus rejecting my email.

The simplest SPF record is:

v=spf1 a ~all

This record is read left-to-right and says: "If the IP you're getting mail from is in my A record, then it's OK, otherwise, reject the email".

Add that as a TXT record in your domain management panel with Host=@. In the case of Namecheap with e-mail forwarding on, I had to instead edit the TXT record under Mail Settings and add a between v=spf1 and include:spf.efwd.registrar-servers.com ~all (the latter says "any IPs that are in the SPF record of efwd.registrar-servers.com" are fine too)

DKIM record

If the SPF record says that the email is indeed from your domain, then the DKIM record verifies that the email wasn't altered in transit. In essence, you publish a public key to another DNS record and then sign your messages with the private key. Strictly speaking, this wasn't required for Gmail to stop classifying my emails to myself as spam, but it's worth doing anyway.

First, generate a key pair:

cd /etc/exim4 && mkdir dkim && cd dkim
openssl genrsa -out dkim-private.pem 1024 -outform PEM
openssl rsa -in dkim-private.pem -out dkim.pem -pubout -outform PEM

Go to your domain control panel and add a TXT record with host (selector)._domainkey (the selector could be anything: I used a timestamp 20171206) and value

k=rsa; p=(your public key from dkim.pem, all in one line)

Now, set up exim to actually sign outgoing emails with the private key. If you are using the single Exim configuration file option, create (or edit) the file /etc/exim4/exim4.conf.localmacros and add:

DKIM_CANON = relaxed
DKIM_SELECTOR = (selector, e.g. 20171206)
DKIM_DOMAIN = (domain name, like example.com)
DKIM_PRIVATE_KEY = /etc/exim4/dkim/dkim-private.pem

Reload the config:

/etc/init.d/exim4 reload

And check that exim picked up the settings:

exim -bP transports | grep dkim

dkim_canon = relaxed
dkim_domain = example.com
dkim_private_key = /etc/exim4/dkim/dkim-private.pem
dkim_selector = (selector, e.g. 20171206)

You should wait for both records to propagate around. To test whether this has worked, you can use any of the online services that do some DNS lookups and tell you whether they have seen your records (e.g. https://www.mail-tester.com/spf-dkim-check). You can also send an email to your personal address. If it arrives (or arrives into your spam folder), you can inspect its headers:

Received-SPF: pass (google.com: domain of hello@kimonote.com designates 91.220.127.149 as permitted sender) client-ip=91.220.127.149;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@kimonote.com header.s=20171206 header.b=K0nBKUQ7;
       spf=pass (google.com: domain of hello@kimonote.com designates 91.220.127.149 as permitted sender) smtp.mailfrom=hello@kimonote.com

PTR record

Are we done? Not completely. Some ISPs do what's called a reverse DNS lookup on the IP that's sending them emails by contacting the (via several levels of referrals) DNS servers of the organization that IP belongs to in order to find out what domain it corresponds to. Then they compare that to the domain the email is claimed to be from. This is similar to having an SPF record and I thought it wasn't necessary until I had some emails from users saying that they weren't receiving their registration confirmations.

First, check that it's actually the case:

 dig -x (your IP address)

If the Answer section doesn't contain your domain name (yes, it is supposed to end with a dot), then you do need to add a PTR record. The PTR record has to be added on your hosting service's side: there usually is a control panel (in my case I had to ask them nicely). This will probably take up to 24 hours to propagate around the world.