SPF, MTAs and SRS

Last month, we learned how to flag outgoing e-mail as authentic using DNS. Now, it's time to check incoming mail and protect our users from forged spam and worms.

Qmail

Qmail does not have the same kind of plugin interface that
the other MTAs do. Instead, SPF provides a patch that
integrates SPF directly into Qmail. In addition,
many Qmail users screen their mail with qpsmtpd: if you
do, SPF is a plugin you can turn on easily.

James Couzens is the primary author of the C SPF library.
libspf comes with a patch for Qmail and for other MTAs as
well.

Testing the Plugin

Once you've installed the plugin and turned it on, you
should perform two tests.
First and most important, legitimate mail needs to get
through. If something broke, maybe you're not running
something you need to—double-check. If it's still broken,
back out the patch and report your experience to the
spf-help mailing list.

Second, confirm that forged mail is rejected. If
you can speak SMTP by hand, engineer a
message with MAIL FROM:<linuxjournal-test@altavista.com>.
The domain altavista.com is not used for mail, so it always
returns a FAIL message. They have asked that test messages contain
the word test. This can be tricky to execute because
if they recognize a trusted
client, both your MTA and SPF will turn a fail into a pass. Therefore, don't telnet to
localhost; use your machine's actual hostname, and if
possible try to open the connection from an outside host.
If you receive a 550 response and an error message that refers
to spf.pobox.com/why.html, it's working.

If you use a secondary MX, tell your SPF
client not to reject its mail. How to do this is described in
detail in the installation instructions for your plugin.

Received-SPF: What the Codes Mean

You should notice that your mail now contains a Received-SPF
header that carries a number of result codes:

NONE: the domain does not publish SPF records. Your
MTA should proceed as usual.

PASS: the mail is not forged, but that doesn't
mean it's legitimate. Remember, spammers can publish
SPF too. You still should test its domain against
a right-hand-side block list (RHSBL). But if the
sender is on your trusted whitelist, you can skip
further antispam checks with confidence.

FAIL: the mail is a forgery, and you can reject
it with confidence. There is a miniscule chance the
message is legitimate but was sent by a misconfigured
sender. In that case, the error message they receive
tells them they need to configure their MUA with SMTP
AUTH. SPF's design philosophy is that it's better to
fail obviously with a hearty error message than to
risk silently burying mail in a spambox.

SOFTFAIL: the message could be a forgery, but the
domain's ISP is working on switching its users to SMTP
AUTH, so the message could be legitimate. You should accept
the message, but subject it to more stringent antispam
checks.

NEUTRAL: the domain just has started down the road to
SPF, and their default response is ?all. They would like you
to pretend the response was NONE while they consider
moving the default toward SOFTFAIL and
FAIL. Big ISPs
with millions of users move slowly; it's not their fault.

ERROR: there was a temporary DNS lookup error.
Normally, your MTA should return a 450 temporary
failure
when this happens.

UNKNOWN: a permanent error caused the
SPF lookup to abort; perhaps there was a syntax error in the
record, or maybe the record pointed to another domain that
doesn't have an SPF record.

The Price of SPF

In the past ten years we have grown tremendously dependent
on e-mail; we are made aware of just how dependent we are
every time a worm hits. Analysts routinely announce that
spam and viruses cost the economy billions of dollars. The
success of SPF shows that people are desperate for change.

But, change has its own price. If there were such a thing as a
painless solution to spam, we already would have adopted it.
The war on spam has dragged on so long in part because the
best experts on spam simply could not agree on exactly what
trade-offs they wanted to make, but that phase of debate is
drawing to a close. In every antispam future they have
discussed, sender authentication is the first and
fundamental step. Now, many possible sender
authentication models are available, but the designated-sender scheme
that SPF provides is probably the easiest to implement.

Cryptography definitely is in our future, but it's not
here yet. Like first aid, SPF offers immediate benefit, and
it's something we can do right away.

What is the price of SPF? Every designated sender scheme
breaks two things.
First, SPF breaks verbatim e-mail forwarding (Figure 1). Services that
provide permanent e-mail addresses, such as pobox.com,
are used to forward mail the way UNIX .forward and
/etc/aliases files do. When the mail leaves their
servers, the return-path address in the envelope is
unchanged. But in an SPF world, resent messages now look a
lot like forgeries. To fix this, forwarding services need
to rewrite their return paths. So do other sites that
depend on .forward and /etc/aliases to send mail off-site.

Figure 1. Old-school e-mail forwarding breaks under SPF.

The solution is called SRS, sender rewriting scheme. It
encapsulates the original sender address in the rewritten,
SPF-compliant, return address. If a message should bounce,
it comes back to you, and you unwrap the address and forward
the bounce back to the sender. Forwarding services would
have to do this even in a world without SPF, because
ISPs already are performing pseudo-SPF checks. SPF simply
gave everyone a standard way to do what they already were doing
piecemeal. In the same way that
responsible sites closed down their open relays over the past few years,
in the
coming months responsible sites will begin to operate
SRS-compliant forwarding; pobox.com already is doing SRS, and
other forwarding services are expected to follow.

The good news is the community that developed SPF
already has produced SRS code for your MTA. Those patches are
available from the same place you got your SPF patches. By
the time you read this, they even might be bundled into your
MTA. The goal is for the average installation to be able to
upgrade to the latest version and have SRS magically
work (see Resources).

So, this solves the e-mail forwarding problem. Getting
SRS into the field is simply a matter of time.
But SPF also breaks Web-generated e-mail. Greeting card
sites and “e-mail me this news article” sites tend to use
your e-mail address not only in the From: header but in the
envelope sender too. In SPF terms, that kind of behaviour
is indistinguishable from forgery.

To solve this problem, those sites can do one
of two things.
First, if the mail they send isn't that important, they
can set the return-path address to nobody@example.com
and eat the bounces. Newer, more progressive sites, such as
Orkut, already do something like this.
But if the mail is important, was sent on behalf of a
user who was logged in to the Web site properly, and if the
Web site had previously confirmed the user's e-mail address,
then the Web site could perform SRS on itself—encapsulating the user's return address so that bounces
would be properly forwarded.

What about the transition period, you ask. Won't there
be a time of disruption while the forwarders groan their way
toward SRS-compliance? What about the sites that are
unwilling or slow to adapt?

Well, here's a little secret. We have a fairly good idea
who the major culprits are; we know, for instance, that eBay
sometimes sends mail with a legitimately forged envelope
return-path. The people who developed SPF use eBay, too,
and they don't want to lose e-mail any more than you do.
So they came up with a hack. They set up a whitelist that
identifies all these legitimate forgers; pobox.com is on
the list, as are acm.org, eBay and the
newspaper Web sites that do “e-mail me this article”.

Every SPF client we've talked about in this article knows
about that whitelist. Every SPF client we know of gives that whitelist a chance to override a fail. If your
mother sends mail from her AOL account to your acm.org
address, your SPF client accepts that message, even
though it's technically a forgery. (If you get forwarded
mail through a system that's not on the list—from, say,
a friend's home Linux box—you should whitelist that
box in your MTA.) When acm.org implements SRS, the problem
will go away.

SPF's critics tend to say “it breaks forwarding”. The SPF
community rose to the occasion and did their best to ease
the transition. They offered two solutions, one short-term
and one long-term, that meet in the middle. Together they
sugarcoat the bitter pill.

Change means pain. The transition to an SPF world won't be
painless, but it's like the pain of an injection that makes
the illness go away. E-mail is very sick. Some say it will
not survive spam, but I don't agree. I think SPF will set it
firmly on the road to recovery.

Meng Weng Wong is founder and CTO of pobox.com, the e-mail forwarding
company, which celebrates its tenth anniversary this
year. He is working on a science-fiction novel set
on a planet where traditional fantasy magic turns out
to be implemented, following Clarke's famous dictum,
using nanotechnology.

Comment viewing options

I receive e-mails from things I subscribe to. And some things I don't. Today I received something from my credit card company. It was valid, had links to go to. But I always look at "view source" to see if it's from someone else who does tent to "play games" with me.
Here is what I saw on "view source" that is confusing me. And why does it have to be misconfigured? Here is it:

Trending Topics

Upcoming Webinar

Getting Started with DevOps - Including New Data on IT Performance from Puppet Labs 2015 State of DevOps Report

August 27, 2015
12:00 PM CDT

DevOps represents a profound change from the way most IT departments have traditionally worked: from siloed teams and high-anxiety releases to everyone collaborating on uneventful and more frequent releases of higher-quality code. It doesn't matter how large or small an organization is, or even whether it's historically slow moving or risk averse — there are ways to adopt DevOps sanely, and get measurable results in just weeks.