SolutionBase: E-mail sender authentication: Is it the solution to spam?

It's getting to the point where spam is crippling e-mail, making it almost useless. Here's how sender authentication technology using Sender Policy Framework and Microsoft's Sender ID can help get rid of unwanted e-mail.

Life is such that soon after someone comes up with a good
thing, someone else comes up with a way to ruin it, and e-mail is no exception.
Spam or UCE (unwanted commercial e-mail) has proliferated out of control.
Literally thousands of spam messages flood into the typical company's mail
server; luckily, good spam filtering software can prevent most of them from
finding their way to users' mailboxes. Laws have been passed with the goal of
curbing the spam tide, but they are difficult to enforce, in part because
spammers often disguise the real source of their messages by forging return
addresses and e-mail headers.

An even more dangerous development on the e-mail front is "phishing."
Unlike traditional spam that merely attempts to sell a product (albeit through
annoying and sometimes offensive means), phishing messages are fraudulent,
purporting to come from the recipient's bank, credit card company or a company
with whom he does business and attempting to entice the recipient into
revealing confidential information such as passwords, account numbers, or
social security numbers. The phisher then uses that information for identity
theft and other criminal purposes.

Because of the growing amount of "forged" e-mail
from spammers and phishers, a mechanism for authenticating the senders of
messages is growing more desirable. Toward that end, technologies have been
developed to provide a way to verify senders' identities. In this article, we
take a look at the Sender Policy Framework (SPF), a platform-independent sender
authentication technology, and we also discuss Microsoft's Sender ID, which is
based on SPF (and which has been declared dead in the water by more than one
industry pundit).ï¿?? Then we look at where
cryptographic solutions (digital signing) fit into the picture.

The trouble with e-mail

Despite its usefulness and popularity, e-mail poses a couple
of big problems. The system is based on the Simple Mail Transport Protocol
(SMTP), which was designed with speedy delivery, rather than security, as its
prime objective. In that way, e-mail is modeled on the "snail mail"
system. Anyone can drop a letter in a physical mailbox with no return address
or false return address information. Likewise, e-mail can be sent with an
inaccurate or "spoofed" return address.

A major "loophole" in SMTP is its lack of
authentication; it is, after all, the Simple
Mail Transport Protocol, and at the time it was created, security was much
less a concern than it is today and spam and phishing were nonexistent. Thus,
there's no mechanism for detecting spoofed header information.

A three-pronged solution

It's important to note that authentication will not, by
itself, stop spam. The best it can do is help stop the falsification of sender
information. Spammers can use authentication mechanisms, too. Just because the
sender is authenticated, that does not mean
the message is one that you want. Spammers can register domains and publish SPF
records just like anyone else, and in fact, a number of studies have indicated
that spammers are early adopters of the authentication schemes. Thus,
authenticating the sender is only the first step in stopping spam and phishing
messages.

The Aspen Policy
Institute came up with the "Accountable Net" framework (also
referred to as the Aspen framework in Internet circles) in December 2003. This
outlines a community-based approach to verifying identity and fostering
accountability on the Internet, based on three factors:

Authentication

Reputation

Accreditation

Authentication

Most e-mail sender authentication systems rely on those who
own domains to publish the servers or e-mail addresses from which legitimate
mail from that domain can be sent. These lists of legitimate address-domain
correlations are than checked when a message arrives. If the sending address
matches the address that is related to that domain in the list, it is
authenticated. If the mail came from a server that is not listed by the domain owner as a legitimate sender for that
domain, authentication fails.

Note that a big drawback of these IP-based authentication
schemes is that only the domain is validated (not the user). SPF authenticates
using the return-path (envelope sender) in the MAIL FROM field, not the From
field in the header. One way to verify the identity of the individual author of
the message is through digital signatures (cryptographic authentication). The
most effective approach is to combine IP and cryptographic technologies.

Reputation

Reputation refers
to the sender's known history. Once the sender is authenticated, the next step
is to determine if the sender is a known spammer, a known legitimate sender, or
a "mystery sender" (legitimacy unknown). Although a spammer's domain
might be using SPF to authenticate, as it sends spam it will acquire a "bad
rep."

Blacklists are a form of reputation system. The problem with
published blacklists in the past has been the high rate of false positives. For
example:

Addresses were deliberately reported as spammers
when they aren't. What better way to disrupt the life of someone you don't like
than to get his name on the spam blacklists so his mail won't get through to
his friends and business associates?

Entire domains were blocked as spam sources when
only a single user from that ISP sent spam.

Spammers forged the addresses of legitimate
users as their return addresses, and the legitimate users' addresses ended up
on the blacklists.

This last situation, at least, would be helped by widespread
use of sender authentication and blacklists would have more credibility.

Accreditation

Accreditation takes reputation a step further, allowing a
third party (accreditation provider) to vouch for the reputation of senders,
based on sophisticated reputation analysis. Accreditation services may require
that senders pay to be listed and may be backed by a financial bond.

Accreditation creates another element of reputation, the
whitelist, which contains "known good" senders.

SPF: What's it all about?

SPF is the most popular sender authentication technology. It
was developed as an extension to SMTP in 2003 by a group led by Meng Weng Wong,
who founded pobox.com and runs the SPF Web site.
It was originally called Sender Permitted From but later morphed into the Sender
Policy Framework. The concept grew out of two previous sender authentication
proposals: Reverse MX (RMX) and the Designated Mailers Protocol (DMP), and is
based on the original ideas of Jim Miller, Paul Vixie and Hadmut Danisch.

How SPF works

A mail transfer agent (MTA) is the mail server software that
handles delivery of e-mail messages (for example, Exchange or Sendmail). Under
the Sender Policy Framework, owners of e-mail domains authorize specific MTAs
(mail servers) to be the designated senders for their domains. They do this by
publishing the information in the DNS records in TXT format. You need to
publish an SPF record for each domain and subdomain or hostname that has an A
or MX record in DNS. You must use the DNS server that hosts your domain on the
Internet (rather than a local DNS server) to publish the SPF records.

The MAIL FROM command initiates SMTP messages. The
return-path address is part of the MAIL FROM command, which is shown in the
e-mail headers. SPF compares the client IP address with the SPF record for the
domain that's shown in the return-path address to make sure they match.

You can view the Internet headers in most e-mail clients, as
shown in Figure A.

Figure A

SPF checks the domain in the return-path address against the client IP
address

Once the identity of the sender has been authenticated, MTAs
can use reputation and accreditation (DNS blacklists and whitelists, or DNSBLs
and DNSWLs) to decide whether to block or allow the message.

Future plans for SPF include mail user agents (MUAs), which
we normally think of as e-mail clients such as Outlook or Eudora, to be able to
recognize messages that have passed the tests of authentication and
reputation/accreditation and mark them or route them to a folder indicating
that they are highly likely to be legitimate mail (the opposite of the "junk
mail" folder where e-mail clients currently put messages that are likely
to be spam).

Drawbacks of SPF

Once widely adopted, SPF would operate on a "guilty
until proven innocent" philosophy in regard to e-mail. The default would
be to consider messages that are not authenticated or don't pass the
reputation/accreditation test to be spam. The danger in this would be the same
as current overly aggressive spam content filters: the likelihood of false
positives. Receiving mail you don't want is annoying and can even be costly in
terms of time and lost productivity. But not
receiving a critical message could cost much more.

Keep in mind that the information in DNS is available to the
public. You might have internal servers that need to send mail from your
domain, but you don't want those machines' addresses published in the public
records. In that case, the simplest solution is put the servers' addresses in a
DNSWL. This list is not available to the public, just to your SMTP server.

Unfortunately, SPF can cause problems with forwarding. To
use SPF, the forwarding MTA has to rewrite the sender address. This can be
fixed by using the Sender Rewriting Scheme (SRS).

Sender ID: The Microsoft solution

In early 2004, Microsoft unveiled an anti-spam initiative
based on a technology they called Caller ID for E-mail and implemented a pilot
program for Hotmail. Then in the summer of 2004, Microsoft merged its product
with SPF to form a new authentication specification called Sender ID.

Sender ID is based on SPF but there are some differences. To
use Sender ID, domain owners would publish the IP addresses of their mail
servers in DNS using XML files rather than the TXT format used by SPF (however,
Sender ID is backwardly compatible with already-published TXT files used by
SPF). Sender ID originally used the Purported Responsible Address (PRA) instead
of the MAIL FROM return-path address. The PRA is extracted from RFC 2822
headers; in other words, it's the address that users see as the sender in their
e-mail client software. The return-path address and PRA address are the same on
many messages, but they can be different.

A later submission of the Sender ID specifications to the
IETF allows using both the return-path address and the PRA could be used. If
the return-path address is checked, it's called SPF classic mode, and if the
PRA is checked, it's called PRA mode.

Microsoft applied for a patent on the PRA checking portion
of the technology, and this resulted in much controversy and a number of
organizations that had formerly endorsed Sender ID dropping their support.

From the perspective of domain owners, if messages with
different return-path and PRA messages need to be sent from their domains, they
can create PRA (SPF2.0) records in
addition to the SPF (SPF1). Most domain owners don't need to worry about this,
because Sender ID checks the SPF1 records for both the return-path and PRA
addresses by default.

Cryptographic solutions

A different approach to authenticating senders of messages
is to implement digital signatures on the message content. Most of these
methods are dependent on a Public Key Infrastructure (PKI) to issue
public/private key pairs. The signing is done by the MTA and the senders
publish their public keys in DNS.

Digital signature technologies

There are a number of different technologies that can be
used to digitally sign messages:

PGP (Pretty Good Privacy) can be used to sign
the body of the message. Keys can be self-signed.

S/MIME (Secure Multi-purpose Internet Mail
Extension) can be used to sign the message body. Keys are signed by a
certification authority (CA), a trusted third party.

DomainKeys (Yahoo) can be used to sign the
message body and headers.

Advantages and disadvantages of
cryptographic solutions

Cryptographic solutions have an advantage over IP-based
sender authentication methods in that they are able to handle forwarding. The
IP-based methods will incorrectly flag a forwarded message as a forged one (unless
the forwarder rewrites the return-path address).

On the other hand, because most cryptographic solutions sign
the message body, messages may be incorrectly identified as forgeries if the
content of the message changes after the signature is applied. This creates a
problem with mailing lists that may change the message content in transit.

The final solution

There has been a great deal of skepticism and criticism of
SPF/Sender ID within the industry. Many pundits question the value of IP-based
authentication of sender domains in stopping spam. But these technologies were
never intended to be "spam busters" in and of themselves.

Wong and others involved in developing sender authentication
solutions suggest that a combination of IP-based and cryptographic methods is
the best way to verify the identities of e-mail senders, and they stress that
authentication is only the first step in controlling spam; it must be combined
with other mechanisms such as reputation and accreditation to be effective.

It's likely that spam—like the junk brochures in our snail
mailboxes—will always be with us to some degree, but thwarting spammers'
efforts to disguise their identities through sender authentication and
development of accurate, reliable blacklists and whitelists would go a long way
toward making the problem more manageable.

About Deb Shinder

Debra Littlejohn Shinder, MCSE, MVP is a technology consultant, trainer, and writer who has authored a number of books on computer operating systems, networking, and security. Deb is a tech editor, developmental editor, and contributor to over 20 add...

Full Bio

Debra Littlejohn Shinder, MCSE, MVP is a technology consultant, trainer, and writer who has authored a number of books on computer operating systems, networking, and security. Deb is a tech editor, developmental editor, and contributor to over 20 additional books on subjects such as the Windows 2000 and Windows 2003 MCSE exams, CompTIA Security+ exam, and TruSecure's ICSA certification.