MAAWG is the Messaging Anti-Abuse Working group. It was started by Openwave, a vendor that sells e-mail hardware and software to large ISPs and originally consisted only of Openwave customers, but has evolved into an active forum in which large ISPs and software vendors exchange notes on anti-spam and other anti-abuse activities. Members now include nearly every large ISP including AOL, Earthlink, Yahoo, Comcast and Verizon is a member, along with ESPs like Doubleclick, Bigfoot, and Checkfree, and vendors like Ciscom, Ironport, Messagelabs, Kelkea/Trend, and Habeas. They've also been quietly active in codifying best practices and working on some small but useful standards like a common abuse reporting format.

Earlier in July their technical committee quietly released an evaluation of SPF and Sender-ID. Although it is worded very tactfully, the message is clear from phrases like;

While MAAWG neither endorses nor discourages the use of SPF or Sender ID, the technical committee's findings highlight real-world risks to the delivery of legitimate e-mail when the specifications are implemented.

At about the same time, Earthlink equally quietly removed the SPF records they'd been publishing for at least a year. That was particularly surprising because SPF originator Meng Wong had been working with Earthlink to get their SPF set up. If Meng can't make SPF work, who can?

I particularly look forward to see what happens in November when Hotmail says they will start showing a yellow warning box (the Big Yellow Box Of Death, or BYBOD to the cognoscenti) on any incoming mail that doesn't pass Sender-ID. With no SPF records at all, Earthlink's mail won't pass Sender-ID, and will, we assume, be 100% BYBOD compatible. Will Hotmail blink and add their own synthetic SPF records for Earthlink? Will Earthlink publish SPF records that only Hotmail can see (and if they do, how could we tell?) Should be interesting.

(Claimer: most of MAAWG's members are companies that pay a substantial membership fee, but they also have a few invited individual members, including me.)

By John Levine, Author, Consultant & Speaker. More blog posts from John Levine can also be read here.

Until DKIM is ready to go (and by that, I mean supported by major MTAs and anti-spam vendors), SPF and Sender-ID are our best bets. The ethic of waiting for a perfect solution has consistantly failed our standard bodies. Imperfect but available is frequently more effective than perfect but theoretical.

The effects on mailing list and e-card senders are known but pale in comparison to the ability to gain authenticated sender identity which can inform reputation services.

First find a good reputation service to back it.
Second, convince all the twits out there who treat spf failures (or even softfails) as an excuse to hard reject email.

SPF works in a limited set of use cases - sure, these could be a thick chunk of the middle of a bell curve for normal email, and a few interesting edge cases where spf is used in ways way outside what it was specced for [AOL saying it'll maintain its whitelist using spf - so that lots of bulk mailers rush to publish spf records].

It has the potential to fail badly in several edge (and not so edge) cases, as the MAAWG whitepaper John linked to correctly points out.

This article seems to be mostly speculation. Well, maybe John knows some inside info about the MAAWG Technical Committee and such, but without publishing those details, this seems to be largely rumors. I have heard some details about Earthlink and Meng, but I haven't heard enough from all sides to go speculating in public.

As far as a reputation service, I guess it all depends on what you want to do with the SPF results and how sensitive you are to errors, etc. There was a nice research paper (with published data!) that shows that many people can simply use their own mail stream to leverage a reputation system that works well with SPF. See: http://www.ceas.cc/papers-2005/176.pdf

I also realize that some people reject email based on things like SoftFail. Their servers, their rules. I hear some people also reject email based on a listing in the BLARSBL.

The actual MAAWG technical report is a nice summary of the SPF/SenderID situation, but there isn't anything new in it. The problems with SPF and forwarding/roaming users were well understood 2 years ago. The same problems were understood for the SenderID/PRA over a year ago.

Actually, the technical report has a bunch of errors in it. For example, the claim that SoftFail is functionally identical to Neutral, the bogus "original SPF", the claim that most records end in ~all, etc. Does anyone know who was on this committee and why they didn't pass it by the SPF community for review first? Or, if this report is based on draft-newton-maawg-spf-considerations, why the corrections that were sent to Andy were never applied?

Nobody's waiting for a perfect solution; we're waiting for a solution that doesn't cause more problems than it solves. SPF and Sender-ID have been grossly oversold. If you already have a list of domains from which you know you want mail and that send all their mail from fixed places (well-run ESPs, mostly), SPF is a fine way to track the IP addresses those domains use. AOL uses SPF for that purpose.

But that's it. It's not an anti-spam system (as Meng agrees if you pin him down), and for mail from anywhere other than an ESP, it fails too often to be useful. That's why people who have tried it out aren't using it any more.

I really wish people would think more about reputation systems, since authentication systems, be it SPF, DKIM, or S/MIME are nearly useless without them.

Confidential to Wayne: yes, we know who was on the committee at MAAWG who wrote the whitepaper. They're writing about their actual experience testing it at large ISPs, at least one of which hired Meng to help them do it, not what the SPF community hopes might have happened.

Many of errors that I saw in the technical report have nothing to do with the testing and such.

I guess some of the claims, such as "Most SPF records today end with ~all." depend on whether you mean most SPF records that have been published, or most email that large ISPs receive that have published SPF records.

I ran a survey of all .com domains right before the Email Auth Summit last month. Checking the records, I find:

(I'm sure there are records with out "all" in them, or with multiple "all" mechanisms. I would be surprised if the numbers added up to the total.)

So, what is the basis for the claim in the report that "Most SPF records today end with ~all"?

And, as far as over hyping things go, I think you should be careful claiming things like "That's why people who have tried it out aren't using it any more." There are still a lot of people using SPF. Actually, the data I have indicates that more and more people are using SPF. Maybe some people know how to use it effectively and some people don't.

Of the 650k domains in .com that I found publishing SPF records with -all, 303k of them are "v=spf1 -all".

Now, as it happens I am trying to do some analysis of the data I've collected. If you would be interested in helping me which domains are owned by cybersquaters, bulk mailers and small domains, please let me know. I would greatly appreciate collecting and publishing data, rather than just speculating.

However, my point was simply that there is stuff in that MAAWG technical report that just doesn't make any sense, Stuff like claiming that the Lentczner I-D didn't require SPF records at the HELO domain level, but the Schlitt I-D does, or that the Lentczner I-D in any way represents the "Original SPF".

If you can't find any use for SPF, fine. Don't use it. The numbers I'm seeing show that there are more people publishing SPF records, and there are more people checking SPF records. Until I see actual numbers that show that SPF is really losing mindshare, I'm not going to worry.

That'd give you a whole lot of clusters - large providers that do spf. And it'd likely give you lots of small sites all over the place that do SPF. In any case it'd be a good starting point for analysis.

Wow. Someone should probably contact those other 347K domains that have published -all with at least 1 IP record, and make sure they understand that if they send mail to any university or ISP that their email mail has a reasonable chance of being forwarded and thus rejected because of their SPF record.

Golding: There are alternatives to SPF that are better and 'ready to go'. Even my hybrid draft from last September is better than today's SPF:
http://elvey.com/draft-ietf-marid-spf3-00.txt
...not that I'm terribly proud of it, but I do know that Google's gmail has implemented it, more or less.

It isn't clear to me all of the differences between your draft and the marid-protocol draft is. One thing appears to be the restoration of SPF HELO checking.

As I think you know, 2821.HELO checking has been a part of SPF since the very first draft, as part of the 2821.MAIL FROM checking when the MAIL FROM is null. In the May 2004 draft of the SPF specification, HELO checking was documented to be OK to do, even when the 2821.MAIL FROM was not null. That is, HELO checking was made an optional independant check.

During the IETF MARID WG, both HELO checking and the MAIL FROM checking were removed when SenderID was created.

wayne:
SPF3 (like CSV) still seems to be orders of magnitude easier to deploy than SPF*. SPF works well (ignoring the forwarding, security, etc. problems for the moment) if every legitimate domain appearing in a MAIL FROM has an appropriate SPF record. This means that of the 100 million or so TLD Names, all that are used in MAIL FROM for legitimate email need SPF records. Contrast this with SPF3 and CSV. They just require records for the domains used in HELO. (I just posted to MARID about this.)
The current SPF draft seems to leave poorly defined how the situation where the HELO passes and the MAIL FROM fails, or vice versa, shoud be handled. That strikes me as a bad thing. Since the MAIL FROM check is required, while the HELO check is just recommended, folks might jump to (I would say inappropriate) conclusions based on that. I don't know what the SPF libraries out there do. The draft doesn't give them any clear guidance AFAICT (not a good thing).

You are right, the relationship between the SPF HELO checking and the SPF MAIL FROM checking is purposely left undefined. The draft-mengwong-spf-01 of May 2004 gave a recommended way of combinining them. When trying to standardize existing practices and previous specs, we removed a lot of Receiver Policy stuff and concentrated on Sender Policy. Different implementations and mail admins have chosen to use the SPF HELO checking in different ways.

For what it is worth, earlier this week I did a survey of 135k domains used in the SMTP HELO command. I found over 3750 domains with SPF records at the HELO level, while I found only 24 CSV records. (The number of records is more than the number of domain owners because there are often several MTA hostnames per main domain name. For example, the 24 CSV records were published by just 5 domain owners. There were probably only 2500-3000 domain owners publishing SPF records, although some of those domains send a huge amount of email.)

I think HELO checking is useful. If you like HELO checking, I highly recommend publish SPF records at your HELO domain level. Systems such as spamassassin have been checking the HELO domain for SPF records for at least a year now. If you want to do HELO checking, I highly recommend checking for the SPF records. They are far more widely published than CSV or DMP records.