Companies sensible to effective delivery of email to all free email services may have noticed problems with deliveries to Hotmail addresses. Despite the SMTP dialog ending with a successful "250" return code, recipients don't see the message. In their Guidelines, MSN require thorough compliance with IETF standards. However, it seems they have their own interpretation about provisions for Delivery Status Notifications, a.k.a. bounces, that servers must send after they have accepted responsibility for delivering the message. According to the Windows Live Hotmail Fact Sheet: "...messages delivered to the junk mail folder will not automatically show or render message content."

Dropping messages without notice is cool if the server is fairly certain about the nature of the message. Most servers drop viruses that way, as it's the only way to effectively stop their propagation. For spam, the best practice is to block messages at an early stage in the SMTP dialog by responding with a permanent failure code. The connecting MTA, normally the sender's outgoing SMTP server, should then issue a bounce. DNS Black Lists (DNSBL) are an ISP's best friends when it comes to discriminate connecting clients. Let me remark that the "DNS" acronym here refers just to the lookup protocol being used. DNSBLs' responses are based on no authoritative delegation.

Hotmail doesn't say what DNSBLs they consult, if any. They are not going to open their filtering technique, out of concern the disclosures would allow spammers to bypass the defenses. It turns out that an ISP may gain the privilege to send mail to their addresses after subscribing to Smart Network Data Services. Microsoft suggests one can use that service, as well as other services, some of which require payments, neither of which guarantees a result. With more than 280 million active accounts (according to the mentioned fact sheet) Microsoft can establish the policy they want. They are not going to own email that way: just a piece of it.

On the opposite, tiny to medium ISPs have a hard time blocking mail from large ISPs' hosts that happen to get listed, e.g., in SpamhausSBL-XBL. Telecom Italia, for one, is a major connection provider in Italy who regularly gets blacklisted for not having an effective anti-spam policy. Users can't stand not receiving messages and don't want to know why. Therefore, small ISPs whitelist the offending IP. Again, the result is that various senders are happy with their email addresses, and don't realize they only work inside a small piece of the Internet.

That tendency toward fragmentation can be turned the other way around.

Perhaps, we've dismissed proposals like SPF somewhat hurriedly. After all, SPF syntax allows to create domain-specific white lists or to endorse existing DNSBLs. We didn't really explore the possibilities that SPF has to offer before dismissing it. Regional clustering is the easiest case because of the way IPs are assigned by IANA. However, I found out that many domain administrators didn't even think that, using CIDR notations as "?ip4:90.0.0.0/7", a whole Region of IPs can fit within a single SPF line. Why would an Italian domain administrator allow sending from any RIPE address? Because it is much less practical for him to prosecute spammers in the US or China. And then the whole Europe is just a piece of the Internet, much less than "~all". Divide et impera. Conquer or be conquered.

Comments

SPF is broken by design, the signing of emails (either by server, or by sender) allows the authenticity to be established, independent of the route an email took to get to you, and thus avoids the key failing of SPF.

I don't see what SPF has to do with the Telecom Italia issue. Telecom Italia sent people spam, and advance fee fraud invitations, millions of them.

If you control where email from your domain comes from SPF lets you say so. If you don't, why would you want your users email rejected, because they happened to be sending it from Africa rather than Europe?

The idea it is easier to prosecute say "in Europe" is laughable. Did Telecom Italia get prosecuted for supporting advanced fee fraud? Thus I conclude that legal remedies won't solve our current email woes, if you can't get big Italian companies to behave legally and decently, what hope if there of controlling shady businesses in parts of the world with less effective legal systems and governments.

Carrying on legal suits is just one of the actions one may engage against spammers in the real (not virtual) world. Complaining and eventually changing ISP may result in a similar effect, when the people are many. Companies may institute a policy of disqualifying ISPs that appear anywhere on Spamhaus' top 10 list from doing any business with them. All these actions capitalize on human discernment and aim at definitively ruling out spammers.

Forcing spammers to operate locally looks like a good strategy, because human discernment emerges more naturally within local or restricted communities. In addition, if Telecom Italia behaves badly it should be up to Italian users to deal with that issue. An American user, for example, should not suffer because of it. DNSBLs implement that possibility. However, their operations are not well known in Italy, where their long term effect should happen. Maybe, the fact that they act unofficially hinders their rise as opinion makers and conveys the idea that messages can be rejected arbitrarily. Language barriers and email clients that hide the real reasons of bounces do the rest.

SPF does not directly help when the source of spam are the ISP's relays. Apart from that, it is misunderstood, rather than broken. SPF does not authenticate the sender. Message signing does that. Signing messages on the server is a trade-off between security and convenience. It is useful because very few users spontaneously take on themselves the burden to set up a mechanism to sign every outgoing message. At any rate, the "From:" header, i.e. the message originator's name and/or id that recipients normally see and reply to, can only be authenticated after the body of the message has been received. Thus, in terms of both what it does and when, message signing is complementary to SPF.

Another common misunderstanding about SPF is that users should change their email address whenever they change connection provider. They don't have to. A user connecting from an AfriNIC IP can apply SMTP authentication on any server she wants. Vanity email addresses can be exploited in the "From:" header, since the effective address, I mean the SPF-authorized address assigned by an email service provider, needs only to appear on the envelope "MAIL from:", a.k.a. Return-Path, of a message.

Email service providers should control where email comes from. In the past, it was believed that using the connection provider's SMTP server for outgoing mail would result in a better resource usage, as that allowed to optimize the delivery paths of outgoing messages. After recognizing the amount of resources burned for delivering spam, that statement is not true.