Think Like an Email Expert: How to Solve a Sudden Boost in SPF Failure Rate

Posted by Aaron StevensonJanuary 13, 2016

Authenticating legitimate email with open standards such as SPF (Sender Policy Framework), DKIM (DomainKeys Identified Mail), and DMARC (Domain-based Message Authentication Reporting and Conformance) is the best way for companies to combat phishing and spoofing.

Ultimately, however, it is the mailbox providers (Google, Microsoft, Yahoo!, AOL, Comcast, etc.) that have the power to let good email in and keep bad email out of the consumer inbox. They decide (1) whether or not the received message passes or fails authentication and (2) what information they will report back to the sender.

Often, these decisions differ from one mailbox provider to another and the information included in reports can be esoteric, making it difficult for senders to know why legitimate messages fail authentication and what to do about it.

In this blog series, we will examine email authentication issues from real Return Path clients and explore how to diagnose and remedy them. From this analysis, you will discover why it is crucial to work with a partner that can help interpret complex authentication data and suggest changes. Such changes will both ensure consistent email delivery results and maximize DMARC’s domain spoofing prevention capabilities.

The issue: a sudden boost in SPF failure rateOne common authentication problem senders can face is a sudden change in SPF failure rate. Recently, a Return Path client started experiencing up to a 25 percent SPF failure rate for one of their sending domains which previously had a passing rate of more than 99 percent. This boost in failure rate was far higher than could be explained by transient errors.

Seven-day rolling data for SPF pass rates before, during, and after the failure period

There were no obvious explanations for why SPF checks for messages sent from this domain began failing. Emails were sent from IP addresses included in the SPF record both before and during the intermittent failures and no other changes had been made to the DNS record for this domain. The sender was at a loss.

The diagnosis: subdomain TXT records contained within the parent DNS zone file
The first thing our team did was check DMARC reports from each specific mailbox provider. It became apparent that across these different mailbox providers, some emails were passing SPF at the previously high rate and others were failing every SPF check. The failing mailbox providers accounted for approximately 25 percent of the DMARC reports, leading to the 25 percent overall SPF failure rate.

Data from one week during the failure period

The sending domain in question happened to be a subdomain of one of the client’s Top Level Domains (TLDs) and we found that its DNS text records were contained within the parent zone file of the TLD. That is, they had not been delegated to a separate zone file.

The DNS zone file is the file which contains all of the DNS text records and authorized IP addresses for a specific domain. Normally, if a subdomain’s records are contained within a parent zone file, there is no issue with SPF checks. Indeed, most ISPs were able to report a successful SPF pass for this subdomain. But the exception in this case was causing legitimate messages to fail.

The solution: delegating the subdomain’s DNS text records to a separate zone file
After discussing this issue, we suggested that the client’s DNS administrator simply move all of the subdomain’s DNS text records (excluding the Name Server (NS) record, which was unchanged) to a separate, delegated zone file containing the complete details of all records and authorized IP addresses for the subdomain in question.

Once this change was made, the DNS propagation and mailbox provider cache refresh times meant that for approximately 48 hours there were varied SPF results from all mailbox providers. Ultimately, once this period elapsed, the overall pass rate returned to more than 99 percent.

Data from one week after the failure period

The lesson: you have to know where to lookBreaking down the pass rate at each mailbox provider was the key to uncovering how and why messages were failing SPF. Knowing where to look for problems is an essential part of diagnosing them. And often, this knowledge requires partnering with a team like Return Path’s that analyzes the full scope of email authentication patterns and issues regularly.

Coming up, in part two of this series, we’ll explore how mailbox providers can interpret DKIM differently, resulting in similar intermittent failure reports, and what senders can do to get back on track. Stay tuned.

Want more blog posts like these? Subscribe to our blog and stay up to date on all the latest email authentication news and best practices.

About Aaron Stevenson

Aaron Stevenson is a Strategic Project Manager at Return Path. He works closely with our clients to help them diagnose and resolve Email Authentication issues so that they can make full use of the Email Fraud Prevention capabilities of DMARC. Connect with him on Linkedin https://uk.linkedin.com/in/stevensonaaron