Top 10 Blogs: Our Year in Review

We’re finishing out the year with a roundup of our top 10 blogs from 2016. The Mandrill announcement in April impacted our community, and as a result our blog, in a big way. We’re recapping that along with other top posts on deliverability tips and email marketing best practices down below. As always, our ears are open, so if there’s a certain topic you’d like to see on the blog, leave us a comment, tweet us, or ping us in slack.

It’s no surprise that our Mandrill alternative blogs dominated our top 10 list (5 out of our top 10). We responded in real-time to the Mandrill crisis, and our CEO even weighed in and made you a promise he intends to stick by for the long haul. The Mandrill incident also inspired us to create SendGrid and Mailgun migration guides, check them out when you have a chance.

For developers who want to get the most out of SparkPost templating capabilities, this post was meant for you! In this straight-forward post, Chris Wilson makes sending email easy and gives you some pro tips along the way.

One of the most useful elements of SparkPost are our webhooks and in this post, Ewan Dennis walks you through the basics and beyond. Knowing what to expect functionally beyond the raw API spec is half the battle when consuming new data sources like SparkPost webhooks.

The Outlook inbox is one of the major destinations for most email senders, especially those with large numbers of consumer subscribers. It also has a reputation for being somewhat tricky to get into. In this post, one of our deliverability experts, Tonya Gordon, shares what senders need to know in order to get the best Hotmail/Outlook deliverability and ensure their messages reach the inbox.

Thanks to your feedback, the Mandrill event helped us expedite our release of subaccounts ahead of schedule. Our VP of Product told you about how we process your feedback and what’s available with subaccounts.

Sometimes you need to go beyond a top 10 list and in this case we did — 17 tips on how not to be labeled an email rookie. In this post we put together a list of common mistakes, with a heavy dose of snark, on how to avoid being labeled an email marketing rookie.

You know you need to send email, but you don’t want to spend a lot of time or effort on it — you just want something that works out of the box. It’s not too much to ask! Many frameworks, languages, and tools come with SMTP support, but the last step is the most important – an SMTP server. In this post, we walk you through how to set up SparkPost as your SMTP Relay.

And that rounds out our Top 10 Blogs for 2016! Any industry trends or topics you think were under-represented? Leave us a comment below, or tweet us!

-Tracy

A common question for anyone configuring an app or mail system to send (or relay) email is “what SMTP port should I use?” Let’s run through the different ports commonly used for sending email today.

(TL;DR: If you’re configuring your systems to use SparkPost as an SMTP relay, you should use port 587, with 2525 as an alternate in certain circumstances when port 587 is not available.)

Any SMTP Port in a Storm

Email, like every other system that connects over a network like the Internet, uses the concept of addresses to determine where a system can be found. All of us today are familiar with the textual version of these addresses, like www.sparkpost.com. And most of us know that an easy-to-remember text address stands in for a numeric IP address like 107.21.246.67. But not as many of use know that these network addresses also include specific “port numbers” that are a little bit like an apartment number in a real-world street address.

For example, the web and HTTP use port number 80. For email and SMTP that port number is… well, it depends. You might see information that tells you to use ports 25, 465, 587, or 2525 for SMTP. Which SMTP port should you use? Here are the details about the most common SMTP ports.

Port 25

If you’re a systems administrator of a certain age, you know the answer used to be straightforward: SMTP was designated port 25 in IETF Request For Comments (RFC) 821. Today, the Internet Assigned Numbers Authority (IANA), the group responsible for maintaining the Internet addressing scheme, still recognizes Port 25 as the default SMTP port. But in practicality, it’s not as simple as it seems.

Although port 25 continues to be used for server-to-server SMTP relaying, most modern SMTP clients should not be configured to use this port, because Port 25 usually is blocked by residential ISPs and business cloud hosting providers alike. Why? Thank the spammers. Port 25 is blocked on many networks to curb the amount of spam that is relayed from compromised computers or servers. Unless you’re specifically managing a mail server, you should have no traffic traversing this port on your computer or server.

Port 465

IANA initially assigned port 465 for an encrypted version of SMTP called SMTPS. However, IANA since has reassigned this port for a different use, so it should no longer be used for SMTP.

However, because of this brief historical use, there are some legacy systems that still use port 465 for SMTP, and some help pages on the Internet still suggest port 465 as the recommended setup. Our advice? Don’t do it unless your application absolutely requires it—port 465 is no longer an accepted standard for SMTP. (By the way, that’s why SparkPost does not accept connections on port 465.)

Port 587

Modern email servers use port 587 for the secure submission of email for delivery. For example, if you use an email client software like Outlook or Apple Mail, it most likely is configured to use this port to send your messages. It’s not just personal email client software, however. Systems that transmit messages to an email delivery service like SparkPost also should be configured to use this port.

All SparkPost customers should use port 587 as default, unless you’re explicitly blocked by your upstream network or hosting provider. Using port 587, coupled with TLS encryption, is the best way to ensure that email is submitted securely and reliably to SparkPost (or nearly any other provider).

Port 2525

Port 2525 is not an official SMTP port, and it is not sanctioned by the IETF nor IANA. However, SparkPost and many other email service providers support the use of port 2525 as an alternative to port 587 for SMTP, in the event the above ports are blocked. (One notable example where this is required is for services hosted on Google Compute Engine.) If you’ve tried port 587 but experience connectivity issues, try port 2525. Just like port 587, most implementations that listen on port 2525 also support TLS encryption.

Learn More

In summary, SMTP port 587 is the best choice for nearly every use case for connecting to SparkPost and other email delivery services.

Note: If you’re using SMTP to route all of your personal mail through SparkPost, awesome! However, be sure to use an email address with a different sending domain (not one associated with your SparkPost account) for your account login. That way, if you ever run into any issues, you’re still able to contact us for help.

You know you need to send email, but you don’t want to spend a lot of time or effort on it — you just want something that works out of the box. It’s not too much to ask! Many frameworks, languages, and tools come with SMTP support, but the last step is the most important – an SMTP server. SparkPost fills that need with SMTP support and a simple setup process.

Today, I’ll be demonstrating how to set up an SMTP relay, so you can use your own email client to send emails from your personal domain. I’ll be using Gmail as my email client, and shopwithkindness.org as my sending domain.

How to Setup SparkPost as your SMTP Relay

An e-mail client or service which allows you to enable SparkPost as your SMTP relay.

For this walkthrough, I’ll be using Gmail. To begin, navigate to the settings.

From there, click on the “Accounts” tab.

Next, click on “Add another email address you own”.

In the pop-up menu, enter the (verified) email address and press next. I’d like to be able to send with “vincent@shopwithkindness.org”, so that’s what I type in.

Then, enter “smtp.sparkpostmail.com” as the SMTP Server,“SMTP_Injection” as the username, and 587 as the port. Your password should be your API key with “Send via SMTP” enabled. This information can be found under Account -> SMTP Relay in your SparkPost dashboard.

Let’s get started!

Lastly, you’ll need to login to your inbox to confirm. After that, we’re done! Time to send some Shop With Kindness emails.

Other Resources

If it turns out that SMTP isn’t the right email solution for you, consider taking advantage of the SparkPost API. The API has many pros (and cons). Take a look at Dave’s blog for more information regarding the differences between SMTP and API.

Lastly, if you’re having problems setting up an SMTP relay, join our Community Slack channel, tweet us at @SparkPost, or shoot us an email!

-Vincent

Sending email through SparkPost to your subscribers and/or customers can be done two different ways: using our API, or sending via SMTP. The deciding factor will usually be some combination of convenience for your use case, availability/cost of coding/hardware resources, and the relative priority for your business of things like sending speed and ease of migration.

There are pros and cons for both approaches. Naturally, we think our API is pretty great, and that it has advantages over SMTP, otherwise we wouldn’t have it. Offering SMTP in addition to our API lets us support a wider range of use cases. Let’s take a look at pros and cons for both SMTP and our API, as well as examples of each approach.

Sending via SMTP with SparkPost

Sometimes SMTP is the only choice that makes sense, given your constraints. Maybe your legacy system uses SMTP, and nobody is available to write the code to send via API instead. SMTP lowers the friction for this migration path. Common steps such as modifying existing messages by adding a custom header to set the campaign for different message streams, or enabling open/click tracking tend to be significantly less effort than starting to use a new API.

This has a resource (hardware & bandwidth) cost, especially for bulk sends

SMTP is a chatty protocol

Each message requires several round trips to our servers

This adds up to longer bulk send times

Here’s an example of injecting some test content into SparkPost with SMTP, using swaks. The API key you substitute below will need the
Send via SMTP permission, or authentication will fail with
5355.7.8Sorry. .

Sending with the SparkPost API

We like our API, and we hope you like it too. We think you should use it, as there are quite a few advantages over SMTP for many use cases, for example, triggering mail directly from your app’s server-side code. Besides, with a paid account you’re really not getting as much as you could for your money without offloading everything from your systems onto SparkPost that you can, which the API allows you to do.

For larger sends (10k+ recipients) batching or concurrency is recommended

This is a performance sweet spot, not a hard limit

Here’s an example with the same test content as above, using cURL. The API key you substitute below will need the
Transmissions:Read/Write permission, or the API call will fail with
HTTP/1.1403Forbidden, and
{"errors":[{"message":"Forbidden."}]} in the body.

So there you have it! As with most things in life, the answer to which will work best for you is (spoiler alert) “It depends.”. We recommend using our API, unless there are reasons that won’t work for you, such as lack of development resources to make the switch. SMTP is there as a safety net to help support those cases.

Whichever way you choose, make sure you set up DKIM! Authenticating the source and content of your email is very important, and can have a huge impact on the deliverability of your email. Instructions for setting up DKIM are here.

Working with an Email Recipient List

Whenever you’re sending bulk email, or even email to more than a handful of recipients, it’s best to put them in an email recipient list. This is particularly true if you plan to send multiple emails to the same recipients. Otherwise, you’ll have to reference every recipient every time you send an email, and the packet will be large. You can avoid these issues by referencing a recipient list. You can create, retrieve, update and delete recipient lists using the SparkPost API.

A recipient list is a collection of recipients that can be used in a transmission. The Recipient List API allows you to manage these lists. When creating a new transmission, you can submit recipients “inline” as part of the transmission data or specify a stored recipient list ID attribute instead.

When working with recipient lists, you have flexibility in deciding who does what. One person can be in charge of everything (using one API key) or you can implement separation of duties by having multiple API keys. One person can be responsible for creating recipient lists, someone else can be responsible for creating the templates and yet a third person can be responsible for creating transmissions. Each person simply needs his or her own API key with the appropriate permissions. This way, you have three different people working together but separately, and you can marry those functions together whenever you create a transmission.

You can also use the SparkPost API to retrieve a list of all the available recipient lists in an account. This allows you to see a summary of all recipient lists.

You can update an existing recipient list, but it will completely replace the existing list. If you would like to append to a list, we recommend retrieving the existing list, modifying the recipients, and then performing an update.

You can permanently delete a recipient list, but once it’s deleted, it cannot be recovered, so we advise you to keep a backup copy and double-check that you won’t be using it again. If you do need to send to a list that was deleted, you will need to resubmit the list using the Create API call. This is often not a problem because many people use a marketing platform for list maintenance and then use SparkPost for email transmission through our SMTP servers. (To understand how to use SparkPost as an SMTP relay, see this video tutorial)

The Recipient List API operates on lists as a whole and does not currently support management of individual recipients. I’ve made some training videos that explain, step-by-step, how to create, retrieve and delete recipient lists. We’ve also just added a new feature on how to update recipient lists and the tutorial is below:

Email systems have grown increasingly complex since they first appeared in the early 1970s. The growth of the Internet (as measured by users of internet-enabled services) is largely responsible for this complexity. Spam, phishing, and forgery attacks have plagued email users across the world in recent years.

The fundamental problem with modern email systems is that, although we ascribe identity to individual email addresses, that identity is generally not enforced. To this day, it is common to find poorly configured email servers and web email forms that allow malicious users to forge emails.

To understand how this is possible and why it is a problem, we need to understand how email works in the first place.

How does email work?

The internet protocol describing email (called SMTP: Simple Mail Transfer Protocol) functions very similarly to what we now know as “snail mail”. (It turns out that this similarity is very unfortunate). In snail mail, we have two major components: the envelope, and the letter.

The Envelope

The envelope contains addressing information: Who is this letter to? Who did the letter come from? The postal service uses the “To” address, printed on the front-center of the envelope, to get the letter to the right place. Sometimes, something is wrong with this address, and the “From” address (usually printed on the top-left or rear of the envelope) allows the postal service to return the letter to the sender.

If neither of these addresses make any sense, the letter will eventually be destroyed as it can neither be delivered or returned.

The Letter

A letter from a company to a legal office may not mention names of individuals on the envelope. However, the letter will usually have information embedded in it describing who the letter is individually to, and who individually is responsible for the contents of the letter.

What Could Possibly Go Wrong?

The protocol for delivering letters via a postal service requires a large amount of trust. The example letter above is rather inflammatory, and we might have some doubts as to its validity. Consider the following questions:

Who is B. Baz?

Does B. Baz exist?

Did B. Baz write this letter?

Did Mr. Foo receive the mail, or did someone else?

Was the letter altered?

We could probably come up with many other questions related to this as well. Fundamentally, these questions boil down to trust. The postal service does not provide us any way to verify that the sender is who they claim to be. It does not provide us with any way to verify that the sender actually wrote the contents of the letter.

For years, people have sought solutions to these problems. We have invented opaque envelopes to prevent people from reading contents. We use better adhesives to prevent tampering (or at least make it obvious if the letter was tampered with).

With hundreds of years of experiencing mail forgery, delivery tampering, and the like, email must have considered these cases, right? Wrong. Email does not solve any of these problems. Why are they problems? Malicious individuals can exploit these issues of trust to send computer viruses or unwanted marketing email (spam). People exploit these issues of trust to send diseases like anthrax, and unwanted marketing mail (coupon books, credit offers, etc.)

Email and snail mail are not so different. But in email, we can do better.

Trusting An Identity Crisis

We have established two problems with email:

The identity of the sender and recipient of a mail are unknown.

The contents of the message are not known to be correct.

So, if we are the recipient of a letter (or of an email, for that matter), how do we know who sent it? How do we know what we are reading is what the sender actually wrote?

With regular mail, we rely heavily on context and signatures. Humans are generally pretty good at recognizing visual patterns (and discrepancies in those patterns). If we receive a letter from someone we know, we can make an educated guess as to whether they sent it based on factors like writing style, handwriting, signature, and address information.

Imagine receiving a letter from a friend. This letter is asking you for money. You know your friend is on vacation in the Galapagos Islands, and the signature looks funny. You are probably going to request additional information before you send money to your friend.

Email does not enjoy these same protections. There are no signatures — just addresses. And most people are not technologically inclined. So when King Mubotu emails from Nigeria telling you to send him $50 so he can send you $50,000,000, you do it. I mean, it is King Mubotu, right?! Right!!?

So many people are still unaware of this attack, that it is still successful. Again, the fundamental problem is that identity has not been established, and our normal contextual clues for establishing it are gone. So we fall back to trust. If you follow one rule on the Internet, it should be not to trust anyone unless you have any technical knowledge that would make you think otherwise.

Enter DKIM

DKIM (pronounced “DEE-kim”) allows organizations to claim responsibility for messages they send and guarantee the contents of the messages they send. Senders may also guarantee the identity of the individual responsible for sending the message (although this is not yet widely used).

How does the guarantee work?

In a word: math. DKIM relies on what is called “asymmetric cryptography” (also known as “public-key cryptography”). After a message is received, and before that message is delivered to its destination, DKIM uses a “private key” to create a signature for both the contents of the message, as well as some key headers of the message. This signature is attached to the message.

When the message is delivered to the destination, the destination server asks the sender for a public key to verify that the signature is correct. If the public key allows the destination server to decrypt the supplied signature to the same value it computes as the signature, it can assume the sender is indeed who they claim to be.

(Note that we can only assume that the sender is not lying about who originated the email. If the malicious party is the sender in the first place, we have to trust that they are not lying to us!)

As an analogy, consider a message sent between two corporate offices. The message includes a reference number and a phone number. The receiving office can call the sending office, provide the reference number, and read the message. The sending office can verify that this is in fact correct.

What if someone else answers the phone?

In the hypothetical scenario of a letter sent between two offices with a reference number and a phone number to verify, it is logical to assume that a malicious person could attack the security of the system. The attacker would have to:

modify the message,

hijack the phone line,

impersonate the sending office, and

lie about the validity of the message.

This is a difficult attack to pull off. But can we pull off the same attack against a DKIM-signed message? Not really.

When DKIM asks for the public key, it either receives the public key or it receives something that is not the public key. If it receives the wrong key, decryption will fail, and the message will appear invalid. If the message appears invalid, we can assume some aspect of the system has been tampered with, and we can try more strict verification of the message (such as an in-person discussion). This case does not break the guarantee made by the DKIM signature, because it can still be validated with the proper key.

If the attacker responds with the valid public key, all they have done is prove that the message was sent unmodified.

But people have hacked DKIM!

To mount an attack against DKIM, you have to own or reconstruct the private key. This has happened before in, for example, the attack against Google in late 2012. In this case, Google used an insecure key size for their private key and the attacker was able to reconstruct the key. The solution? Use a bigger key! Although this seems like a weak argument, the justification for why this works is outside the scope of this article. I do not wish to attempt to explain the complexity of integer factorization in simple terms. (At least not in this article.)

For the time being, this is a fine solution.

Invalidating Keys

If a key is compromised (either because someone obtains the key by “hacking” the sender or because they were able to reconstruct it by brute force), one can simply change the key. The problem with doing this is that once a key is known to have been compromised, it is impossible to guarantee with certainty that a message has not been tampered with. There are two reasons for this:

Inherently, once the key is compromised, someone can forge messages that would have validated with the key. With some clever tricks, it might be possible for an attacker to even forge dates to make it look like the message was sent when the key was valid.

The public key is generally no longer available. Once the private key has changed, the public key must change as well. If both the public and private keys are subsequently lost or forgotten, it is nearly impossible to reconstruct them.

When doing forensics work with DKIM, if the original key was compromised, it is usually not difficult to prove that messages were not forged. Most people lack the skill to forge keys and emails, and the ability to forge dates requires access to both the sending and receiving machines.

Conclusion

Hopefully this is a useful, lucid, and digestable explanation about the security weaknesses inherent to email (and snail mail, for that matter). If you have additional questions, feel free to contact me either via comments or via email.