Using Amazon SES in Python with Postman and Postfix

Email can be challenging to set up correctly for high-quality delivery, but this article helps you through the process so that Amazon SES does the heavy lifting, leaving you with the energy to focus on building great software.

Details

A recent addition to the Amazon Web Services (AWS) family is Amazon Simple Email Service (Amazon SES). This article discusses the application programming interface (API) calls to Amazon SES through boto, a Python library for AWS. It also walks you through a sample command-line tool called postman, which is designed for use in your Postfix configuration to send mail through Amazon SES.all while remaining transparent to your applications.

In addition, you will discover an alternative to postman for use with Django.django-ses.and get the pros and cons of both solutions. Finally, the article looks at the information you can pull from the quota and stats APIs as well as how to avoid rejection and Internet service provider (ISP).level spam filtering.

Getting Started

This article assumes that you have already signed up for AWS and added Amazon SES to your account.

Install Postman

Let's start by installing postman, a command-line client for Amazon SES built on top of boto, the leading Python library for AWS. The library is designed to be fed raw email messages from Postfix through its send command, but it also has useful commands for interacting with the service from the command line. For now, install the library and configure your system to work with your AWS account:

Postman

I wrote postman for two reasons. First, I wanted an example for this article.a concrete example rather than abstract ideas that would provide more value to the reader who wants to get something working. Second, it solves a real-world problem for me. I wanted to send mail from a Django-based project I had been hosting on Amazon Elastic Compute Cloud (Amazon EC2) without having to deal with proper email configuration. I wanted to use Postfix so that applications on my server could send email out of the box without special configuration.

The first thing you'll notice is that this is just a simple command-line utility that does some simple wrapping of the API that boto exposes. There's nothing fancy or remarkable about the code, but it serves my dual purposes well.

In standard boto fashion, you get a connection object for the service. Next, call the send_raw_email method on the connection object with content from standard input. That's all there is to sending an email message using Python and boto through Amazon SES. Some notable improvements here would be to catch quota/rate exceptions and try again after a sleep period or.better yet.return the appropriate return code so that Postfix could manage the retry.

Again, this is a simple call to a single boto connection method, verify_email_address. You need to call this method for every email address from which you want to send a message. In fact, while in the Sandbox, you will also need to call this method for the email addresses you are going to send mail to. After calling this method, Amazon SES sends an email with a confirmation link. The recipient must click the link before the address is considered verified. Once verified, you can send mail as that address (or to that address during the Sandbox period).

The list_verified Command

To check which email addresses are verified on your account, you can run the list_verified command like so:

$ postman list_verified

The code for this command is slightly more involved, but it is just cleaning up the return data from the single boto method to provide cleaner output:

The rest of the __main__.py module consists of Python code that parses input arguments and calls the right command function. The module is missing one API call that boto does provide, however: a more structured email send that does not require a raw email message body. Adding this call provides a clean method for sending email messages from the command line without having to structure and PIPE in content with email headers. I will leave that addition as an exercise for later (or perhaps an ambitious reader).

Postman and Postfix

Now that you have installed postman and are familiar with what it's doing, you're ready to hook it up as your default transport for Postfix. I am no postfix expert, but after a bit of searching and finding instructions for hooking up a Perl script in Amazon SES as a Postfix transport, I thought I could do the same thing with postman send:

If, like me, you have a Django project being served on this machine and want it to be able to send email through this postman transport, you need to update two settings in your project's settings.py file:

After saving this change and bouncing your server to reload the settings.py changes, you must verify the email addresses you are going to be sending from:

$ postman verify user@gmail.com

Then, reload Postfix to pick up the new changes to main.cf and master.cf:

$ sudo /etc/init.d/postfix reload

While you're in Sandbox mode, remember that you must verify any emails you are going to send to, as well, so after doing that, try sending a few test messages from your Django project. You can tail the Postfix logs to aide in troubleshooting this setup, but you shouldn't have any problems:

$ tail -f /var/log/mail.info

Quotas and Statistics

There are two limits on your Amazon SES account: a daily quota and a send rate. The daily quota is how many emails you are permitted to send within a 24-hour period. The send rate is how many emails your account can send per second. For example, you start out with a daily quota of 1000 and a 1/email/sec rate, so if you had a batch of 1000 emails to send, you would have to throttle the sending to 1 per second, or else you would get an exception. So, it would take approximately 17 minutes to send all 1000 messages, but after doing so, you would need to wait another 23.75 hours until you could send anymore. The postman show_quota command will assist you in monitoring these limits, which Amazon raises according to your usage over time.

Amazon provides an API to fetch statistics on emails sent grouped in 15-minute intervals for a rolling previous two-week period. These statistics are useful for assisting in monitoring how your application is using and sending email. They provide counts on delivery attempts, complaints, bounces, and rejects. The postman show_stats command prints these figures to the console for you:

You receive Bounce and Complaint notifications via email with the address that either bounced or complained. It is wise to take action and remove the affected email address from your application to avoid future, repeated sends to that address.

Avoiding SPAM Filters

To avoid spam filtering or outright rejections from ISPs, it's a good idea to set Sender Policy Framework (SPF) and Sender ID records. These are Domain Name System (DNS) TXT records that have the following content:

SPF:

v=spf1 include:amazonses.com ?all

Sender ID:

spf2.0/pra include:amazonses.com ?all

If you already have either of these records, you MUST add these entries as additions or replace the current entries, as ISPs will query and see a different authorization and reject them, whereas if they are simply missing, it might allow the authorization through and/or mark it as spam. Bottom line: Add the records to the domains from which you are sending email to ensure high-quality delivery of your email.

django-ses

One alternative to the postman solution presented earlier is a project by boto core committer, Harry Marr, called django-ses, which you can find at Github. It is a Django mail back end. Obviously, this tool only works within the confines of a Django-based project, so it's not exactly an apples-to-apples comparison to postman. However, in the context of a Django project, the tools solve the same problem.

The django-ses project includes user interface elements for graphing and displaying statistics, which may be more useful than pure console output. In addition, depending on where you were deploying your Django project, you may not have control over your Postfix configuration or have a good email solution in place at your host provider. So using postman would not be an option, whereas django-ses is 100 percent Python and is deployed as part of your site.

The downside to django-ses is that sending mail is a blocking call. Depending on what is triggering the email to send, the message may have to wait on the request to Amazon SES to finish before returning. This delay could become problematic in high-performance scenarios.

Summary

With the aptly titled "Simple Email Service" becoming one of the latest additions to its plethora of cloud services, Amazon continues to impress. Indeed, the service is simple.just as email should be. Using postman, django-ses, or even boto directly, you are well on your way to integrating email services into your application with the backing of an email platform that will scale when you need it to.

Resources

This article highlights a aspects of working with Amazon SES. Here are a few more resources available to help you learn more: