No Cold Calling Losers Allowed.

May 02, 2014

Sendgrid vs. Mandrill vs. GreenArrow vs. Postfix

UPDATE May 5th: GreenArrow contacted me after seeing this post, reviewed my installation, and found a mis-configuration that appears to have been responsible for the poor deliverabily to Yahoo inboxes. A new test achieved Yahoo deliverability. In fairness to them and also to determine which solution I'll ultimately keep, I'm going to continue mailing via GreenArrow side-by-side vs. PowerMTA to determine who the winner is.

11+ years into owning and operating NeverColdCall.com and one thing has become abundantly clear: Email is the very lifeblood of my business! Even a slight lift in email deliverability equates to tens of thousands of dollars per month in added income directly into my pocket, so keeping a close eye on my email performance and diligently working to continually increase it is something I do personally, that probably takes up more of my time than anything else in my business - including writing the daily email copy that goes out to my subscribers - and that I'm not willing to outsource to anyone who is not highly trusted and/or strongly recommended from a trusted colleague.

And as my list grows, now to well over 250,000 subscribers, getting emails delivered continues to get more challenging as well as more profitable. The economies of scale are at work here. A 1% improvement means 2,500 more people reading a promo email, which will easily add several thousand dollars to just one business day.

Up until 2009, when my list was still well under the 100,000 mark, my setup was simple: I sent mail directly from my web server via Exim. It was very slow, but 30,000 emails/hour isn't bad when your list is that small, and performance was superb: 100% deliverability according to ReturnPath. Of course, there is probably no such thing as 100% deliverability, and all stats are relative, but that's still a damn nice number to see. (I still have that screenshot saved somewhere!)

When I moved to Texas in 2009 and gave up the endless partying-and-drinking-and-eating glutteonus lifestyle that I lived during my first six years of NeverColdCall.com, three in Scottsdale, Arizona and three living in Newport Beach, California, two things happened: I began paying more attention to my business and gave up the "four hour workweek", I mysteriously lost about 25 pounds without really trying, and my email list grew and grew and grew. So did my profits - and with it, my ability to invest in email infrastructure.

(Since moving to Dallas in 2012, that 25 pounds is starting to reappear so back to the gym I go.)

Hence my hodgepodge of email services and servers.

In 2009 a friend who owns a web hosting company in San Antonio offered me some spare office space in his building that I happily accepted. I not only had unimaginably fast Internet speed sitting in a datacenter but also had access to the company's PowerMTA mail server.

That was the upside. The downside is that the person who managed it, who no longer works there, did a lousy job and spent more time at the office working on his own MLM nonsense than keeping emails going.

After one particularly infuriating incident when I was on vacation and learned that none of my email had gone out for several days, I did a Google search, called SendGrid, set up an account, and started sending with them immediately.

The upside was an immediate and noticeable increase in open rates and sales volume. The downside was the cost - with a list as large as mine and near-daily email broadcasts, I was on their $1,200/month program, and at that cost is absolutely, positively makes sense to own your own infrastructure. Heck, that's $14,000 a year, which is the cost of a PowerMTA Enterprise license, the kind that SendGrid themselves use. And you only have to buy it once!

Needless to say, I explored other possibilities. A referral to a mail expert resulted in a custom Postfix setup that cost me $400 to have built and was able to load-balance/rotate up to 8 dedicated IP addresses. Rackspace happily sold me the IPs and off I went.

I used that for the next couple of years with solid results. Zero cost to me other than the $20/year to be on the Barracuda whitelist and great results.

Then I decided to try SendGrid again. And Mandrill. It was damn near impossible to gauge results, and the ESP route was getting expensive again, so I decided to finally buy PowerMTA. When a Google search brought up GreenArrow Engine MTA which advertises as a lower-cost alternative to PowerMTA but is equally good.

I gave them a try on a 2-month trial, which is still happening even though I stopped sending through GreenArrow due to dismal results.

But with the GreenArrow trial came a month of free GreenArrow Monitor, a seed list email monitoring system similar to ReturnPath's monitor. The seed list is about 550 emails strong which means accurate results, versus just one Yahoo inbox, one Gmail inbox, and so on.

The results were pleasantly surprising and disappointing at the same time - pleasantly surprising because my free Postfix server performed best, at least in terms of my lists's composition, with Mandrill taking second place, while GreenArrow, the most promising but fairly expensive of the bunch, performed poorly, as did SendGrid. (Again, this is all dependent on my list's composition - GreenArrow may be great for your list if it's low on Yahoo subscribers.)

First of all, my list composition - because NeverColdCall.com includes a huge percentage of professionals and business owners, Gmail dominates my list composition - here it is:

Gmail: 37%Yahoo: 21%Hotmail: 18%AOL: 4%Other: 20%

For the purposes of this article, we'll just look at the first four - results were fairly comparable across all MTAs and ESPs for the "other" category since my emails tend to get delivered very well to corporate email servers regardless of who I use to send them.

SendGrid:

Mandrill:

GreenArrow Engine:

Postfix:

Not surprising to any email marketer, the big challenge is getting delivered to Yahoo, which is notoriously difficult. And most surprisingly to me, Postfix is the one that got the job done. GreenArrow and Mandrill failed nearly 100% with SendGrid hitting about 50% spam on Yahoo.

Interestingly, all four achieved 100% inbox on Gmail, obliterating the myth that Gmail is especially difficult when it comes to getting to the inbox.

These results also show that infrastructure isn't necessarily as important as management - both SendGrid and Mandrill use PowerMTA but the results are dramatically different. I think this has a lot to do with Mandrill/MailChimp's very strict requirement for low bounce rates and spam complaints. SendGrid is a bit more liberal, and when your dedicated mail IP is on the same Class-C block as email marketers with poor reputations, you're bound to get penalized as the entire Class-C is greylisted, so props to Mandrill/MailChimp for their high sender requirements, despite users' whining about being suspended by them.

The Verdict

For me, it's a toss-up between Mandrill and Postfix. Postfix is not reaching Hotmail - 18% of my list - while Mandrill is missing 9 out of 10 Yahoo users, which, ironically, is about 18% of my list. What matters most for me is getting into the Gmail inbox, which I'm doing on any and all platforms, perhaps due to the age and reputation of my hostname vs. the individual IPs, MTAs, and ESPs.

Since Mandrill is pay-to-play and could suspend me at any time for a mailing such as an affiliate promo that might produce a high percentage of spam complaints and/or bounces, Postfix it is - for now.

The real verdict is that I've bought PowerMTA and it's being installed as I write this. As Andrew Dice Clay famously said, "If you can't party with the big boys, don't show up."

New York Times best-selling author Frank Rumbauskas is the author of the Never Cold Call Again® System and has won numerous accolades, such as Readers Choice for Business Book of the Year from 800-CEO-READ, and has been named one of Fast Company’s top 30 most influential people online. To learn more, and to download a free 37-page PDF preview of his Never Cold Call Again lead-generation system, visit http://www.nevercoldcall.com/

Comments

Dear Frank J. Rumbauskas Jr.,

I am very interested in your blog with this post "sendgrid-vs-mandrill-vs-greenarrow-vs-postfix".

I am finding a solution for our email marketing system and I have to decide between PowerMTA and GreenArrow.
After I read this post of you, I am confusing about the actual values that PowerMTA & GreenArrow brings to us.

Could I have an advice from you who have many experience with email system. I'd appreciate a reply from you.

Frank Rumbauskas is a New York Times Best-Selling Author, one of Fast Company's Most Influential People Online, Winner of Business Book of the Year from 800-CEO-READ, The World's Best Sales Trainer, The #1 Sales and Marketing Strategist on the Planet, and a very humble and loving husband, dad to two beautiful daughters, and all around great guy!
To get Frank to explode your organization's sales numbers until your head is spinning, your wallet is bursting at the seams, and everyone thinks you're a genius, contact him at frank@frankr.me