We've had an incredibly frustrating experience over the last few months and based on our recent experience, I think it is important that other users know some accurate information about Dreamhost's blacklist problems.

We send SMTP emails to our customers frequently through our dreamhost servers. Over the last few months we've been averaging near 30% of these emails that get rejected due to the fact that the relay that dreamhost uses for these emails is constantly on several well-known blacklists. Probably due to the fact that they offer incredibly cheap hosting to anyone who wants it and many of those people are using these inexpensive servers for malicious purposes. I understand that this is a difficult problem to combat, but this has been going on far too long. There's absolutely no way to operate a site with Dreamhost if this is going to be a constant problem.

We have reached out to customer support about this issue many times providing them with the rejected email headers and error messages to try to help them combat the issue. In our latest ticket we were explicitly told that upgrading to a VPS would solve this issue as we would be using a unique IP and would not be sharing it with other accounts. After being convinced by the customer support staff we went ahead and made the switch. Many hours of troubleshooting and many more customer support emails later we were told that the PHP SMTP emails we are sending from the VPS account were still going through the exact same relay and being blacklisted! The VPS does NOT do anything to solve this problem as Dreamhost apparently still funnels all the emails through the same servers that are currently blacklisted for spamming.

I am extremely upset that Dreamhost's customer support staff misled us by saying that the VPS was the only way to resolve this problem. It's incredibly frustrating and we lost many hours of labor trying to resolve it. At this point I think we are going to have to move to a more reputable hosting provider that doesn't constantly get placed on numerous blacklists. We just can't operate like that.

Dreamhost - please get your act together and tell your customer support team to stop misleading your customers to try to trick them into VPS service.

Essentially you can leave your hosting/dns with dreamhost, and route the email over to an Exchange Online Server.

I think you're going to find out that to solve this problem you will have to separate mail and hosting. Any of the lower cost hosts seem to have mail problems.

Not sure what your hosting needs are but it sounds like you could move back off the VPS to shared, and spend your money on different email. You can even configure email being sent by your webbserver to use the microsoft exchange smtp server, bypassing all of dreamhosts blocked mail queues.

If your not familiar with what an Exchange Server can do for you then you should definitely experiment with the trial. Exchange is a whole different class of mail server designed to work with multiple clients like outlook and popular devices like phones and tablets.

On top of that we have been getting constant "unable to connect to mail server" errors. Have been getting these errors frequently over the last 2 days from multiple email clients. I end up having to try to send an email 3 times before it will go through. And even then there's a high probability it gets blacklisted anyways.

Can we get a response from the Dreamhost staff on this? Email is a major part of website hosting and this is a serious problem.

dreamhost mail is no worse (or better) at email now than it ever has been. Dreamhost does try, and they have adopted several policies over the last few years that help greatly. The only time dreamhost will have a good mail service is when they offer two levels, the current level being one and a premium mail service at additional cost. One of the features that premium service will offer is your own IP addresses for originating mail. Thus if you end up blacklisted it's your own fault.

Of course you can currently get that somewhere else too =] Few DNS entries (and some propagation time) and your email issues can be gone.

Dreamhost's strength is web hosting not email. Check around you will find that the companies that offer great email either aren't in the hosting business, it's and entirely separate division of a very large organization, or they offer poor hosting alongside there awesome email service.

I had similar problems and found that moving from PHP's mail() to using authenticated SMTP (which adds dkim keys as well, useful to avoid being considered spammers) not only resolved most of my problems with outgoing emails but also gave me back the bounces in case of unreachable addresses, that previously were saved to servers's mail directory no matter what reply-to address I was setting.If this doesn't solve, you can always use services like SendGrid for transictional mails (notification, password recover and so on).

We've got quite a large list going of all the blacklists DreamHost's servers are on. Comcast, Roadrunner, AT&T, and many of the other 3rd party lists. It's absolutely ridiculous that DreamHost can't offer a separate relay for their VPS or dedicated customers. They need to take action to do something about this.

And we are already using SMTP authenticated emails. That has done little to solve the problem.

Our sincere apologies for the frustrating experience you had, and for the prior misunderstanding. Indeed, whether you are on a shared web server or a VPS, messages sent using SMTP to connect to your DreamHost-hosted mail accounts on the DreamHost mail server will be sent through there. Mail block is a rather extreme form of spam filtering, where all emails from a single server or IP are rejected, regardless of their contents or anything about them at all. Mail sent using PHP's mail(), sendmail, or similar, would go through the VPS' local postfix mail system and then out directly to the recipient; it will not pass through any other DreamHost relay/server. If you use SMTP, your code connects directly to whatever SMTP server you specify (ours) and mail is sent through that server. Mail sent through the VPS' local mail system is only your mail, and no one else shares that IP address for anything, therefore should not run into blacklisting issues (unless of course you're spamming all over the place). It's not our intent to mislead you, and if using SMTP is absolutely necessary, I can help with canceling the VPS or with anything else on your account. Please let me know and I'll be glad to assist.

Aside from the VPS postfix option, we can also offer a move to newer shared servers (that are unaffected at the moment), and our security team has been working to relay the affected IPs to AT&T and other providers to have them unblocked as quickly as possible once reported. It did take some time initially for us to get the situation straightened out with them despite various efforts, but we now have it under much tighter control and are keeping eyes out constantly for any new blocks. If you have any on your list that are still currently posing an issue, please contact support from your panel with it, and we'll get it to our security team right away!

This all said, LakeRat does also make a valid point. While we stand by all services we provide (especially web hosting) and are working hard to improve our shared email service, it may still be worth looking into specialty/primary email providers when you need it for important business purposes.

Sean - thanks for the response. Always good to know the staff is aware and working on the issue. And I also appreciate the clarification about the different relays for the various PHP mailing options.

That being said, we have still been having consistent issues with SMTP blacklists on our current server and I think it would be good to take you up on the offer to "move to a newer shared server (that is unaffected at the moment)". Please let me know if there's any information I need to provide to make this happen. The board doesn't seem to allow sending PM's, so please let me know if there is a better method to communicate.

....please let us know what information you need to move us to a new shared server.

If you haven't opened a ticket to move to a new server, you will need to do that. Be sure tolink to DH Sean's post above so that when they pick the new server they will take into account your need. It's been said in another thread that servers are hand picked when the customer requests a move.

I just resurrected an old thread to request an API for email when I stumbled on this thread. In that one I said I believe DH has a better handle on block lists than other companies, but I might have been overly optimistic.

For reference, I've been using Everyone.Net as a dedicated email host, even though I get all other shared host services here at DH. This company specializes in email, nothing else, and I use them because I originally thought that they'd do better about keeping my domains from getting affected by spammers that are completely unrelated to me except for the hardware we share.

Unfortunately all of these companies use the same RBLs and when any of those guys blocks a server, it doesn't matter who you're paying for email, you're going to be just as messed up right with everyone else. (This is what we get for depending on 30 year old protocols for modern critical business needs, but I digress...)

DH would make a killing in the entire industry if they took a lead in this rather than chasing after blocks after they occur. Stop being a constant victim to the RBLs that service you. Assign specific mail server IPs to your clients who have been doing business with you for years. Make sure all of these RBL services keep their hands off of these DH IP blocks by giving them confidence that you're going to handle this stuff internally. Get brutal on UCE/spam. Make it a specialty to sell email services to companies that have owned their domains for some number of years, and make agreements with RBLs to shutdown services to specific domains no matter what their IP blocks if you can confirm that these domains are chronic spam sources. Make arrangements with credit card companies to have spammers flagged as card abusers, so that other email service providers can see if a card belongs to a known abuser before providing services. Your typical gmail, hotmail, cox.net, and other massive hosts won't fit in that category but a huge number of other companies will - and that will fund your efforts.

I'd be happy to host my own email services on a DH VPS if I knew that would keep my domains off of RBLs. But frankly I'm not an email server admin and I'd guess most of the people aren't either. We want to pay "someone" for server administration, and for reliable service, which means we aren't affected when someone else does something bad.

C'mon DH, make it happen. The spam wars started in the 90's but we're still victimized by the exact same abuse today. Take a lead and get industry recognition and the revenue rewards for the effort. You're big enough now where you can do it.

No, there is no definitive solution.Let's say you use a mail server with some host and their address is 123.456.789.001. Some group/company maintaining a block list might get a report that spam is coming from 123.456.789.201. They decide to black-list the entire group of IPs: 123.456.789.* (001-254). All domains on all servers and all of their users now have services interrupted, perhaps for one spammer on one domain on one server. And the spam report might have been bad. Some idiot user might have registered to get email from a company and then when they got email in their AOL or Yahoo browser they might have clicked it as spam, intentionally or accidentally. With a few of these the RBLs take action. Your site could be blocked because some idiots out there hit the wrong button on email that nothing to do with you.

And yes, this happens with all email service providers. And you can't host your own email server and avoid the problem because you can't control what some other company says about your IP address.

I said earlier "This is what we get for depending on 30 year old protocols for modern critical business needs". What I meant there is that the email protocols we use today were devised 30 years ago and little has changed since. We spend billions of dollars in patches and anti-malware and RBLs to avoid problems stemming from the use of the old protocols, rather than migrating to a completely new system that should avoid the problems. It's insanity on a global scale but everyone accepts it as though there is no solution.

That said, a company like DreamHost is now big enough to raise a voice in their industry to eliminate some of the insanity, but I'm sure (a) their marketing people aren't aware of how this might work in their favor, and (b) until they can see a way to draw some revenue from a new initiative they're not inclined to even discuss the matter.

OK DreamHost, here's the message you need to convey to the the world:- To RBL maintainers: We own this block of mail servers and we will police it mercilessly. Every RBL needs to keep our block off of their list. Period. If you get a report related to our block, send it to us and we'll take care of it.- To mail server developers: In addition to running RBLs, allow servers to maintain a white list of IPs that never get blocked by any RBLs. Add DH servers to that list.- To DH customers: We have zero tolerance for spam. If you are caught with verified spam from our servers, we will not only remove you from our service but we'll black list you through your credit card company so that no other host will sell you a domain, sell you web hosting services, or sell you email services. You will be banned from using the internet by any legitimate hosting company, and companies that ignores this list will eventually learn to stop providing services to you.- To anyone who uses email (the multi-billion dollar prospect base of companies and individuals worldwide who need services) you can rely on us to keep your email flowing because we've taken pro-active steps to make that happen - and we want your business.

I don't think that will go anywhere, but if anyone at DH has cajones to strike out for some new revenue or at least to make a huge marketing splash, that's a vector to do it.

Until then, no, there's nowhere to hide, at least with IPv4. With IPv6 everyone on the planet may get an IP address and we can address spam at an individual level. But that's just another world-shaking radical approach...

A few things that you really are not taking into account (and there may be others) with your dream scenario is that -- a few pieces of reported spam does not get a server blacklisted... thousands, perhaps 100's of thousands will.-- most legitimate account owners abide by the rules and don't spam, but their accounts may have been compromised and the hackers choose to abuse. Dreamhost deals with this with sending limits, but if the same hacker/hackergroup compromises many dreamhost accounts thru the same exploit then they bypass the sending limit, but the mail from all those accounts may be coming from the same IP.

You're however right that the industry needs to unite an solve the problem with a new RFC. The problem is no one can agree. SPF, DKIM, DMARC (may have missed one or more but those come to mind) were all created to solve the problem, but sadly what's solved it the best so far is RBL's, the drawback as everyone points out is that legitimate email gets caught in the crossfire.

Dreamhost will never say it because it's a negative marketing point elsewhere, but I suspect that the largest portion of the spam leaving dreamhost is sent using compromised accounts. Note that its not so much dreamhost systems that are at fault for the compromised accounts, usually it's an exploit in a web app (like WP) which even when fixed won't close the holes because you can still find many many using an older version that still has the exploit.

Dreamhost and WP have made huge strides to plug the holes and get secuity updates pushed to live site automatically. BUT... what about the person that created a sub-domain a few years ago and installed a test version of (insert name of any web app here) to play around with, they either choose to use or simply abandoned... in in case that old outdated pieced of software is still sitting there on a live sub-domain and all the hacker has to do is dig around to find it, then they run a script that tests for known exploits and poof they are in.... next thing you know here comes spam generation.

No, there is no definitive solution.Let's say you use a mail server with some host and their address is 123.456.789.001. Some group/company maintaining a block list might get a report that spam is coming from 123.456.789.201. They decide to black-list the entire group of IPs: 123.456.789.* (001-254). All domains on all servers and all of their users now have services interrupted, perhaps for one spammer on one domain on one server. And the spam report might have been bad. Some idiot user might have registered to get email from a company and then when they got email in their AOL or Yahoo browser they might have clicked it as spam, intentionally or accidentally. With a few of these the RBLs take action. Your site could be blocked because some idiots out there hit the wrong button on email that nothing to do with you.

And yes, this happens with all email service providers. And you can't host your own email server and avoid the problem because you can't control what some other company says about your IP address.

I said earlier "This is what we get for depending on 30 year old protocols for modern critical business needs". What I meant there is that the email protocols we use today were devised 30 years ago and little has changed since. We spend billions of dollars in patches and anti-malware and RBLs to avoid problems stemming from the use of the old protocols, rather than migrating to a completely new system that should avoid the problems. It's insanity on a global scale but everyone accepts it as though there is no solution.

That said, a company like DreamHost is now big enough to raise a voice in their industry to eliminate some of the insanity, but I'm sure (a) their marketing people aren't aware of how this might work in their favor, and (b) until they can see a way to draw some revenue from a new initiative they're not inclined to even discuss the matter.

OK DreamHost, here's the message you need to convey to the the world:- To RBL maintainers: We own this block of mail servers and we will police it mercilessly. Every RBL needs to keep our block off of their list. Period. If you get a report related to our block, send it to us and we'll take care of it.- To mail server developers: In addition to running RBLs, allow servers to maintain a white list of IPs that never get blocked by any RBLs. Add DH servers to that list.- To DH customers: We have zero tolerance for spam. If you are caught with verified spam from our servers, we will not only remove you from our service but we'll black list you through your credit card company so that no other host will sell you a domain, sell you web hosting services, or sell you email services. You will be banned from using the internet by any legitimate hosting company, and companies that ignores this list will eventually learn to stop providing services to you.- To anyone who uses email (the multi-billion dollar prospect base of companies and individuals worldwide who need services) you can rely on us to keep your email flowing because we've taken pro-active steps to make that happen - and we want your business.

I don't think that will go anywhere, but if anyone at DH has cajones to strike out for some new revenue or at least to make a huge marketing splash, that's a vector to do it.

Until then, no, there's nowhere to hide, at least with IPv4. With IPv6 everyone on the planet may get an IP address and we can address spam at an individual level. But that's just another world-shaking radical approach...

Thank you, great post, and I really wish that someone else read you !

What about SPF and some ways to tell that Emails are not spam ?

It seems that it doesn't work at Dreamhost ?!

I manage shops and mostly Microsoft email services make problems: messages sent by shops do not even go to Spam folder but just never reach the destination box ! Weird and I don't know what to do ?

@Makeonlineshop: SPF does work but it's not always respected. That is, I think there's sometimes so much email that claims to be from a given source that the RBL filter logic can sometimes dismiss SPF as being a definitive confirmation of authenticity.

And @LakeRat explains extremely well how legitimate servers are compromised into spamming engines, so even a SPF-certified email can be spam. I confess I have a bunch of old test subdomains that would be ideal targets for this sort of abuse, but I also use Huge and complex passwords and other mechanisms to minimize the chance that my account will be used for this kind of abuse. (Yeah, I need to clean that up...)

@LakeRat, yeah, I skirted around some details for simplicity. Thanks for acknowledging the core of the matter.

As to agreement on a new RFC, people are really being penny wise and pound (dollar, euro) foolish. The expense of spam and malware avoidance (and basic anarchy) is in the multi-billions yearly. The only thing surprisingly mitigating some of the damage these days is that an increasing number of people are using Facebook, SMS, Skype, and other social protocols for casual exchanges rather than email. But the wires are still flooded with bogus traffic. That's something else that makes no sense. It's dirt simple to trace the source of user activity, to log traffic per user on paid and free services, and yet somehow it's never done, as though it's technically not possible. If we can't stop server abuse at the transaction level it should be possible at the user level. But having provided detailed information about specific abuse sources to Google, Yahoo, AOL, BT Internet, and other companies, I can tell you none of them really care about abuse. "Abuse@" email addresses and similar support vectors are just marketing and legal mechanisms. It's really quite astounding how little these companies care about abuse of their servers ... and to keep this on topic, that leads to RBL bans and other ramifications for paying customers. People at a high level should really be paying more attention because lack of attention costs them money, and being more attentive could increase their revenue. How tough is that for bean counters to understand?