The Official HostGator Company Blog!

Yesterday (August 22nd, 2013) a massive number of IP addresses used for email gateways on virtually every webhost in the world became blacklisted on multiple networks. This resulted in a global inability for email to be received (any time the email originated from one of the blacklisted IPs and was “received” on one of the blacklisting networks).

The issue is on-going at the time of this writing, and some customers are still being affected at this moment, however HostGator was one of the first companies to successfully mitigate the situation and we have since been assisting other companies with this issue. As it stands, we are presently working to now get our IP’s removed from the blacklists and restore full worldwide email deliverability from our network.

This situation resulted from a combination of multiple factors stretching back a few months. Before we explain the circumstances, we want to once again stress the importance of keeping all scripts on all hosting accounts updated. Failure to update scripts, as well as not exercising basic security practices, is what allows situations like this to continue to occur. An out-dated script on a hosting account is akin to an unlocked car left in a parking lot… it’s an invitation for maliciousness by unscrupulous individuals.

Unlike the situation back in April that affected WordPress, this time the target was Joomla. Back in May, there was a string of exploits against known vulnerabilities in Joomla. These vulnerabilities, related to a component called JCE, had been previously addressed via certain mod_sec rules. However, a workaround was discovered that allowed malware to be installed, and later activated, to allow the uploading and execution of mailing scripts.

These mailing scripts were activated en masse yesterday, beginning a massive spamming campaign resulting in the blacklisting of email gateway IPs worldwide. One of the largest networks with users reporting issues initially was AOL, resulting in us creating this forum post.

As with all issues of this nature, there are lessons to be learned. The most important lesson here is to (again) keep all scripts on your hosting account up-to-date. Most scripts have a one-click feature to update them anytime a new version is released. Keeping scripts up-to-date is paramount in ensuring a secure hosting account.

HostGator has now added additional monitoring capability to our systems which will alert us to situations like this even faster than yesterday. Our work is on-going, though we should have the majority of the blocks resolved by tomorrow (spam lists move slow, with good reason). But remember, there is no better way to keep your car safe than to lock it. Please take this moment to log into your hosting script back-ends and ensure they are up-to-date. Don’t give the bad guys an open door to walk through.

This is not true, all email with anything to do with AOL is being blocked for the 3rd day. My clients cannot conduct their businesses like this.
This is a disaster. First Provo and now this. What a mess.

The answer to that wholly depends on the receiving network and how long it takes them to properly clear out their incorrect blacklist entries. We do wish we had a better answer, but the final solution truly is beyond us and in the hands of the other networks.

This is why you need to use Google Apps. You can setup your DOMAIN to use google/gmail very easily and still use Outlook or any other email program you are used to. Having email on your own server nowadays is foolish.

It’s unlikely, as I don’t think AOL will be blocking Google anytime soon. And if there is a block, I trust that the biggest email provider and internet monster that is Google will have no problem removing the black list.

NIXSPAM, SORBS, and now SPAMCOP blacklisted HG!!??!
AOL, LIVE, and thousands of other sites use above RBL listings. We and our clients can’t run business this way…Not acceptable!

Use a new set block of IP addresses and switch over until you clean up the old set. There must be much higher resolution than simply email AOL and to ask to be delisted. We can’t just sit around and wait…too many customers and revenue at stake here.
Are you saying that every time HG gets blacklisted, all your clients have to wait for weeks? Really?
Is HG planning to give us credit for using a third party SMTP service provider while this mess is sorted out with solid backup plans in place?

1. HG has multiple gateways. Why the gateways with delisted IPs are not being decommissioned while you are working on fixing the problem?
It’s better to send a delayed email instead of none at all!
2. Why HG is not bringing online new IP addresses?
What has HG done in the last 7 days? That’s a loooong time in technology terms in 2013!

There is no magical “Remove all blacklists” option though.. ANY webhost, not just HG wouldn’t be able to call the CEO of AOL for example and resolve the whole situation in an hour. That is why you’ve been waiting a week.

I think you should educate yourself a bit more before making claims such as “Simply change the IP address blocks” … or “decommission problematic IP’s”… That would cause more havoc on the server than you can even fathom.

Of course this isn’t acceptable. Nobody likes these issues. HostGator didn’t ask to be blacklisted, nor was it planned, so why should they compensate you..? Perhaps because you fail to have a backup plain?

The servers are not offline. Your host doesn’t need to compensate you for something they didn’t cause. Use Gmail or Hotmail or something. Takes less than 5mins to sign up.

Why can I not find anything else that is related to this BL of IP addresses aside form this article on Hostgator and a post on your facebook wall? This leads me to think that the problem is only with hostgator.

I honestly dont see how this is hostgators fault. This affects many different hosts apparently as HG wasnt the only hosting company targeted. IF any of you actually read this, its apparent that the user level is at fault by not keeping your CMS up to date. Now HG has to fix all of this and if anyone has ever been blacklisted or infected with mail sending malware, you know it takes forever to get resolved. Now HG has to work with many different providers, at once, to get everything back in order which is going to take time since the blacklists are in place for a reason, simply “undoing” them all is not an option as many still need to be in place. HG has always treated me well as a customer, righting the wrongs that may or may not be their fault, heck, I even got a free month on them for the provo outage. Moral of the story, keep your CMS up today so you are not a contributing factor to something that someone else has to fix and inform yourself of a situation before placing blame.

Just wondering….how are we, HostGator customers/clients, going to know when this issue is fixed? I logged a ticket about a day ago and it is sitting in a queue w/out HG Tech Support letting me know what the issue is.

Also, I am using Gmail with my HostGator email addy but I’m am having the issue of mail blocked/not getting through..so I guess I don’t have things set up correctly and/or gmail is also blacklisting HG? Sorry…I’m trying to understand the big picture of the issue and might have it a bit confused and I’m still reading through all the comments.

Can anyone comment on what will happen to the emails currently in the ‘holding pattern’ that this interruption is causing? Will these emails bounce back to senders? Will HG have the ability to attempt redelivery once the dust settles? I need to tell my clients something other than, ‘Just wait and we’ll see what happens’.