Archive for August, 2014

During the maintenance window there will be expected downtime of between 5 – 15 minutes for each service. We expect each service, ie mail, stores, admin, ftp etc to go through a brief outage period of approximately 5 mins while the server restarts and after which service will be restored.

The email server is currently down and we are working to resolve the issue as quickly as possible.

The email service went down at approximately 6:00 am PST. Please note that emails sent while the server is down will NOT be lost, they will be delivered once the mail server is back up.

We are working to resolve the issue as soon as possible but do not currently have an ETA on when it will be back up. We understand that this service is essential to your business and are very sorry for the inconvenience.

Update 11:45 am

The email server is back online and working and email is being delivered. Everything is working with the exception of secure connections from third party clients such as Outlook etc. If you are having problems connecting via Outlook turn off SSL security and it will work. Regular connections are working as normal and waiting emails are starting to be delivered (note that it may take several hours for all waiting emails to be delivered based on when external servers will retry).

Update 7:05 pm

Secure email connections are now operational. All email services are working as normal.

What happened?

The issue was caused by a automatic update to our email server anti virus program which caused a few problems including corrupting the ip stack and blocking all secure connections over the RC4 security cipher. While the issues were easy to correct they were hard to locate and our team and Rackspace spent the better part of the day combing through logs and information to figure out what had happened.