Network Status

Whenever any of our Network Services has a problem, outage, or maintenance, we’ll update this page . All times listed in Pacific Standard Time (PST).

Note: We reserve every Friday Night from 10pm PST to Saturday 3am PST as our maintenance window for Network Services. We do not always use this time but we need to reserve this window of time to enable scheduled maintenance and upgrades of the network.

Event

Status

Time

Update: We have discovered that not only were we not receiving all tickets into our support desk, many of our replies were not getting sent by the mail server. We believe that this has all been fixed and communication from our Support Team will be vastly improved because of it.

November 7, 2016

2:04AM

Update: We have a new uploader ready to use in a beta version. Currently, it can be only be reached by going here when logged in to your Redframe admin area: http://admin.redframe.com/_1/index.cfm?manage=S3_uploader. It is in Beta but ready to use. Feel free to test with a few images before starting a large upload. This is currently only available for Redframe galleries, not Online Proofing. You should notice much improved performance speed with images being ready for your website. *there is a possible issue with IPTC information not always processing. If this is crucial to you, please test with a few images first to see if you are getting the result you want.

It is in Beta but ready to use. Feel free to test with a few images before starting a large upload. You should notice much improved performance speed with images being ready for your website. *there is a possible issue with IPTC information not always processing. If this is crucial to you, please test with a few images first to see if you are getting the result you want.

November 5, 2016

11:39AM

Update: The uploader is experiencing heavy volume and is taking time to process all images. It is currently functioning and we are monitoring the process closely

November 4, 2016

1:12PM

Update: You can resume editing and uploading on your website. As we still have not resolved the issues plaguing the network we ask that you please limit your uploads to a maximum of 1000 images at a time. This will help to not overload the network while we continue to debug. We are continuing work to restore missing galleries, broken contact forms, and other bugs.

Update: We are at about 85% complete on the database restoration at this point.Once we replace the database, we will do some testing and hopefully announce the admin safely open after that.

November 3, 2016

5:06PM

Update: We are in the process of reverting to a backup version of the database. From the current speed of the process, we estimate it will be another few hours before the database information is restored

November 3, 2016

2:27PM

Update: In order to restore websites as promptly as possible we are rolling back to a version os the database from midnight. Please refrain from uploading and edits until further notice as changes from midnight last night will most likely be reverted.

November 3, 2016

1:01PM

Update: we are stillworking on restoring galleries that have disappeared.

November 3, 2016

10:45AM

Update: The proxy server was overloaded again. Some sites and admin areas were unreachable for several minutes. This has ben restored.

November 3, 2016

9:22AM

Update: We have reports that some recently uploaded galleries are no longer showing up on websites. we are looking into issue.

November 3, 2016

8:15AM

Update: We have reports thatcontact forms on websites are not functioning properly. We believe this to be an issue with our mail server. we are working on resolving it right now and believe it will be resolved in the next 2 hours.

November 3, 2016

7:46AM

Update: There is an extremely high volume in the upload process at the moment resulting in a long delay in the images being fully processed to display on your website. We are working to optimize the process to improve processing efficiency.

November 2, 2016

9:57PM

Update: The proxy server was overloaded again. Some sites and admin areas were unreachable for several minutes. This has ben restored.

November 2, 2016

3:55PM

Update: Proxy server was down again causing another outage. Has been restored.

November 2, 2016

12:16PM

Update: Proxy server was down again causing another outage. Has been restored.

November 2, 2016

11:42AM

Update: Proxy server was down for 5 minutes interrupting service for many sites. Service is back up. There are still reports of getting errors when trying to access the admin. Hitting refresh on your browser should load it.

November 2, 2016

10:07AM

Update: We still have a very high volume of traffic and upload on the servers. All servers and processes running as expected. There have been several issues with service of sites and access to the admin today. We have been working to clear them up and prevent them from continuing.

Recovery of the content missing from the crashes servers is still in progress. At this point, we are recommending that you replace or edit missing galleries or pages if possible.

November 1, 2016

10:00PM (PST)

Update: Due to an extremely high volume of traffic and uploading, we are experiencing difficulties with the servers. If you encounter a page with an error message, please try refreshing your browser. We are working at having the issue solved shortly.

N;ovember 1, 2016

4:16PM (PST)

Update: The admin is operational and open for site edits and images uploads. The volume of the uploads is extremely high at the moment so there is some delay in images processing on the server, but processing is occurring. We are aware that some sites are still missing content, especially recent uploads, and our next focus will be restoring that data.

Ocotober 30, 2016

8:37PM (PST)

Update: The servers are now running optimized. We are aware that there are still some sites with missing content, and continue to work to resolve those. We are working at having admin access available by this evening to allow for edits and uploading.

October 29, 2016
9:09AM (PST)

Update:Theservers are now running efficientlyandload balancing is optimized. Most sites are now delivering quickly. There are still some sites that are missing content from the migration that need to be rectified and we are addressing these on a case by case basis. Our next step is progressing towards opening the admin area to allow editing and uploading.

October 28, 2016
9:28PM (PST)

Update: The transfer to the new servers has been completed. Though most sites affected by the server outage should be up and running smoothly, we recognize that there are still sites missing content or not delivering properly. We will continue to progress with improvements throughout the day to get all aspects of your site operational again.

October 28, 2016
10:07 (PST)

Update: Progress to report at last. We have made the switch to point all websites to our new server network. This should take an hour or two to resolve, and I expect that most currently affected sites will be delivering properly. It will not be perfect for all sites and there may be some glitches that we can address on a case by case basis. The admin will not be operational right away. We ask that you not try to begin uploading images as this is not fully ready. For the most part, all currently affected sites should be fully operational for your clients to view and purchase existing images. Some data that has been uploaded in the last 7 days may not be available yet. This will be synched up in the next few days. Thank you

October 27, 2016
4:35 (PST)

Update: As many of you who are checking for updates can tell, we have still not got many of the content for our client sites back online. I am desperate to provide you with better news and the sporadic nature of the updates is mostly due to the fact the I realize the only update you want to hear is that the issue is fixed. At this point we are still trying to complete the server migration as well as attempting to get the original failed file server back online and delivering website content, attempting whatever way we can get your website back quickest.

October 27, 2016
9:25 (PST)

Update: As you may be able to tell, we have yet to complete the migration to the new servers and get affected sites delivering correctly. We know that you entrust us with your website hosting and service needs and we are letting you down at the moment. Our team has been working continuously since yesterday to get the migration completed. We will continue to work through the night tonight until we have all sites delivering properly. Thank you.

October 26, 2016
7:23 (PST)

Update: We have almost finished the server migration. We will need to a round of testing before delivering websites from the new server. We are estimating between 2 or 3 more hours before all sites are back online.

October 26, 2016
1:16 (PST)

Update: The server migration is still in progress. We are aware that many sites are still missing content. We are working to have everything up and running as soon as we can.

October 26, 2016
7:55 AM (PST)

Update: Due to the unfortunate technical difficulties encountered by one of our existing servers we have decided to commence with the planned migration of our platform to a new server network. If your site is currently affected by the server problem, it should be delivering properly in the next few hours.

During the transition to the new platform we ask that you temporarily suspend making any uploads or changes to your site so that they do not get lost in the server migration.

October 25, 2016
6:48 PM (PST)

One of our servers have been down since 12:15 pm PST and it is affecting the content of a portion of the sites which we host. We are working on finding out the cause and a solution. We will update this page as we have more information.

October 25, 2016
1:45 PM (PST)

Our Network has been restored as of 7:17 PST. All sites are up and running. It appears the majority of the trouble was finally isolated to a faulty switch (which is the hub of the network and connects all the devices within the network ). Once the switch was finally pinpointed and replaced the network was back up and running.

Warren H.

August 26, 2016
7:30 PM (PST)

Our servers have been down since 11:57 am PST. We are working on finding out the cause and a solution. We will update this page as we have more information.

August 26, 2016
1:00 PM (PST)

All websites are operating normally. The 209.80.10.130 IP address has been restored. If you were affected by this issue, then please click here for information to remedy the issue.

April 5th, 2016, 8:30AM (PST)

Around 7:00AM (PST) we were able to get our 209.80.10.130 IP address closed. This allowed the rest of our network to work smoothly. If you are unable to see your website at this moment, then please click here for information to remedy the issue. For everyone else your site should be running smoothly now. The .130 port should be open again on Tuesday 6:45AM PST. We will update you when this happens.

April 4th, 2016, 7:00AM (PST)

Around 6:45AM (PST) we received alerts that some clients were unable to reach their website. We are currently experiencing a DDoS (Flood attack) on our 209.80.10.130 IP address. If you are unable to see your website at this moment, then please click here for information to remedy the issue. We are working with our hosting company and bandwidth providers to resolve this issue as quickly as possible. We apologize for the inconvenience this may cause you and your customers. We understand how important your website is to your business and are doing our best to get this resolved.

April 4th, 2016, 6:45AM (PST)

Well the recovery was complete earlier this morning. Redframe and ifp3 clients now have all their galleries and sites back to normal. A good part of the day was spent testing and responding to issues as they popped up. We do have a dozen clients or so that we are still trying to clean up. We think we should have them mostly taken care of by tomorrow. If you have an issue or something is not correct about your site please let us know. We truly want to get everything worked out for each and every client.

We’ll get an email out tomorrow morning to everyone saying the recovery is complete for those that were not checking the status here. I will also have an email and blog post sent out to everyone in the next couple days explaining what went wrong, how and why we recovered so poorly, what we will be doing to ensure this never happen again and how I can make it up to all the clients we messed up. Up next – a great deal of sleep for everyone involved 😉 .

Thank you for all your wonderful support. We truly messed up some of your sites yet most everyone hung with us. That is truly encouraging since the reality of the internet is that you are really just a few clicks away from putting your business somewhere else.

This will be the last network update for the failed file server issue.

Warren H.

March 30th, 2016, 11:30PM (PST)

All affected Redframe clients are now delivering our recovered galleries. We have personally looked through a few hundred thousand images and have not seen one missing image. Ifp3 clients with a flash site: we almost have the connection to your archived gallery files. There are a few more workarounds we need to do to get them to show on your sites. We expect to have them all worked out by tomorrow.

You may notice that if you try to make some edits to these older gallery images we will prevent it. You can move images around to different galleries or the library but you can’t rename or rotate them. We have locked these features down until we can get these galleries into a new fully backed up location. This is because we are putting all these files in the next couple of weeks to the Amazon cloud and we want to ensure the data stays the same until then. Since these files are a few years old we expected many clients would not attempt these types of edits.

I will post another update as soon as we know more.

Warren H.

March 30th, 2016, 3:00AM (PST)

We sent out our first email letting everyone affected know that website layouts and pages have been restored. The response into support has been very positive so we must have done it correctly. But Wow was that a painful process! We will ensure in the future that we never ever have to do that again. There are a few clients who had small bits of corrupted data and we have been rushing to help fix each request as it comes in. If you find anything wrong with your site (not related to gallery images) please let us know. It appears the odd case seems to be a page that won’t revert to 3 weeks ago or some page image missing. We will do our best to get a solution to you quickly.

The recovered gallery drives have been going less smoothly. We did anticipate this was going to be the case with the format we received the files in. Yet like last Friday we will get it all working. It will just take a little extra work. Once we get it well tested we will turn the data live to our affected client websites and let you all know. I will post another update as soon as we know more.

Warren H.

March 29th, 2016, 7:42PM (PST)

We have finally completed the hand checking of hundreds of sites. I’m sure there will be the odd anomaly but we feel like it went very well. We will be sending an all clear email to look at the changes to the site tomorrow afternoon. I wanted to give the support team some time to clear out the morning tickets incase there are a lot of questions. If you are reading this go ahead and check your site and if you have any questions let us know.

We are installing the recovered gallery drives into our network and working to get it all flowing error free by or before Wednesday. I will post another update as soon as we know more.

Warren H.

March 29th, 2016, 2:20AM (PST)

We have finally received the gallery image data and we will be working through the night to get that restored. Website layouts and pages have been restored. We are in the process of hand checking sites to insure that everything is where it needs to be. We will update you as soon as either of the above tasks are complete.

March 28th, 2016, 10:29PM (PST)

Well 99% of the recovery of our client website layouts and pages is complete. I believe there are only a couple dozen clients we are still cleaning up. We completed a bit less work than we expected but everyone got a bit more time with family. Two of us put in a 14 hour day to clean up as much as we could and 3 support members kept popping in throughout the day to help with issue tickets. We will be at full strength tomorrow though so our last remaining fixes should get completed and if all goes well we will get an email out to everyone by the end of the day saying this part of the recovery is complete.

The Data Recovery Team called us this evening to say our gallery recovery is complete. This is where the serious amount of data is. We have already worked out the kinks of how to access the format they will hand this data to us in but we have had to alter our plan a bit. Transferring the shear number of files into our replacement file server will likely be too time consuming (days) so we have worked out a way to access the files directly from the recovery drives while still having fast delivery times to each website. My best guess is we will have all those older galleries accessible to your sites by Tuesday or Wednesday.

I am sincerely sorry for this mess I have put you through. There is a solid plan in place to be sure this never happens again.

I will post another update as soon as we know more.

Warren H.

March 28th, 2016, 5:18AM (PST)

We are about 95% of the way through the restore process. We have run all the scripts to transfer over the recovered data. Around 80 clients failed to update during the script (long story there). So we have been having to partial script each client individually and then hand transfer over the remaining recovered files. Long and tedious process. Then we are hand testing every client we are aware of that is affected and confirming and/or fixing their sites. The team is a little tired, to say the least. So I have them hitting the hay for a few hours before we get back up and complete this process.

Tomorrow is obviously Easter. Typically we are completely closed. Yet we need to clear this mess up and have everyone’s sites back up and functioning well. So I will have 2 crucial team members and myself working while everyone else will be taking the day off to be with their families. Officially support is closed tomorrow but we will keep looking into the ticket desk and do our best to help the most urgent of issues.

We expect to complete this process in the wee hours of the night. Yet I am sure some work will be pushed until tomorrow when we can get to it with clearer heads. Older archived gallery images will not be a part of this restore process. We are hoping to get that data next week from the data recovery team. We will post another update as soon as we have more to say.

Warren H.

March 26th, 2016, 9:22PM (PST)

We are about 80% of the way through the restore process. Some clients are getting back with us saying their sites look great again. What a relief! We expect to complete this process within the next couple of hours. Older archived gallery images will not be a part of this restore process. That will be next. We will post another update as soon as this restore process is complete.

Warren H.

March 26th, 2016, 3:55PM (PST)

Your website is being restored! We are about half way through the restore process. We expect to complete this process within the next few hours. Older archived gallery images will not be a part of this restore process. That will be next. We will post an update as soon as this restore process is complete.

March 26th, 2016, 11:18AM (PST)

Things are moving much better now. The data can be accessed from our network yet we’ve had to change our plan to accommodate the format that it is in. Instead of plugging in the recovered file server and updating it (original plan) we are doing the reverse. With 98%+ of the data actually on our backup fileserver (a fast device designed for our network), the new logic is we update our backup file server with the “old” files that were not correctly updated. Obviously it is a bit more complicated than that since all changes over the last couple weeks need to be appended. Yet there is a finish line in site. And that has everyone excited. I will keep you posted as we progress.

Warren H.

March 26th, 2016, 2:45AM (PST)

I am very sorry for the late update. We have been struggling all day to get the recovered data to run on our system. Evidently the data recovery team, to expedite the process, put the data in a format our system did not natively work with. The data for all the pages and backgrounds do appear to be there though – we can see them. So our team is now in the process of making our network communicate (at high speeds) with the data. Now it is up to our team (finally) to work through the night to get this all working. I will keep you posted as we progress.

Warren Hollinger – President/Founder Redframe & ifp3.
Note: I have been posting these updates for the last couple days and I am intimately involved with the team to get this resolved.

March 25th, 2016, 8:50PM (PST)

We are plugging the recovered data into our system. Then we will be inspecting it to be sure all the data is there before we spend the next couple hours updating it. We will keep you posted as we progress.

March 25th, 2016, 12:50PM (PST)

We are waiting at the data recovery office for the team to deliver us the data. The file copy process is at 99%. We will update as soon as we have the data in hand.

March 25th, 2016, 9:02AM (PST)

We have another short delay of around 6 hours. The best way I can describe it is like this: If you have ever done a large download and when you started it said 20 minutes. A few moments later you looked down and it changed to 9 minutes. You walk away and when you get back it says 11 minutes. It is sort of like that. The data recovery team has a meter that works similarly giving them a best guess as to when the data will be fully copied. When it is going through easy chunks of data it gives a nice low number. When it it hits large chunks of data – like a client who has 500,000 images – it gives out a longer completion time. Luckily we don’t have too many clients with that much data so we feel good about their 7am PST prediction. Once we plug this into our network we can predict it being available around 3 hours or so later as described in the previous update. Our IT team is on 24 hour standby and we will update you again in the morning when we know more.

March 25th, 2016, 12:00AM (PST)

Great news from the data recovery team! They will be handing off the recovered data to us by around 2:00am tomorrow morning (Fri Mar 25th) 8 hours from now. We will get it plugged into our network soon after and then start the process of updating all new changes over the last 2 weeks to that data. (Note: all pages from 2 weeks ago will be how your sites will display.) If all goes well that will be another 3 hours or so. At that point we will be responding to immediate tickets to see how the recovery went. Once we are sure all looks solid we will get an email to everyone. We will update here as we know more.

March 24th, 2016, 6:30PM (PST)

Good news from the data recovery team! The data is now aligned and they have rebuilt the directory tree. The files we need are now being copied out for delivery to us. We do not have an estimate yet as to when we will receive the files. The data recovery team will need to copy more of the data before they can provide a rough estimate. We will post another update as soon as that happens.

March 24th, 2016, 9:45AM (PST)

Things are moving along… The data recovery team is currently rebuilding the directory tree. This is the last step before they can begin copying the files out for delivery. This process will take about 7 hours, but this estimate can very. After that they will begin copying the files. We do not yet know how long the files will take to copy, but we should have some kind of an estimate once that process begins. However, we do have a plan in place to copy all non-gallery files so every site affected will have their sites looking and running properly. Then the data recovery team will start copying any missing gallery images next. These gallery images are pre-2012 and are mostly archived images. We will be restoring any data we receive the moment its in our hands. We will post another update as soon as we can get an estimate on the file copy process.

March 23rd, 2016, 6:15PM (PST)

Good News! The data recovery engineers have confirmed that the data has lined up correctly. They are building a directory tree now to begin the file copy process. They will notify us as soon as the file copy begins at which time we will provide another update.

March 23rd, 2016, 3:55PM (PST)

The data recovery engineers are looking through the results of the last round of tests right now. They will let us know the results within the next 30 minutes at which time we will post an update.

March 23rd, 2016, 2:15PM (PST)

The data recovery engineers just provided us with another update. Again, they are 100% confident that they have all the data. The custom program the engineers created to “align” the data “partially” worked. They are working on a modifications to that program and will test again within the next few hours. The data recovery engineers believe they are very close. We will have more information by 2:00pm PST today at which time we will provide another update.

March 23rd, 2016, 10:25AM (PST)

The data recovery engineers just gave us another update. They will be working all through the evening on this recovery. They are 100% confident that they have all the data. However, aligning the data has proven to be a difficult task. The engineers are writing a custom program to reach further into the data in order to get the alignment correct. They expect to have the data aligned around midnight tonight for another run. If the alignment succeeds, then they can begin copying out the files immediately. We will know more by 10am tomorrow morning. We will post another update by then or sooner if we receive more information.

March 22nd, 2016, 8:15PM (PST)

Our technicians are working on this 24 hours a day. We truly understand your frustration. I know we are close to getting that data back into our network. The Data Recovery Specialists have a complete copy of the file server now, but the data on it is not going to be usable until it is properly aligned. This is what the data recovery team is in the process of doing. It’s exciting to know all the data is complete – just frustrating to not be able to plug it into our network yet. Around 8:00PM PST we should know if the new alignment process worked and the files are being copied. At that time we will update our Network Status page with the results. When the final copy process begins we should have a more accurate picture of when this data will be restored.

March 22nd, 2016, 2:10PM (PST)

There was a delay on the restore this morning around 6AM which temporary halted the process. Some of the data needed to be recovered from the original source. The restore is back underway. The engineers working on the data are still hopeful to have the data restored at some point today. However, they warn that we be looking at tomorrow. We will update this page as soon as new information becomes available.

March 21st, 2016, 1:30PM (PST)

We are still in the process of restoring the data on the file server that failed. The technician is working around the clock until this is complete. There is a lot of data to move around. We expect a full restore of all files and images between Sunday, March 20th and Monday, March 21st. We are doing everything we can to get this resolved as soon as possible.

March 20th, 2016, 11:00AM (PST)

On Thursday, March 10th one of our file servers failed. We are in the process of restoring the files and images on that server. Some sites may be missing images and page text until the restore process is complete. Unfortunately, this process is taking longer than we expected. We expect a full restore of all files and images between Saturday, March 19 and Sunday, March 20th. We apologize for the inconvenience and we are doing everything we can to restore the data as quickly as possible. We understand how important your website is to your business and are doing our best to get this resolved.

Note: We had been updating this thread dated March 18th for the last 7 days. The person responsible for updates was just overwriting the earlier update and changing the date. I did not add those below because that was not at all how we presented it at the time. Sorry for the confusion.

Warren H.

March 18th, 2016, 7:13PM (PST)

All websites are operating normally.

March 12th, 2016, 4:20PM (PST)

Around 4:00PM (PST) we received alerts that some clients were unable to reach their website. We are currently experiencing a DDoS (Flood attack) on our 209.80.10.130 IP address. We are working with our hosting company and bandwidth providers to resolve this issue as quickly as possible. We apologize for the inconvenience this may cause you and your customers.

March 11th, 2016, 4:00PM (PST)

All websites are operating normally.

March 7th, 2016, 3:15PM (PST)

Around 1:10AM (PST) we received alerts that some clients were unable to reach their website. We are currently experiencing a DDoS (Flood attack) on our 209.80.10.130 IP address. Attackers will commonly use a bot or virus installed on many computers to flood a specific server or IP address. This is a common attack since flooding a server with requests does not require any authentication, even a highly secured servers are vulnerable. Our servers have not been compromised and your information is safe. We are working with our hosting company and bandwidth providers to resolve this issue as quickly as possible. We apologize for the inconvenience this may cause you and your customers. We understand how important your website is to your business and are doing our best to get this resolved.

March 7th, 2016, 10:15AM (PST)

All websites are operating normally.

December 24th, 2015, 1:55PM (PST)

Around 11:55AM (PST) on we received alerts that some clients were unable to reach their website. We discovered a problem with our primary database.
We were able to isolate and fix the issue. We apologize for the inconvenience. There should be no further interruption of service.

December 24th, 2015, 12:15PM (PST)

All websites are operating normally.

November 11th, 2014, 9:15AM (PST)

Our servers are processing images waiting in the que. These images will be processed within the next 30-45 minutes, and will appear in your Admin designated Gallery or Library.

If you attempted your upload multiple times, your images will upload multiple times and show in your Admin with a new image name, for example Birthday.jpg, Birthday1.jpg and Birthday2.jpg, etc.etc.

Any images uploaded last night/yesterday 10/10/2014 between 8pm and Midnight PST will not be processed by the servers. If you attempted an upload during this time, please upload your images again.

We apologize for the issue and problems you encountered uploading.

November 11th, 2014, 7:40AM (PST)

All websites are operating normally.

November 10th, 2014, 10:38AM (PST)

We have isolated the issue to a single server on our network. We are taking the server offline for further investigation.

November 10th, 2014, 9:45AM (PST)

We have been notified that some clients are not able to reach their websites. Some clients report that their customers can reach their website, but they cannot. We are working on the issue.

November 10th, 2014, 5:38AM (PST)

All websites are operating normally.

March 12th, 2014, 9:10PM (PST)

We received notification from our hosting facility that the issue has been resolved. We have requested an After Action Report and we will investigate what we can do to prevent outages like this in the future.

March 12th, 2014, 8:57PM (PST)

Looks like all sites are back on line. We are still waiting for the ALL CLEAR from the network engineers.

March 12th, 2014, 7:38PM (PST)

The network engineers at our hosting facility have isolated the issue. One of the networks hosted within the facility was broadcasting millions of routes and bringing down one of the central shared routers. They are working to shut that client down now. We should be back to 100% within the next 30 minutes.

March 12th, 2014, 7:15PM (PST)

Around 5:55pm (PST) on March 12th we received alerts that some clients were unable to reach their website. Our hosting facility reported a major outage affecting their entire network. They have their emergency engineering team looking into the issue now. We have been told to expect a full report along with an ETA shortly.

March 12th, 2014, 6:10PM (PST)

All websites are operating normally.

July 7, 2013, 5:10AM

Around 3:00am on July 6th we received alerts that some clients were unable to reach their website. Our hosting facility reported that a shared firewall had been impacted by a DDOS attack on another company within the facility. The target IP addresses have been blocked and the issue has been resolved. We have asked our hosting company for a report on exactly what steps will be taken to prevent this in the future. We have been told to expect a full report by July 10th.

July 6th, 2013, 10:40PM

All websites are operating normally.

Jan 11, 2013, 4:23AM

At approximately 3:40am PT we received alerts suggesting a network related issue. Our hosting facility reported that a shared firewall appears to have been impacted by a DDOS attack on another company within the facility. The target IP under attack was blocked at approximately 4:23am PT and all services were restored.

Jan 11, 2013, 3:40AM

All websites are operating normally.

Dec 11, 2012, 12:10AM

The network was offline momentarily this evening between 11:07 and 11:35 (PST) in order to replace a switch that may have contributed to last nights outage.

Dec 10, 2012, 11:45AM

All websites are operating normally.

Dec 9, 2012, 11:40PM

There was an interruption of service between the hours of 7:15PM and 11:40PM (PST) on Dec 9, 2012. The load balancer responsible for distributing traffic across our webservers failed. We are taking steps to insure that such a failure will not result in an interruption of service in the future.

Dec 9, 2012, 11:40PM

Status Legend

– Issue resolved or maintenance complete.

– Still working on what the issue might be.

– Having issues with an aspect of the network and actively working on to repair it.

– Announcing the start of planned maintenance.

– Notice: We are announcing important information regarding the network.