KeithKanneg schrieb:

Hey everyone,

I want to start by apologizing for the unexpected and prolonged downtime today. This isn’t something we ever want to happen, but when it does, we take it seriously and do everything we can to resolve the issue or issues. We are adding additional processes to try and prevent this particular situation from happening again. Here’s essentially what took place today and what’s behind the long downtime:

What happened? Wasn’t this tested?
The issues we encountered were fairly complex, and having more than one just made things complicated. The first problem was the patch for our BitRaider and SSN launchers were inconsistent and ended up causing different issues after we took the servers out of maintenance. As a result, some of you actually received a game patch, while others didn’t get anything. You’d think we would catch that in our normal tests, but it was an oversight in our process which we have since corrected. The big issue was part of the patch that should have been included was missing data, so it was pulling information from 5.4 instead of 5.4a. The net result was a variety of issues – including some of the patch changes didn’t seem like they were deployed.

Although we extensively test all of our updates at all stages of the process, the missing parts just caused enough oddball problems, we had to take the game down again to figure out what was going on. In reality, had I been a little more patient, we would have caught the issues beforehand, yet the long downtime would not have been avoided. In addition, not everything we had planned to fix is actually included. For example, we have resolved the exploit with the Umbara Stronghold, but it didn’t make it in the 5.4a patch. Therefore, we have our normal infrastructure maintenance planned for next Wednesday, so our current plan is to deploy the change at that time. We implore you to not participate in this exploit as we don’t want any further disruptions in your game play.

Players lost time today, especially in the EU, is anything being done?
Losing playtime is always painful, especially when there is a “double” event running and Dailies awarding crazy high CXP. I’m kicking around a variety of options with the team, as we want to apologize and reward your patience. Some of you have asked to have our Double XP/CXP event extended and others have asked us to keep the current CXP amounts for the Daily Areas. Others have asked for different compensation, and we’re looking at all options as the last couple weeks have been tough on all of us. We’ll lock down what exactly we’re doing and let you know in the coming days.