How was your launch day?

Well, Launch Day is over. Thank goodness. For me, the day started VERY early, with great fun, and ended well before midnight with great frustration. So I suppose on average it was an adequate launch.

After grabbing a couple hours sleep Wednesday night, I got up around 2 AM local time, which is the same as my server time, for the scheduled 3 AM launch. Brewed coffee, logged in, jumped on to Vent to gab with sleepy guildies, found Khadgar, parked myself next to him, and waited. The launch popped early, around 2:45, and I easily jumped through the portal.

For the most part, I thought the intro quests went pretty smoothly. There was some lag in a few areas, and some people did get dc’ed but were able to get right back on, and there was the inevitable annoyance owing to the mix of too many players and too few mobs, but in general it seemed to be a good launch.

Got off the ship in Draenor, started leveling, and for a few hours it was terrific fun. Yes, my hunter one-shotted everything so the actual game play was not very challenging, but I was having a great time running around to different areas, picking up quests, seeing story lines unfold, watching my XP bar fill up. (I’m pretty sure when I start to level some of my alts they will have a bit more trouble with this, as they are not nearly so well geared as my hunter.)

And garrisons? I am really liking the idea so far. I started mine as early as I could, quickly launched my first followers on missions, and built as many buildings as possible. I had a garrison plan going in, so I am sticking with that one at least until there is a reason to amend it. My only complaint about garrisons is that I am spending so much time developing mine that I am neglecting my own leveling!

So first half of the day I call an unqualified success for Blizzard. I logged off about noon to run some errands, hit the gym, and grab a nap.

I logged on again late afternoon to find everything had gone to hell.

No reason to catalogue all the problems, they have been described in detail if not very imaginatively by thousands of irate forum posters. To its credit, Blizz is admitting huge problems and seems to be doing everything it can to put band-aids on them. Along with all the other problems, there may have been a denial of service attack. The servers are all down as I write this, for much needed unscheduled maintenance. Fingers crossed that it works.

The baseline problem, so says Blizz, was too many players active in the same zones, compounded by the entire garrison phase mechanism. When peak play time hit in North America later in the day, it all snowballed into a gigantic unanticipated — and disastrous — failure. (And certainly if there was a denial of service attack at the same time it did not improve the situation.)

Now, I work in IT, and these vastly complex systems are, well, unimaginably so. To call them finicky is like calling that North Korean nut job “eccentric.” Some very respectable studies have shown Chaos Theory to be effective for predicting the behavior of some of the largest networks. The equivalent of a digital butterfly flapping its wings somewhere in the system can result in a network tsunami. So I get it, Blizz: this is hard.

But I still have this nagging suspicion that Blizzard failed to plan adequately for this rollout. They have been planning this expansion for AT LEAST two years. They have the disastrous examples of previous launches (the helicopter quest in Mists immediately comes to mind). They have spent the last couple of months hyping the hell out of this launch, doing everything possible to entice fallen-away players to come back. They have had a year of testing the interplay of zones and garrisons. They deliberately set up quest lines such that everyone pretty much needs to do the same ones in the same sequence in order to level. Presumably they have people who can read world clocks and figure out when players get home from work in North America.

So Blizz, do you really mean to stand there with a straight face and tell us you were surprised and unprepared for lots of people playing in the same zones at the same time????

To me, this is a perfect example of Blizzard’s Achilles heel: They simply do not have the demonstrated ability to carry out complex long-term planning at the macro level. They are very good at doing discrete pieces of it — I cannot even begin to imagine how complex the individual parts of the gaming software are, for example. But they can’t put it all together. This is not a failure of developers or of tech support or of the server and network team. This is a failure of executive management. It seems clear that no one was doing top level management of the entire WoD planning and implementation. No one asked the question “How are we load testing the whole zone versus garrison phasing interplay?” “What other major changes we have made could scale badly and crumble under heavy loads?” No one posed a series of what-ifs:

How will our recent server mergers affect this rollout?

What if 90% of players show up on our heaviest servers? How about 75%? 50%? How many can we accommodate and what is our gating plan to ensure we keep a manageable number?

We’ve hyped this a lot, what is our plan if some hackers looking for glory hit us on launch day?

What have historically been our heaviest play periods on launch day and what is the plan to accommodate those fluctuations?

Garrisons are the foundation of game play in this expansion, let me see all the testing we’ve done to ensure their success in a variety of situations.

What team will be on the tech support forums, how will they be tied in to our developers and network experts, what underlying tone and message should they strive for, who is overseeing our responses?

Each department give me your worst case scenario and how you plan to deal with it. Now how can we deal with multiple worst cases at once?

Et cetera.

For goodness sake Blizz, do your business and your customer base a favor and hire some world-class executive project managers for the next expansion.

All in all –if and only if today’s maintenance is fully successful — an adequate xpac rollout. But after 10 years Blizzard’s customers have a right to expect better.