Business Disaster Recovery

I’ve posted a fewtimes before about Disaster Recovery (DR) as it relates to IT (ITIL calls this IT Service Continuity Management), but this post is more about Business DR. The business must be the driver for all DR projects. Without the business in in the forefront, it is hopeless to attempt developing an IT DR Plan. Truthfully, the Business DR Plan is much harder than the IT DR Plan.

I was once offered a position in a Fortune 500 company as a Business DR Manager. I declined the job.

In the first place, the job is disheartening. Have you ever sat in a room full of people and discussed what options would be available if no one could make it into the office? (Answer: remote access tools) How about if the office was destroyed? (Answer: pre-defined hot site) How about if 10% of the workforce were dead? (Answer: jeez-louise, this is a downer) How about if 50% of the workforce were dead? (Answer: now you’re just being morbid) At what point is it not viable to try and restart the organization? When should you cut your losses and simply call it quits?

These are the questions that the Business Disaster Recovery process tries to answer and it is depressing.

The second reason I declined the job is because it is a no-win situation. You will be the person taking up everyone’s valuable time trying to peg them down on what business processes are critical to their operation, and how long can they go without those processes before the organization as a whole starts to suffer irreparable damage. After you have talked to a lot of people about very depressing subjects and come up with the DR Plan, you will be the scapegoat for whatever bad things happen if the organization ever has to implement the plan (and trust me, bad things will always happen regardless of how perfect the DR Plan is).

Thank you, but I’ll stick to IT Disaster Recovery planning.

Business DR Questions

Questions that must be addressed in the Business DR Plan include:

Safety – Can you safely implement your DR Plan? If you have a DR event, when is it safe to resume normal operations?

Operations – Will the key individuals be available to implement the DR Plan? How long can the business withstand the loss of productivity while in a crippled state? When will you call an end to the event and resume normal operations?

Cost – Every hour the organization is in DR, the cost is astronomical. Was a budget defined in the DR Plan? Does it say how long you can operate in the various DR modes of operation?

Cooperation – Do you have agreements with other organizations that allow you to share resources? What if the other organizations are affected by the same event? How are the resource contentions settled?

Corporate Image – This one is typically managed by the CxO level individuals of the organization, but it is perhaps the most key element to ensure the long term viability of the organization. Let me give you two examples of how the management of corporate image during a disaster event can be handled both poorly and amazingly well.

The Bad

On March 24th, 1989 the Exxon Valdez struck Prince William Sound’s Bligh Reef and spilled 38 million gallons of crude oil. The corporation’s response to the spill was so unbelievably poor that many (myself included) will never purchase gasoline from them again. From the cause of the spill (failure to maintain the collision avoidance system and cutting the number of ship staff in half thereby forcing the remaining employees to work 15 hour shifts) to the company’s response after the spill (dropping oil dispersant on the wrong area and setting off intentional explosions that lead to health problems of native villagers downwind) was so bad as to be laughable. The company then fought against paying for the cleanup effort for the next 20 years in court appeal after court appeal. Sorry Exxon, I will never willingly give you any of my money.

The Good

In 1982, some wacko in Chicago decided it would be fun to kill some people. Before they were finished, seven people had died. The person who did this horrible thing targeted one product. The maker of the product stopped production of the product and then put out a request to have all their product in the pipeline, on retail shelves, and in people’s house be immediately destroyed (at a cost of about 100 million dollars). They did not resume production of their product until they could be (within reason) certain that it was safe. The product was Tylenol, the company was Johnson & Johnson, and the crazy person was someone who thought that putting cyanide in over-the-counter pain medication would be a fun thing to do.

And then the world changed.

Open a bottle of water – Did you hear that clicking? Do you hate having to rip off the cellophane seal around your pre-packed Safeway salad? Before Tylenol was spiked with cyanide in 1982, tamper resistant packaging did not exist. Now it is on everything you might even accidently ingest. When Johnson & Johnson returned to the market with Tylenol packaged in new, tamper resistant packaging a year after this event, their market share went up. Soon everyone else was following suit to the point where tamper resistant packaging is ubiquitous.

Thank You DR Managers

To those of you who step up and take on the role of Business Disaster Recovery Managers, I salute you. Do your job well, plan for all contingencies, and make sure that upper management doesn’t ruin everything by failing to manage the corporate image correctly.