When the data center at one of the country's biggest co-location facilities experienced an unprecedented power outage, bringing nearly half its customers down with it, the entire industry learned some painful but powerful lessons.

Massive System Failures">

GUARDING AGAINST MASSIVE SYSTEM FAILURES

Although the root cause of the outage is rare, the lessons
gleaned from the experience are useful for data center managers
seeking ways to guard against massive system failures:

Work closely with vendors: If 365 Main's technical staff had
known the meaning of the error code or had had access to
relevant online technical information, they'd have been able
to solve the problem much faster.

Distribute data center resources: If you have the budget,
establish a disaster recovery plan that allows for redundancy.

Communicate immediately and openly: "One of the things
CIOs can learn from our experience on July 24th is that we
communicated as transparently as possible," Kelly says. Sun's
Snow agrees: "Their remediation and open correspondence
on the problem and resolution was adequate."

Be ready to deal with the unexpected: "Sometimes in live
environments things happen that you don't anticipate," Kelly
says. "One hundred percent uptime is not a reality."

Never get complacent: "The extent of the disruption caused
by 365 Main's power outage to the industry as a whole causes
us to realize that, more than ever, a company can never be too
prepared," Snow says.
365 MAIN AT A GLANCE

Doug Bartholomew is a career journalist who has covered information technology for more than 15 years. A former senior editor at IndustryWeek and InformationWeek, his freelance features have appeared in New York magazine and the Los Angeles Times Magazine. He has a B.S. in Journalism from Northwestern University.