Power Outage Hits SF Co-location Center, Web sites

An underground explosion in downtown San Francisco temporarily cuts power to a major hosting facility.

An explosion beneath a manhole cover on Mission Street in downtown San Francisco July 24 knocked out power, darkened traffic signal lights, and cut service to about 51,000 Pacific Gas and Electric customers in San Francisco and the surrounding area-including a major IT co-location center only a few blocks away.
The blast and outage occurred at about 1:45 p.m. PDT, and electricity flickered on and off as many as five times in downtown office buildings before power was fully restored at about 4 p.m., a PG&E spokesperson said in an interview on KCBS-Channel 5 news.

The official said the source of the power outage was an underground failure and that authorities were still investigating the cause.

The 365 Main data center, a city block-size hosting facility that houses servers for a number of major Web sites, was heavily affected by the outage and immediately switched to backup generator power, a company spokesperson told eWEEK. 365 Main's main line brings in 34,000 volts of electricity.
365 Main hosts Craigslist, C/net.com, Technorati, Typepad, LiveJournal, Yelp, RedEnvelope, SecondLife.com and a portion of Charles Schwab's financial transactions, among other companies. Sun Microsystems also utilizes a portion of 365 Mains facility for its grid utility service.
All of those Web sites went offline for at least a portion of the afternoon due to the outage. eWEEK's offices at Second and Mission streets reported five brief outages within the span of an hour.

Click here to read how the explosion of an 80-year-old underground steam pipe affected life and Internet service in midtown Manhattan.
Ironically, 365 Main issued a press release earlier in the day announcing that it has provided online retailer RedEnvelope with "two years of 100-percent uptime at 365 Main's San Francisco facility ... since moving to 365 Main in July 2005."
Miles Kelly, vice president of marketing at 365 Main, released the following statement to eWEEK just after the outage:
"At 1:45 pm today, there was a major power event in San Francisco that impacted business operations for many San Francisco-based companies. Some customers within the 365 Main facility were temporarily affected by the utility failure. The building is currently 100 percent operational and running on back-up power (generators). 365 Main will continue to operate on back-up generator power until PG&E can confirm that utility power is stable."
Click here to read what Cisco Systems did to resolve a campuswide Wi-Fi outage at Duke University that involved the new Apple iPhone.
Kelly told eWEEK in an interview several months ago that the 365 Main facility has a UPS (uninterruptible power supply), but the outage did bring down a number of Web sites for various lengths of time.
Craigslist's home page ran the following announcement for several hours after the blast: "Craigslist and many other sites are having issues at the colo facility. Please sit tight, and try again later. We are aware of the situation, and the happy Craigslist elves are scurrying to make it better, even now."
Craigslist was back up and running by 8 p.m.
Check out eWEEK.coms for the latest news, views and analysis on servers, switches and networking protocols for the enterprise and small businesses.

Chris Preimesberger was named Editor-in-Chief of Features & Analysis at eWEEK in November 2011. Previously he served eWEEK as Senior Writer, covering a range of IT sectors that include data center systems, cloud computing, storage, virtualization, green IT, e-discovery and IT governance. His blog, Storage Station, is considered a go-to information source. Chris won a national Folio Award for magazine writing in November 2011 for a cover story on Salesforce.com and CEO-founder Marc Benioff, and he has served as a judge for the SIIA Codie Awards since 2005. In previous IT journalism, Chris was a founding editor of both IT Manager's Journal and DevX.com and was managing editor of Software Development magazine. His diverse resume also includes: sportswriter for the Los Angeles Daily News, covering NCAA and NBA basketball, television critic for the Palo Alto Times Tribune, and Sports Information Director at Stanford University. He has served as a correspondent for The Associated Press, covering Stanford and NCAA tournament basketball, since 1983. He has covered a number of major events, including the 1984 Democratic National Convention, a Presidential press conference at the White House in 1993, the Emmy Awards (three times), two Rose Bowls, the Fiesta Bowl, several NCAA men's and women's basketball tournaments, a Formula One Grand Prix auto race, a heavyweight boxing championship bout (Ali vs. Spinks, 1978), and the 1985 Super Bowl. A 1975 graduate of Pepperdine University in Malibu, Calif., Chris has won more than a dozen regional and national awards for his work. He and his wife, Rebecca, have four children and reside in Redwood City, Calif.Follow on Twitter: editingwhiz