Quake and IT: Shaking in Tokyo, waiting in San Diego

Patrick Thobideau |
March 11, 2011

No warning for Tokyo, but in the U.S., IT had time to prepare

FRAMINGHAM, 11 MARCH 2011 - When the earthquake struck in Japan mid-afternoon, Jason Park was in his office in Tokyo on the 39th floor. It started with a mild tremor, something that happens every few months.

"We didn't think anything of it until the tremor didn't stop," said Park, who is general manager of Japan operations for Appirio, a San Francisco-based cloud services firm.

"We saw other high-rise offices and apartments visibly shaking -- it was hard to stand without losing my balance," said Park, in an e-mail response to questions.

The building's elevators shut down, as did trains and subways. Half of his 30-member team walked home, making treks of five to 10 miles, while others waited for the trains to reopen.

Tokyo is about 150 miles away from earthquake's epicenter, and Park said there was minimal structural damage in the city. But certain portions of the Tokyo metro area experienced power outages.

From a communications standpoint, this is the situation Park said he faced: Cell networks weren't working, but land lines were; and 3G-based Web browsing was slower than usual, but worked.

The fiber Internet in his office was up, and his mobile card made a connection. Google Apps, Park's e-mail system, was also up, and there were no problems reaching Facebook and Twitter.

Amazon's newly opened Japan Web Services data center continued to operate, said Park. That was confirmed independently by another source who spoke on background.

In the U.S., the problem was much different.

The tsunami warnings gave Elysia Everett, deputy CIO of the international law firm DLA Piper, time to prepare. She received her first notification of a tsunami threat at 7:30 Eastern. This firm, with 3,500 attorneys, runs its primary data center in Baltimore and a secondary facility in San Diego.

Everett began waking up IT staff in San Diego (4:30 a.m. PST) to begin planning. At that point, all she knew was that there was a threat of a tsunami, and the San Diego data center was located downtown, not all that far from the bay.

The major concern was the loss of telecommunications services in San Diego. Before too long, everyone was assembled on a conference call to review the firm's disaster recovery plan.

During the 30-minute call, the apps and services were reviewed and the status of backup and replications all verified. Other steps included analysis of the storage environment on the East Coast to ensure that there would be enough space available to bring critical applications hosted on the West Coast live from the East Coast data center. In some cases, it would involve a restore of applications from backups, Everett said.

The real question was when the tsunami would hit, and what event would trigger DLA Piper's switch to the East Coast data center. Making the transition would involve some short interruptions in application access, but not in providing data access. Everett said they could still get critical data to the attorneys.

The triggering event for a switch-over might have been either the loss of connectivity, "or watching huge waves roll into San Diego on the news," she said. But neither happened, and as hours passed, so did the threat.

Japan provides something of a preview of what the West Coast may see one day.

Thomas Jordan, director of the Southern California Earthquake Center, has been using the U.S. government's most powerful supercomputers to examine earthquake probabilities and the damage they may cause.

The center has estimated a 99% probability of magnitude 6.7 or larger quake during the next 30 years in California. But Jordan told Computerworld today that high-performance computing systems don't have the ability to predict such an event, though center is working on systems to provide better forecasting.

There weren't any indications that Japan should expect this massive earthquake. A 7.2 magnitude earthquake in Japan two days ago is now considered a foreshock, but had it not been for the subsequent 8.9 magnitude quake, the earlier quake would have been considered a regular seismic event, Jordan said.

Jordan, however, said Japan was "locked and loaded" for a major earthquake, which is how he has also characterized the U.S. West Coast.