Salesforce Still 'In Degraded State' After One-Day Outage

Pages

A persistent Salesforce.com outage that wiped out four hours of data customers entered into their CRMs on Tuesday had still not been fully remediated by Thursday afternoon.

While CEO Marc Benioff personally apologized to one customer on Twitter, Salesforce wouldn't comment on how widespread the outage was, or what regions or services were affected by the database failure linked to NA14 -- one of 45 Salesforce cloud instances in North America.

Several Salesforce implementation partners contacted by CRN said they had not been fielding complaints from customers, but speculated some might be dealing with the problems internally. One partner told CRN a single customer -- one in Brooklyn, N.Y. -- was impacted.

But gauging by customer complaints on Twitter and some sources familiar with the industry, the world's largest Software-as-a-Service vendor was grappling with a major outage, one possibly affecting thousands of customers and millions of users.

Salesforce's system status webpage said the performance degradation started at 8:41 a.m. Eastern on Tuesday, followed by a "service disruption" less than an hour later, at 9:31 a.m., and was resolved by 5:30 a.m. Wednesday.

The database error compromised the integrity of files, forcing restoration of NA14 from a backup made four hours earlier. All data written to that instance between 5:53 a.m. and 9:29 a.m. EDT on Tuesday was lost.

The malfunctioning instance became an Internet meme later in the day under the #NA14 hashtag.

The instance "continues to operate in a degraded state," according to Salesforce's status page as of Thursday afternoon. For that reason, Salesforce has suspended some functionality, including weekly exports and sandbox copying.

The status page said the cloud software vendor is performing a root cause analysis, and will share details once available.

One solution provider who works with both Oracle and Salesforce.com and who preferred to remain anonymous told CRN that there will be no long-term impact from the Salesforce.com outage. "Why would someone make a $1 million decision on a four-hour downtime?" the solution provider said.