Gmail goes down for small number of users, usual panic follows

0.007% of Gmail users affected, but damned if it didn't seem like a lot more.

Google this morning acknowledged service disruptions in Gmail, Google Drive, and Google Docs. Although Google said "less than 0.007 percent of the Google Mail user base" was unable to access Gmail, the outage sure seems like it has been affecting more people than that.

Google listed six affected products: Gmail, Google Drive, Google Documents, Spreadsheets, Presentations, and the Google Apps administration panel for business customers. No problems were reported with Google Calendar and Google Talk. The Gmail disruption was acknowledged by Google just before 9am ET, with acknowledgements of other service problems happening around the same time.

The Gmail, Drive, Docs, Spreadsheets, and Presentations problems were classified as "service disruptions," while the Admin control panel problem was called a "service outage" by Google. The administration panel "issue is affecting a significant subset of users," Google said.

Gmail was the only service for which Google provided a numerical estimate of affected users. Google said last year that Gmail has more than 425 million active users worldwide. With 0.007 percent of users affected, that would be 29,750 people unable to access e-mail.

Empirical evidence suggests that all 29,750 of those people have tweeted about the outage multiple times. The horror was palpable: "I realised that Gmail was down when I hadn't had an email from LinkedIn for twenty minutes," one user said. "[T]his screams Apocalypse to me," said another. One panicked tech writer went so far as to say that the Google Drive outage "remind[s] me why the cloud can be quite a stupid idea."

If you can withstand the graphic nature of these tweets, searching Twitter for "Gmail down" will give you a sense of just what happens when Google services are unavailable to even a small number of users.

I haven't had any problems with Google services myself this morning, although some Ars writers did have trouble accessing Gmail.

For those feeling the pain, it should subside shortly. In its latest update, Google said "Google Mail service has already been restored for some users, and we expect a resolution for all users in the near future." Similar messages of hope were provided regarding Drive and Google Docs. Google is still investigating the admin panel outage and hasn't said when service might be restored.

UPDATE: As of 11 am ET, the Apps Status Dashboard lists Gmail, Drive, Docs, Presentations, and Spreadsheets as being back up. The admin control panel outage continues.

UPDATE 2: The admin control panel outage was resolved shortly after the Gmail, Drive, and Docs disruptions ended.

I have a hard time believing this 0.007% number.In our 3 offices, where we use google apps education edition, about 50 of our 300 users has called in reporting issues.

No knowing how google clusters things what you are reporting could simply be a result of your accounts all living on affected nodes, etc. So google's numbers could be perfectly correct. I personally believe the numbers are correct but only for a specific issue they tracked down, their could be additional unrelated issues they didn't comment on yet.

29K angry technically inclined people (GMail's traditional user base) will make a great deal of noise. Imagine if Ars Technica went down! We'd have the pitchforks and torches out! If reddit went down... let's not discuss that, it's too horrifying to imagine!

29K angry technically inclined people (GMail's traditional user base) will make a great deal of noise. Imagine if Ars Technica went down! We'd have the pitchforks and torches out! If reddit went down... let's not discuss that, it's too horrifying to imagine!

If both were down, that could strongly indicate that something was threatening the integrity of the global DNS infrastructure. And that I can't abide.

I have a hard time believing this 0.007% number.In our 3 offices, where we use google apps education edition, about 50 of our 300 users has called in reporting issues.

No knowing how google clusters things what you are reporting could simply be a result of your accounts all living on affected nodes, etc. So google's numbers could be perfectly correct. I personally believe the numbers are correct but only for a specific issue they tracked down, their could be additional unrelated issues they didn't comment on yet.

I had this happen with Kodak many years ago at my film lab. We had bought a large amount of 11" printing paper rolls, and they were pink. That means hot storage conditions. I called Kodak about that, and was told that we were the only ones who had reported the problem. I called a number of other labs in the city (NYC), and they ALL had that problem, and all were told that they were the only one.

I have a hard time believing this 0.007% number.In our 3 offices, where we use google apps education edition, about 50 of our 300 users has called in reporting issues.

I also have a personal google apps domain for my family, and 1 of the 4 of us have had it there too.

I'd probably go with the 50 out of 300 having a similar issue unrelated to this specific downtime before jumping on this single issue being much larger than Google is saying. Is there a network issue? DNS issue?

Plus if your workplace is like mine in any way, 50 out of 300 actually reporting they can't access email means all 300 can't access the internet at all, they just tried their email first and they're the only 50 to actually check and report it (even that seems high).

On any given day there's probably thousands of people who can't access gmail but it's not instantly google's fault. It's just easy to blame them in this instance because there was a corresponding downtime.

I call BS. Our entire company was offline. That's almost 1000 people right there. Anyone not already autheticated was out of the water. People who were authenticated were hit and miss. I had access to email but couldn't open documents.

This morning, my Google Apps mail seemed to be working fine on my Android device, but login via IMAP was failing. I didn't check via website as I can wait a little for things to start working again. Perhaps the Google Apps authentication issues were part of the noted admin console trouble.

Having seen a lot of these teeny-tiny percentages reported by Google over the years, I have to believe that the 0.007% figure represents the proportion of people who unsuccessfully attempted to log in to the denominator of all registered Gmail users. If that is true, there are quite a few more people who WOULD have been affected if they had tried.

Yep, we are seeing entire offices down so I would say that .007% is creative. I don't know if its the IT nut in me, but whenever I hear about outages like this I always want to know what caused it? Did some server die? Misconfiguration? Network problems?

I wonder if the bean counters consider outages when they outsource IT and services.

Yes they do. Price yourself a clustered Exchange or Zimbra server some day if you want to see what this costs, and then look at the SLA you're able to offer on your own service. Do you want to commit to five nines? Do you want to ask for hundreds of thousands of dollars only to have some unforeseen issue whack it?

Outages happen, no matter who owns the system. Google is actually not that bad versus many other hosted collaboration providers, or your own infrastructure. People are just more sensitive when they can't control it directly.

No dish to Google, but that is one of the drawbacks of single-sourcing online resources. No matter how well-managed a service is, 100% uptime is well nigh impossible, especially with such a large and demanding user base.

Yep, we are seeing entire offices down so I would say that .007% is creative. I don't know if its the IT nut in me, but whenever I hear about outages like this I always want to know what caused it? Did some server die? Misconfiguration? Network problems?

It's often a failure where backup systems don't come on cleanly, a load-balancing or clustering issue and/or scheduled DR and/or upgrades that went awry and backing out made things worse.

Single servers or network problems are relatively easy. Cluster/update/load-balancing issues are evil.

That's completely disingenous crap by Google. They're taking "number of users affected" and dividing by "total number of accounts", including all the inactive or barely-used ones.

To get an actual, honest number, and a better idea of the scale of the problem, the number should be divided by 'number of people trying to reach their mailboxes during the outage'. And I guaran-damn-tee you that is going to be a LOT higher than 0.007%.

I don't know why I still expect better than slimy CYA numbers out of Google. I guess I'm just stupid or something.

This morning, my Google Apps mail seemed to be working fine on my Android device, but login via IMAP was failing. I didn't check via website as I can wait a little for things to start working again. Perhaps the Google Apps authentication issues were part of the noted admin console trouble.

I had the same IMAP issue. Unfortunately the account was in outlook, and outlook couldn't handle gracefully not being able to authenticate. The easiest thing was to delete gmail from outlook. Doesn't bother me much anyway as pretty much all gmail is used for is to text-to-speech voicemail. I have my work cell phone's voicemail routed through google voice. It does a terrible job of conversion, but good enough to get the gist of the message.

One panicked tech writer went so far as to say that the Google Drive outage "remind[s] me why the cloud can be quite a stupid idea."

Only if you implement it stupidly. If your files are synced to your local machine, and your applications work offline, there's no problem even if there's an outage. I never notice when skydrive is down, for instance, because both Office 2013 and the files are available locally. If the cloud goes down and back up repeatedly, this does not impact me at all.

Also, this is something most major cloud providers provide already, so I'm not sure what the big deal is.

I think part of reason this looked bigger than it actually was stems from its impact on *admin* accounts. Admin mail was inaccessible even while many of our users remains online. Naturally, we read more tech sites, write more online, and are more likely to check our e-mail and know that it's Google's problem.