There's no evidence to support claims the outage is caused by Anonymous hackers.

GoDaddy, one of the Internet's biggest webhosting providers, experienced technical difficulties on Monday that prevented many people from visiting sites that relied on the service for connectivity.

A little after 5 p.m. California time on Monday, company officials took to Twitter to say the outage was mostly resolved.

"Most customer hosted sites back online," the dispatch stated. "We're working out the last few kinks for our site & control centers. No customer data compromised."

Searches using the dig utility also indicated domain name system servers operated by GoDaddy competitor VeriSign were being used to resolve the godaddy.com domain name. Wired reported VeriSign was hosting GoDaddy's DNS servers, too, although Ars couldn't independently verify that. One reason GoDaddy might turn to VeriSign is to avail itself of functional domain-name-resolution services. VeriSign operates a denial-of-service mitigation service called iDefense, so another possibility is that GoDaddy is under attack and is turning to VeriSign for help blocking it.

Monday evening's update from GoDaddy came after it spent much of the day combating an outage that affected virtually every domain it hosted.

"We are aware of an issue affecting several services, including e-mail, our website and some customer websites," company officials wrote in a status update posted early Monday afternoon. "We understand your frustration. We want you to know that our team is investigating the source of the issue and is working to resolve it as quickly as possible."

"Update: Still working on it, but we're making progress," GoDaddy representatives wrote in a separate Twitter dispatch. "Some service has already been restored. Stick with us."

Anecdotal evidence suggests the outages aren't universal. GoDaddy's homepage, as well as GoDaddy-hosted sites such as tuCloud.com were inaccessible on Monday afternoon for two Ars reporters. One reporter was located in the UK and was using a custom domain name system lookup server, while the other was located on the United States east coast and used DNS servers provided by ISP RCN. Both of those sites worked fine for a separate Ars reporter located in California who was using OpenDNS, a free service that translates human-readable domain names into the IP addresses relied on by computers that route Internet traffic. There were widespread reports on Twitter and elsewhere from other users who also reported trouble reaching GoDaddy-hosted sites.

"We have a lot of customers who want to know why their whole internet world is down and yet their equipment is running just fine," Tony Kapela, vice president of data center and network services at managed hosting provider 5Nines.com, told Ars.

Shortly after this article was published, some people who previously reported site outages said they were no longer experiencing outages. Others reported precisely the opposite.

At publication time, there was no evidence to support claims that the outage was the result of a denial of service attack. The unverified claims came from someone who identified himself as a member of the Anonymous hacking collective. The North American Network Operators Group has yet to provide any evidence supporting or refuting claims a denial-of-service attack was responsible.

60 Reader Comments

I have a client whose service is unavailable (newly registered domain name and DNS hosting). It's definitely a DNS problem. I can't stand godaddy but I never thought reliability would be an issue. I'm eager to learn who broke what...

That's a good point - until GoDaddy releases more information there's no way to know for certain what is causing the outage, though it's hard to imagine anything but a large DoS causing this situation. Anything physical or accidental should be confined to a small subset of their system.

Yea, something's going on with DNS. Here at work I can't dig it, but on my vps I get all the records just fine. I literally just migrated some friends from Godaddy email to Google Apps for Domains yesterday. Even if not 100% of the DNS for Godaddy is working, probably most of the larger services will have it cached from the update(s) that were done last night.

Sites load for some people and not others because some intermediary client-side nameservers cache the results longer than others. I can verify that GoDaddy's DNS services were down, though, because I had MULTIPLE customers' sites go down, and when first doing a whois on their domain to see the authoritative nameservers (NS??.DOMAINCONTROL.COM and NS??.DOMAINCONTROL.COM, with the two ??'s being numerics that are different for different clients) and then doing a dig @ns??domaincontrol.com domainname.tld, you'd get either a timeout or a SERVFAIL.

GoDaddy's homepage, as well as GoDaddy-hosted sites such as tuCloud.com were inaccessible on Monday afternoon for two users in the UK and on the United States East Coast

So, in the UK the GoDaddy outage is only affecting TWO users? I knew GoDaddy was a dying company but the fact that only two people in the UK seem to be having problems show that it's no longer a company being used by people to host their websites*

Thanks for explaining what the problem is, I was wondering why Coursera was down. I used the OpenDNS CacheCheck (http://www.opendns.com/support/cache/) to get an ip that works and modified my hosts file accordingly.

A CacheCheck for coursera.org shows Miami, USA and London, England as unresponsive (which is a problem for me, being in the UK).

My primary domain uses GoDaddy's TotalControl DNS. I couldn't get e-mails when I first read this article, but I JUST NOW got one. nslookup kept timing out on my phone (VZW). Hmm...

Seems like it's a combination of DNS and the actual hosting service. I subscribe to a 3rd party monitoring service for my godaddy-hosted site, and about half the alerts have been name resolution failures (DNS) while the other half are not responding to ping (hosting).

GoDaddy's definitely got some explaining to do about their (lack of) DNS redundancy. I'd much rather pay a reliable 3rd party DNS host than get an unreliable one for free. I would have thought that such a major hosting provider would have geodiverse, bulletproof DNS servers, and I'm definitely surprised and disappointed that that's apparently not the case.

The issue was their NS as everyone else is saying. All three of their authoritative servers were unavailable for most of the day. Users that had long cache times didn't seem to have an issue, but in the testing process I flushed the cache on my DNS server and poof! no more resolution. Unfortunately, my CEO forces me to use GoDaddy for registration, DNS, and Virtual Server hosting. So we were down all afternoon.

According to TechCrunch, the "hack" came from an Anon member named Anonymous Own3r, not the Anon collective in general. I guess we'll find out soon enough.

GoDaddy's definitely got some explaining to do about their (lack of) DNS redundancy. I'd much rather pay a reliable 3rd party DNS host than get an unreliable one for free. I would have thought that such a major hosting provider would have geodiverse, bulletproof DNS servers, and I'm definitely surprised and disappointed that that's apparently not the case.

I hear you on the redundancy, but this is the first time since maybe June 2005 when I switched everything from the ISP I used to work for to TotalControl + Google Apps that I've had a big problem with my DNS hosting. It looks like we're getting the babycenter.com spam again (thanks, dear...), so we're probably good to go.

My primary domain uses GoDaddy's TotalControl DNS. I couldn't get e-mails when I first read this article, but I JUST NOW got one. nslookup kept timing out on my phone (VZW). Hmm...

Seems like it's a combination of DNS and the actual hosting service. I subscribe to a 3rd party monitoring service for my godaddy-hosted site, and about half the alerts have been name resolution failures (DNS) while the other half are not responding to ping (hosting).

GoDaddy's definitely got some explaining to do about their (lack of) DNS redundancy. I'd much rather pay a reliable 3rd party DNS host than get an unreliable one for free. I would have thought that such a major hosting provider would have geodiverse, bulletproof DNS servers, and I'm definitely surprised and disappointed that that's apparently not the case.

What made you think that? GoDaddy is notoriously as well as conspicuously low-rent in everything they do, and there's no such thing as "bulletproof DNS servers" anyway.

May I suggest updating the article again to make it clearer what exactly isn't working. The whole paragraph about two people from the UK and someone in California is very confusing. To be honest the entire article just seems to wander about and not really say anything. I learned more from the first couple of comments.

I have a client whose service is unavailable (newly registered domain name and DNS hosting). It's definitely a DNS problem. I can't stand godaddy but I never thought reliability would be an issue. I'm eager to learn who broke what...

Yeah. I use other DNS service for one of my sites and that one was available. Others were not and they're on the same server.

My primary domain uses GoDaddy's TotalControl DNS. I couldn't get e-mails when I first read this article, but I JUST NOW got one. nslookup kept timing out on my phone (VZW). Hmm...

Seems like it's a combination of DNS and the actual hosting service. I subscribe to a 3rd party monitoring service for my godaddy-hosted site, and about half the alerts have been name resolution failures (DNS) while the other half are not responding to ping (hosting).

GoDaddy's definitely got some explaining to do about their (lack of) DNS redundancy. I'd much rather pay a reliable 3rd party DNS host than get an unreliable one for free. I would have thought that such a major hosting provider would have geodiverse, bulletproof DNS servers, and I'm definitely surprised and disappointed that that's apparently not the case.

What made you think that? GoDaddy is notoriously as well as conspicuously low-rent in everything they do, and there's no such thing as "bulletproof DNS servers" anyway.

I guess there's no arguing with the idea that nothing is bulletproof. However, I'm older than the hills, have been playing with the Internet since the days of the AOL floppy disk, and managed a web site for over 13 years, and this is the first time I remember anyone's DNS being totally down for a period of time. DNS problems with individual sites, yes, but I don't ever remember anyone's whole service being out. I think both of our arguments - that it shouldn't be down and that nothing is bulletproof - are supported by the fact that sites have (supposedly) redundant servers. Why have two servers if a single problem can bring them both down?

My primary domain uses GoDaddy's TotalControl DNS. I couldn't get e-mails when I first read this article, but I JUST NOW got one. nslookup kept timing out on my phone (VZW). Hmm...

Seems like it's a combination of DNS and the actual hosting service. I subscribe to a 3rd party monitoring service for my godaddy-hosted site, and about half the alerts have been name resolution failures (DNS) while the other half are not responding to ping (hosting).

GoDaddy's definitely got some explaining to do about their (lack of) DNS redundancy. I'd much rather pay a reliable 3rd party DNS host than get an unreliable one for free. I would have thought that such a major hosting provider would have geodiverse, bulletproof DNS servers, and I'm definitely surprised and disappointed that that's apparently not the case.

What made you think that? GoDaddy is notoriously as well as conspicuously low-rent in everything they do, and there's no such thing as "bulletproof DNS servers" anyway.

I guess there's no arguing with the idea that nothing is bulletproof. However, I'm older than the hills, have been playing with the Internet since the days of the AOL floppy disk, and managed a web site for over 13 years, and this is the first time I remember anyone's DNS being totally down for a period of time. DNS problems with individual sites, yes, but I don't ever remember anyone's whole service being out. I think both of our arguments - that it shouldn't be down and that nothing is bulletproof - are supported by the fact that sites have (supposedly) redundant servers. Why have two servers if a single problem can bring them both down?

Well...GoDaddy sucks, obviously.

Their default configuration for providing DNS to their customers were apparently pretty vulnerable to simultaneous attack. They probably figured hey, they're geographically separate and/or using redundant internet links, etc, and called it a day.

Their default configuration for providing DNS to their customers were apparently pretty vulnerable to simultaneous attack. They probably figured hey, they're geographically separate and/or using redundant internet links, etc, and called it a day.

Maybe they'll learn from their mistakes.

I doubt it.

If it were a targeted attack on Godaddy's nameservers. It's unconfirmed at this point but it sort of fits. AntiSec and the like don't really wield the sort of DDoS muscle it would take to slam Godaddy's nameservers - but say they used BTC to buy time on a botnet and ran a DNS-specific targeted exploit.. A lot of BTC was stolen recently from an intermediary company. Whatever the hell is going on I doubt it could happen to Cloudflare, and that should say something about Godaddy.

[snip] I think both of our arguments - that it shouldn't be down and that nothing is bulletproof - are supported by the fact that sites have (supposedly) redundant servers. Why have two servers if a single problem can bring them both down?

I run a small but mighty web site. DNS is supplied by two different organizations, both with geographically diverse DNS servers. So, there are (at least) six different DNS servers, five in the U.S. and one in Canada. I do not worry about hurricanes in Miami, ice storms in Canada, or tornadoes in Kansas because it would take six separate natural disasters to take out all of my DNS.

Someone who didn't like me, and who had sufficient horsepower, could launch denial of service attacks against six IP numbers and potentially cripple my DNS. Even anycast DNS, which I have, doesn't help a bunch; it simply means that the 'bots running the DoS attack have to be as geographically diverse as my DNS servers.

OTOH, there are rumors (but no checkable facts) that this was an upgrade gone wrong. That's easier to believe than a massive DDoS attack.

It's more than DNS, I can't access certain web sites that only have an SSL cert from GoDaddy for https access. It appears the root certificate can't be verified in some browsers.

That would be your browser attempting to double-check that it hasn't been revoked as compromised since the last time it checked. Browsers use to all "fail open" when they couldn't reach the mothership to perform such checks, but it's been pointed out, repeatedly, that this all but defeats the point of revocations in the first place. There's probably a knob for it if this happens again but don't forget to toggle it back afterwards.

(Which just reminds us all that SSL is effectively broken because it's so dang unusable when enforced.)

I'm not sure how much it would take - they use A records in the form NS??.DOMAINCONTROL.COM where the ?? is 01-78, but that only resolves to 39 pairs of IP addresses. Their DNS hosting gives you a pair of nameservers, one in the 216.69.185.x block and one in the 208.109.255.x block (last octet is the same for each in the pair).

Of course it may be more complex than that behind load balancers... or it might be simpler than that behind load balancers; I don't know what their routes look like. Either way it doesn't really suggest a particularly monstrous amount of infrastructure given how many domains (n*10^5? n*10^6? n*10^7?) they host DNS for.

here's a question for people: If you are without DNS service for eight or so hours, would you rather the cause be a) a massive DDoS attack, or b) a massive software screwup on the part of the provider?

I work for a company that interfaces directly with GoDaddy. I can confirm from numerous sources that not only was this a DDoS attack, it was a MASSIVE DDoS attack that came in from multiple directions.

The DNS servers were taken down, the customer DB servers were taken down and GoDaddy HQ internet connection (for the GD employees) was also taken down. I am in Iowa a short distance from their main location and I can verify that they had a complete blackout most of the day.

This was a very deliberate and coordinated attack.

In addition to GoDaddy's servers, our personal servers and various servers of other clients and partners who use some of GoDaddy's services were also attacked.

I can't release details, but I can say that there is a TON of blatantly obvious evidence that this was intentional and malicious.

[snip]I can't release details, but I can say that there is a TON of blatantly obvious evidence that this was intentional and malicious.

Now, c'mon! That's exactly what "AnonymousOwn3r" said.

You've been around Ars for 2-1/2 years, so you have more cred than that, but in the absence of evidence, which you call "details," this could still have been a giant software screw-up on GoDaddy's part.

The Ars article (now updated) and Wired report that Verisign/NetSol took over DNS from GoDaddy. If that is true, I am really impressed with both companies. I'd guess that couldn't happen so quickly without a previous agreement and plan. (Although if someone offered me a hundred sacks of Galleons, I might find that I could move pretty fast.)

If that was true earlier, it doesn't seem to be at the moment, which is 23:00 Eastern. The one GoDaddy domain I've checked currently has authoritative servers of NS17 and NS18.domaincontrol.com. NS17 is at 216.69.185.9 and NS18 is at 208.109.255.9, as expected for GoDaddy. NetSol's BGP could be announcing routes to those addresses, but I don't think so.

A tracert to NS17 grundles off to Chicago, hops on Level3, and zooms directly to what appears to be GoDaddy.