GoDaddy Outage: Anonymous Attack Or IT Failure?

If hacktivists weren't behind the six-hour outage, as GoDaddy's CEO contends, they may still have taken advantage of the situation.

What's worse for a website hosting company: getting taken down by hackers, or failing to properly configure your network, sparking downtime and lost revenue for customers?

The CEO of website hosting service Go Daddy has said that the company's six-hour outage Monday had nothing to do with a hacktivist, despite a hacker having claimed credit for launching a distributed denial-of-service attack (DDoS) that scuttled the Go Daddy network.

"The service outage was not caused by external influences. It was not a 'hack' and it was not a denial of service attack (DDoS). We have determined the service outage was due to a series of internal network events that corrupted router data tables," said Go Daddy CEO Scott Wagner, in a statement. "At no time was any customer data at risk or were any of our systems compromised."

Wagner apologized to Go Daddy's customers for providing less than "99.999% uptime in our DNS infrastructure," and said the company was working to prevent a recurrence. "Once the issues were identified, we took corrective actions to restore services for our customers and GoDaddy.com. We have implemented measures to prevent this from occurring again."

In the wake of the outage, the CEO of one Go Daddy customer, RunningShoes.com, told United Press International that the downtime had been "devastating" for his company, resulting in up to $50,000 in lost sales. He said he was weighing moving his company's 10 retail websites to another hosting provider.

"As this GoDaddy outage reveals, misconfigured network devices and improper changes can be just as dangerous to the stability of our networks as the latest attacks," said Sam Erdheim, director of network security strategy for network security management company AlgoSec, in an emailed statement. "Organizations should take a step back to ensure its processes are in order and its devices are securely configured to avoid these situations in the future."

The GoDaddy hacking denial sees the credibility of Anonymous--which rarely takes credit for attacks it doesn't commit--stretched thin, especially after its AntiSec arm claimed that it had stolen one million Apple UDIDs from the laptop of an FBI agent. But that breach was in fact later traced to a Florida-based app publisher called BlueToad, and had nothing to do with the bureau.

In the wake of Wagner's statement, however, the hacktivist collective Anonymous has distanced itself from claims that one of its number had launched a DDoS attack against Go Daddy. In a statement Tuesday, released via Pastebin--and distributed via the AnonOpsLegion Twitter account--the group admitted that it was unclear whether Go Daddy had been taken down by one of its number or not, although it attempted to spin the outage anyway. "Many of us have concluded that Go Daddy was taken down because of its support for SOPA, the 'Stop Online Piracy Act,'" it said, and called on "the ninety nine percent to boycott Go Daddy and remove (sic) there (sic) hosting to another domain name servers (sic)."

But in a Tuesday Twitter post, Anonymous Own3r, the self-described "security leader of Anonymous," claimed to have broken into a Go Daddy website database and obtained source code, which he claimed to have shared via file-sharing networks. In a Pastebin post, Anonymous Own3r said he'd found 53 SQL injection flaws on the Go Daddy website, which he'd been able to use to gain access to the site, apparently following the network outage.

Those claims couldn't be verified, as a link to the stolen data--hosted at the ISA filehost website--returned an error message, saying the uploaded file had been deleted. Anonymous Own3r also released an image--although it could easily have been a doctored screenshot--purporting to show the "About Go Daddy" Web page having been defaced with the words "Hacked by Own3r." Notably, however, the hacker didn't claim that he'd accessed production systems, or attempted to launch a DDoS attack against the Go Daddy network.

I was at the godaddy website registering a domain name, there was an alert on the website explaining that there would be some connection issues,,,,they even posted what times there may be issues. So if they knew when there would be issues I doubt that it was a hacker,,,unless he was nice enough to notify them of when he was going to hack them.

Here is a comment from a real customer...me! I am a godaddy customer and 2 days before they went down, my unused VPS (server) was hit with a huge spike of traffic (from 200meg normal bandwith to 6 terabytes in 1day) I called the morning, coincidenally, the day that Godaddy crashed, and tech support told me i was hacked by someone sending massive traffic to the server and to take certain steps to fix it, which i could not because then godaddy went down. Today, i got a $6250 charge on my credit card for overage and cannot get anyone in their tech support to acknowledge the anomoly of the huge spike or the timing of this issue. like talking to wall, they just keep saying "check your logs". so for me, their offer for one month free is not going to touch the $6250 charge for an unused server that i pay $50/month for....and..sort of a strange coincidence, right?

I am very sorry to hear about this frustrating experience. Although I am not comprised of the details of your specific situation, I would like to invite you to give our billing team a call so that we can work with you t get this resolved.

So, basically, what GoDaddy is saying at this point is that a failure in their change management system led to a systemic failure of their entire infrastructure. Don't they test things before they put them into a production environment?

Now, since GoDaddy themselves are not a publicly traded company, they don't specifically have to deal with regulations like Sarbanes-Oxley... but the group of investment firms that holds a majority stake in GoDaddy includes KKR Capstone (a publicly traded company).

Maybe it's the conspiracy theorist in me, but is it possible that this was an inside job? Think about it for a minute - you've built out an infrastructure that runs with 5 9s of reliability and has been running that way for a while (prior to this event, when was the last time you heard of a MAJOR outage at GoDaddy?) and suddenly, a corrupted router table takes your entire infrastructure out for 6 hours.

You suffer a major outage that directly affects small and medium sized businesses (those that can least handle instability, especially if they are using their site as a direct e-commerce site that engages customers as well as their ordering/logistics systems). Those customers go elsewhere to "more safe and secure" facilities for domain registration, DNS and hosting. You begin to lose revenue. Your organization begins to lose value.

Meanwhile, roughly 14 months ago, 65% of your organization was purchased by three investment firms for 2.25B USD. So, the value of the firm drops just far enough for those three venture capital firms to offer to buy out the rest of the organization (this would be the next shoe to drop, as it were).

You have an organization that's been doing well - winning awards - but also enjoying a lot of bad press (but since when is bad press really all that bad these days?) and an organization that is in a bit of upheaval since your last CEO spent 12 months at the top and decided to bail out - and the previous CEO is still around in the executive committee.

I'm thinking that maybe this was a way of someone to manipulate the worth of the organization in a way that another organization can purchase a majority stake, "clean house" and move more into the cloud computing arena - GoDaddy has already been taking steps in doing just that, but the opportunity to revamp and distance itself from its own history may end up being good for business in the long run as compared to this "blip on the radar" in the short term.

As far as the whole Anonymous tie-in, that has to do with being at the right place, at the right time, and using social media to make claims that they can't back up. After all, everything you read on Twitter is legitimate, right?

Granted, I could be quite wrong here... but what if I'm not? At any rate, get your popcorn ready - the main feature is about to begin...

The noted SQL injection attach is incorrect. The supplied screen shot image is cross-site-scripting (XSS) which in most cases only affects the user performing the XSS (e.g. session stealing/logging). Have a look at the address bar in the screen shot. The screen shot alone screams script-kiddie not black hatter. Anybody can scan a website for SQL or XSS with free software. Taking down routers in a protected network would require real skill.

Published: 2015-03-31The build_index_from_tree function in index.py in Dulwich before 0.9.9 allows remote attackers to execute arbitrary code via a commit with a directory path starting with .git/, which is not properly handled when checking out a working tree.