Spamhaus DDoS grows to Internet-threatening size

More than 300 Gb/s of traffic aimed at the anti-spam site's hosting.

Last week, anti-spam organization Spamhaus became the victim of a large denial of service attack, intended to knock it offline and put an end to its spam-blocking service. By using the services of CloudFlare, a company that provides protection and acceleration of any website, Spamhaus was able to weather the storm and stay online with a minimum of service disruptions.

Since then, the attacks have grown to more than 300 Gb/s of flood traffic: a scale that's threatening to clog up the Internet's core infrastructure and make access to the rest of the Internet slow or impossible.

It now seems that the attack is being orchestrated by a Dutch hosting company called CyberBunker. CyberBunker specializes in "anything goes" hosting, using servers in a former nuclear bunker (hence the name). As long as it's not "child porn and anything related to terrorism," CyberBunker will host it. This includes sending spam.

Spamhaus blacklisted CyberBunker earlier in the month. A CyberBunker spokesman, Sven Olaf Kamphuis, told the New York Times that CyberBunker was fighting back against Spamhaus because the anti-spam organization was "abusing [its] influence."

Update: Kamphuis has written on his Facebook page that the NYT has gone for "sensational reporting" and that CyberBunker is not, in fact, responsible for the attacks.

When the attack started, on March 18, it measured around 10 Gb/s. On March 19, it hit 90 Gb/s, on March 22 it reached 120 Gb/s. This still wasn't enough to knock CloudFlare or Spamhaus offline. So the attackers escalated.

Today, CloudFlare wrote that one of the Internet's big bandwidth providers is seeing 300 gigabits per second of traffic related to this attack, making it one of the largest ever reported.

This is bad news for the Internet. 300 Gb/s is the kind of scale that threatens the core routers that join the Internet's disparate networks.

As Ars wrote last week, CloudFlare uses a technique called anycast to distribute traffic to nearby servers. This greatly diffuses the potency of DDoS attacks, by preventing the attackers from focusing their traffic on a single system on the Internet. Instead, the attack traffic all gets directed to a nearby machine—one of CloudFlare's geographically distributed mirrors. A sufficient flood of traffic could still knock one of those local mirrors offline, but the impact of that should be relatively restricted, with users throughout the rest of the world unaffected.

Once an attack has been detected, the companies that CloudFlare buys bandwidth from—known as "Tier 2" providers—can then block the traffic to prevent it from entering their networks. That doesn't stop the problem, however; it just moves it upstream.

Tier 2 providers buy their bandwidth from the small number of Tier 1 providers. Tier 1 providers work a bit differently than Tier 2. They don't buy bandwidth from anyone. Instead, they just connect to other Tier 1 providers for free. These Tier 1 providers are the high-speed backbone that joins all the Tier 2 providers together, and hence makes the Internet a single global network, rather than a bunch of separate networks.

If a Tier 1 provider fails, that risks breaking the entire Internet.

Though the Tier 2 providers are blocking the flood traffic, the Tier 1 providers are still carrying it. As the DDoS attack has grown, so too has this load. The 300 Gb/s figure came from one of these Tier 1 providers. CloudFlare says that several of the Tier 1 networks have started to become congested, particularly in Europe. This congestion can make the entire Internet slower for everyone.

This has been particularly significant in London. Dotted around the globe are a number of "Internet Exchanges" (IXs). These are places where multiple networks from different service providers connect to each other. The London Internet Exchange (LINX), through which an average of about a terabit of traffic passes each second, suffered a substantial outage on March 23. At peak time, its traffic dropped from about 1.5 Tb to around half that.

The LINX team has subsequently changed some aspects of their network configuration to make their systems more robust against this kind of large scale attack, and normal service was resumed a little over an hour after the first attack.

The fundamental problem, however, remains. The traffic is being generated primarily from DNS amplification attacks. Small requests are sent to DNS servers, generating responses from those servers that are about 50-100 times larger. The sending address of these requests are spoofed, so the DNS servers think that they originated not from the attacker's machine but from the victim's machine; accordingly, the large responses are sent to that victim, overwhelming it with traffic.

To perform these attacks, the attackers need servers that are open to anyone (and arguably misconfigured). The Open DNS Resolver Project reports that there are about 25 million of these open DNS servers, and hence 25 million servers that can be used to generate enormous quantities of traffic. Making this worse is the fact that, unlike DDoS attacks using home PCs, these DNS servers typically have fast Internet connections.

The number of open DNS resolvers is dropping—CloudFlare reported that it was down by about 30 percent in February—but they're still abundant, and as the current attacks on SpamHaus make clear, still enough to be tremendously problematic.

To guard against these attacks in future, the open DNS servers need to be reconfigured in some way (to either restrict the IP addresses that can use them, or limit the number of queries they'll respond to, or both), and networks need to be reconfigured so that they won't send traffic with spoofed sender addresses.

Both of these fixes are well-known, and the problems have long been acknowledged. However, they require coordinated action from many parties: every DNS server operator and every ISP needs to do the reconfiguration work.

As for CyberBunker, the company boasts that although "Dutch authorities and the police have made several attempts to enter the bunker by force, none of these attempts were successful." Even a Dutch SWAT team allegedly failed to get in. CyberBunker argues that it is currently engaged in a blackmail war with Spamhaus. As Internet wars go, this one is using the nuclear option, and everyone is at risk of being caught in the blast.

258 Reader Comments

Regardless of whether CyberBunker was 'wronged' by Spamhaus, this is an incredibly disproportionate response. These guys are based out of the Netherlands, not some Eastern Bloc state. Where are the Dutch authorities?!?

Since it is the Wild West and a gun fight. Shoot back. Cloud Flare has the resources to knock CyberBunker off line.

A counter attack would illustrate the problems of cyber warfare for governments and others so inclined, and would have the benefit of showing why a DDOS may not be such a good idea.

Because launching DDOS is a criminal activity, and the people running Cloud Flare probably don't want to go to jail

Depends on the country now, doesn't it? As an example, the US House and Senate are trying to pass a bill that makes DDOS illegal, but such a law does not exist yet. In the Ukraine and the Soviet Union there are no laws that prohibit the activity. There are many other countries that such activity is legal.

Gee... there's an article on Ars this evening about a United States citizen facing ten years in prison for a DoS attack.

In case of huge attacks it causes problems to the whole Internet. Maybe terrorism is not the correct word, but they are surely hurting society in general with this actions.

So is Spamhaus when it blocks anything behind spam. I think that spammers and Spamhaus both are big threat to the society.

If spam levels raise then even when services like Spamhaus are voluntary, everyone or most of are forced to use them. When there is only one such service or when the blocking list is shared then this will become great risk to the freedom of speech.

You clearly don't know what you are talking about. Nobody has or needs to use Spamhaus, most email server administrators do because it works. So why do you blame Spamhaus? Blame the thousands of administrators and even more companies that voluntary use their services.

And no, there is not only one lists. There is Spamcop and allot of other similar services. Spamhaus just happens to be most popular, but RBL lists exists for years from different services.

Society in general does not agree with you. People dislike spam and most will do what they can do stop it.

And in Russia, if you're a spammer and you piss off someone enough, you'll get beaten to death, like what happened to the one guy who was a spammer and pissed off the wrong bunch.

Yes, he was found tied to a chair and beaten to a bloody pulp. Nothing was taken from his apartment otherwise. It was linked to his spamming activities.

Now that's really is interesting. Paraphrasing someone on SlashDot: We think we have state-run media when what we really have is a media-run state. The media, in this case, are leading the internet intelligentsia, and everyone else, toward more control over the 'net.

If the Cyberbunker people are telling the truth about resisting the local authorities, they are in a hell of a lot of trouble and when they emerge from their "secure bunker" (which cannot be totally self-sufficient forever) will be doing serious jail time.

On the bright side, perhaps some owners of internet infrastructure will start getting the message.

According to Cyberbunker, there wasn't much resistance going on.

Police turned up to execute a raid and showing their brilliance, tried to knock down a blast door on a nuclear bunker with a hand-held battering ram. This apparently did so little that no-one inside knew the raid was going on.

After reviewing security footage, the police were contacted to ask what they had been doing, initially denied any involvement, then agreed to pay for damage to a security fence. Is it still resistance if you don't know anyone's trying to arrest you?

So Cyberbunker claims to be a legitimate business, and to prove it they (and/or their shady spammer clients) are using illegal and unethical means to launch an attack against their opponents and that attack that endangers the internet.

Yeah, nice work fuckwits, way to win friends and demonstrate your legitimacy.

I hope these scum sucking parasites of the internet go to jail and their business collapses.

Notice the actual crime: "Damaging a protected computer" and "Conspiracy"?

You need to read the article you mention and not just AssUMe.

I did read it. He instituted a DoS and was charged with "damaging a protected computer." So, sayeth the Justice Department, a DoS is "damaging a protected computer" and so illegal.

You are incorrect. If your server is not properly configured, then you can overload the server and cause it to crash by overloading the buffer. This is problematic with Apache, less problematic with IIS7+, and so far, unknown with Ningx. The same thing can happen if too many users hit your site at the same time and you have improperly configured your server.

On to the other points. The man was charged with damaging a protected computer, not a DDOS. The DDOS was the proximate cause of the damage, but it was the damage and not the DDOS. The conspiracy charge came from the perp asking others to join into the attack.

Since you seem to have doubt as to my knowledge, this one of my websites, and I am the person named in the header. The page I will point you at contains legal discussions in another area. You may judge my legal knowledge for yourself. You may also Google the name there, and you will find more than you like, not limited to my educational qualifications. My suggestion is that you remain within your ken of knowledge and not try to split hairs.http://www.normanhaga.nl/blog/courts/

On to the other points. The man was charged with damaging a protected computer, not a DDOS. The DDOS was the proximate cause of the damage, but it was the damage and not the DDOS. The conspiracy charge came from the perp asking others to join into the attack.

This makes as much sense as saying it's not illegal to punch someone in the face because you'd be charged with assault instead of "illegal face punching".

Quote:

Since you seem to have doubt as to my knowledge, this one of my websites, and I am the person named in the header. The page I will point you at contains legal discussions in another area. You may judge my legal knowledge for yourself. You may also Google the name there, and you will find more than you like, not limited to my educational qualifications. My suggestion is that you remain within your ken of knowledge and not try to split hairs.http://www.normanhaga.nl/blog/courts/

I did google your name. All I see is that you're a convicted burglar who's fighting against mugshot sites because you don't like your mugshot being up. And this makes you an internet law super-expert, how?

On to the other points. The man was charged with damaging a protected computer, not a DDOS. The DDOS was the proximate cause of the damage, but it was the damage and not the DDOS. The conspiracy charge came from the perp asking others to join into the attack.

This makes as much sense as saying it's not illegal to punch someone in the face because you'd be charged with assault instead of "illegal face punching".

Quote:

Since you seem to have doubt as to my knowledge, this one of my websites, and I am the person named in the header. The page I will point you at contains legal discussions in another area. You may judge my legal knowledge for yourself. You may also Google the name there, and you will find more than you like, not limited to my educational qualifications. My suggestion is that you remain within your ken of knowledge and not try to split hairs.http://www.normanhaga.nl/blog/courts/

I did google your name. All I see is that you're a convicted burglar who's fighting against mugshot sites because you don't like your mugshot being up. And this makes you an internet law super-expert, how?

I guess you missed the paralegal status, computer/network security, and the educational background while choosing to focus on negative material that has no basis in fact. In fact, why don't you look for the alleged mugshots at mugshots.com or any other site of your choosing - say like ripoffreports and pissedconsumer; sites that conduct no fact checking. You might even gain a glimmer of understanding of by who and why that junk was done.

This isn't the traditional "Nuclear War Bunker" like SAC's HQ or RAF High Wycombe, It was a relay station. It's not built up to the standards of the main HQ bunkers.

Something of note Nuclear war bunkers are meant to deflect LHB/F of a nuclear device.

Light Heat BlastAnd Fallout.

A very direct penetrator could do some serious damage to it, our ability to pierce bunkers has gone up somewhat over the past few years. By the time this bunker was retired we had just released the first "Bunker Buster" of the modern age. That a SWAT team couldn't get in doesn't surprise me. A MOP. That'll be much much different.

As someone who studied criminology, it saddens me when police step out of their bounds and inflict death and violence when the scenario could have gone a totally different, non-violent way.

The training received by law enforcement is in no way adequate and self investigation by police is meaningless. Same thing goes for investigations by departments. In B.C. there is a new civilian unit to study police excessive force claims but they are hiring ex-cops to be investigators, making it near useless.

The prosecutors must be endowed with an truly independent investigative unit and given the teeth to prosecute wrongful behavior. A system where all prosecutors, like judges (at least in Canada) are appointed would be ideal in preventing prosecutors from being afraid of losing elections or alienating the cops and losing their job.

Law enforcement is there to protect the citizens not to harm them. The thin blue line mentality and code of silence must go away., otherwise nothing will change.

In the Ukraine and the Soviet Union there are no laws that prohibit the activity. There are many other countries that such activity is legal.

FYI: Soviet Union has ceased to exist in 1991.

Pardon me that I am in my 50's, grew up with the Soviet Union, that Russia was a major part of the Soviet Union, that Balkan states still exist, that the .SU domains exist today in addition to the .ru domains, and that the .su and certain .ru TLD's are the most highly sought after by hackers and spammers - next on the list are .ua TLD's.

According to the Open DNS Resolver Project there are 25 million or so problematic open DNS resolvers. If the problem were a single DNS server, I guarantee it would be gone or isolated by now. But it's not. In fact, not everyone agrees that open resolvers are bad. For example, Google operates two open DNS servers. (There are many other, and Google does rate limiting to mitigate amplification attacks, so don't jump on Google's case just yet.)

Regardless of whether CyberBunker was 'wronged' by Spamhaus, this is an incredibly disproportionate response. These guys are based out of the Netherlands, not some Eastern Bloc state. Where are the Dutch authorities?!?

In the Ukraine and the Soviet Union there are no laws that prohibit the activity. There are many other countries that such activity is legal.

FYI: Soviet Union has ceased to exist in 1991.

Pardon me that I am in my 50's, grew up with the Soviet Union, that Russia was a major part of the Soviet Union, that Balkan states still exist, that the .SU domains exist today in addition to the .ru domains, and that the .su and certain .ru TLD's are the most highly sought after by hackers and spammers - next on the list are .ua TLD's.

A little FYI for you.

I am also in my 50's, grew up when the Soviet Union was still in existence, and it has ceased to exist since 1991. When we will be in our 80's, it will still have ceased to exist in 1991, when they hardly would have had laws about prohibiting Ddos attacks.

If the Cyberbunker people are telling the truth about resisting the local authorities, they are in a hell of a lot of trouble and when they emerge from their "secure bunker" (which cannot be totally self-sufficient forever) will be doing serious jail time.

On the bright side, perhaps some owners of internet infrastructure will start getting the message.

According to Cyberbunker, there wasn't much resistance going on.

Police turned up to execute a raid and showing their brilliance, tried to knock down a blast door on a nuclear bunker with a hand-held battering ram. This apparently did so little that no-one inside knew the raid was going on.

After reviewing security footage, the police were contacted to ask what they had been doing, initially denied any involvement, then agreed to pay for damage to a security fence. Is it still resistance if you don't know anyone's trying to arrest you?

I can't believe anyone actually believes the SWAT raid story. It's just such obvious bullshit, for the following reasons:1) a SWAT team would need a court order to enter the facility;2) if they had a court order they wouldn't just leave after the battering ram failed;3) they would instead just blow up the door;4) they would have arrested the people they wanted at home, I don't buy their story of all their employees being constantly locked in, they have personal lives don't they;5) the picture above their article is of a riot team, not a SWAT team;

Their other story about zoning disputes is also complete bullshit for the following reason: they claim to have filed several lawsuits against the government and vice versa, but when I search rechtspraak.nl (which contains a record of all court decisions) nothing turns up for "cyber bunker", "cyberbunker", the name of their manager. A search for their lawyer turns up some lawsuits, but nothing related to cyberbunker.

On a different note: can anyone explain why no evidence of this DDOS shows up on any of the internet traffic stats pages? Check for instance the internet traffic report, or the traffic stats of AMS-IX.

CloudFlare - making the Internet more difficult to use - I have stopped going to sites that use CloudFlare because of how intrusive they are, requiring you to enable JavaScript and jump through hoops. Can anyone prove that this attack isn't a publicity stunt by CloudFlare to promote their services?

In fact, why don't you look for the alleged mugshots at mugshots.com or any other site of your choosing - say like ripoffreports and pissedconsumer; sites that conduct no fact checking.

You know who else does no fact checking? Ars. So why should anyone believe a word out of your mouth? Hint: Telling people they are wrong and then signing off with "/eol" instead of a why does not help.

This article is more a cut and paste from numerous news articles circulating right now.

Before everyone gets too worked up, wondering why no-one cuts the cables into Cyberbunker, check out the PC World article www.pcmag.com/article2/0,2817,2417142,00.asp It may be that Cyberbunker is under attack, not Spamhaus.

In case of huge attacks it causes problems to the whole Internet. Maybe terrorism is not the correct word, but they are surely hurting society in general with this actions.

So is Spamhaus when it blocks anything behind spam. I think that spammers and Spamhaus both are big threat to the society.

If spam levels raise then even when services like Spamhaus are voluntary, everyone or most of are forced to use them. When there is only one such service or when the blocking list is shared then this will become great risk to the freedom of speech.

You clearly don't know what you are talking about. Nobody has or needs to use Spamhaus, most email server administrators do because it works. So why do you blame Spamhaus? Blame the thousands of administrators and even more companies that voluntary use their services.

And no, there is not only one lists. There is Spamcop and allot of other similar services. Spamhaus just happens to be most popular, but RBL lists exists for years from different services.

Society in general does not agree with you. People dislike spam and most will do what they can do stop it.

Please read my comment one more time to understand it.

I told that when pressure from spammers growes then more people start using services like Spamhaus.

When services like this start consolidating their lists (when they already do not do that) then there will be only one list to block somebody.

After certain moment this may start become risk because it would become very easy to block somebody who is not actually spammer.

I told that when pressure from spammers growes then more people start using services like Spamhaus.

When services like this start consolidating their lists (when they already do not do that) then there will be only one list to block somebody.

After certain moment this may start become risk because it would become very easy to block somebody who is not actually spammer.

Did you now understood this possible scenario?

We understood you the first time. And yes, false positives are a known risk of block lists. But your second premise (block lists will converge to a monopoly) is incorrect, therefore your argument is invalid. Perhaps your hair is a bird?

First, there are many other sites that offer spam defense services, including but not limited to block lists. Second, Spamhaus itself offers multiple block lists with differing criteria, allowing users to pick and choose their desired level of sensitivity. Third, block list users (i.e. server admins) can insert their own local rules to override a perceived error in the list.

My ventrilo host has been having sporadic terrible latencies with users from the west or central in the last 10days... My host sits on the MAE-East hub. I wonder if they're collateral damage from this fight.

CloudFlare - making the Internet more difficult to use - I have stopped going to sites that use CloudFlare because of how intrusive they are, requiring you to enable JavaScript and jump through hoops. Can anyone prove that this attack isn't a publicity stunt by CloudFlare to promote their services?

I truly don't understand this remark. CloudFlare is a reverse proxy; you should be able to type in the URL of a domain served by CloudFlare, get an IP from DNS, and that takes you to a geographically nearby CloudFlare server through the use of anycast. No JavaScript. No hoops.