How one site beat back botnets, spammers, and the “4chan party van”

Spam, DDoSes, pizza deliveries, plumber calls—TorGuard has seen it all.

One Sunday late last month, administrators at Orlando, Florida-based TorGuard were in high spirits. They had just successfully rebuffed the latest in a series of increasingly powerful denial-of-service attacks designed to cripple their virtual private networking service. Despite torrents of junk traffic that reached peaks as high as 15Gbps, the admins had neutralized the offensive by locking down the TorGuard servers and then moving them behind the protective services of anti-DoS service CloudFlare.

"This seemed to anger the attackers, however, because on Monday things got a bit more personal," TorGuard administrator Ben Van Pelt told Ars. "Unable to spam, DDoS, hack, or social engineer us, they employed the tactics of the '4chan party van.' Throughout the day our office received multiple unrequested deliveries from local pizza chains, Chinese food, and one large order of sushi. A handful of local electricians and plumbing services were also disappointed to be turned away. To my knowledge no fake calls have been placed to law enforcement yet, however nothing would surprise me at this point."

The two-month-long campaign of harassment and attacks, which Van Pelt suspects was carried out by a competing virtual private networking service, illustrates the lengths some people will go to goad their online adversaries. His experience provides a vivid account of what it’s like to be on the receiving end of a relentless stream of distributed denial-of-service attacks and ultimately what can be done to mitigate them.

10 million e-mail onslaught

The attacks began in late August, shortly after TorGuard announced a promotional campaign that slashed normal fees by 50 percent for both new and existing customers. Within 24 hours, the company's support inbox received torrents of junk e-mails, and not the typical kind that flog male enhancement pills or sham investment proposals, either.

"The messages were spoofed to appear as [if] they were coming from our own support desk while the subject and body were left blank or filled with random gibberish," Van Pelt recalled. Referring to the Simple Mail Transfer Protocol many e-mail systems use, he continued: "The SMTP servers generating the massive onslaught of 10 million daily e-mails were in Argentina and we were unsuccessful in contacting the provider. After a few added rules on [Apache firewall module] mod-security we were successfully blocking the 'mailbomb' attack."

The lull didn't last long. A month later, TorGuard sent out a newsletter notifying customers of new network nodes added in Germany, Iceland, Japan, and Australia that were designed to make connections in those countries faster. Once again, about 24 hours after the e-mail went out, TorGuard came under another paralyzing attack. The 10Gbps waves of traffic appeared to come from PowerStresser.com, AvengeStressor.com, and a handful of other so-called "booter services." They directed the junk traffic only at IP addresses used by the new VPN nodes announced in the newsletter. To Van Pelt, the intent was clear—disrupt TorGuard's stated plan to deliver faster services to new customers.

Enlarge/ A graph showing the bandwidth of TorGuard's OpenVPN server in Germany during its first 24 hours in operation. The drop off at 23:00 is the result of the gateway being rebooted at that time.

TorGuard

"In this particular attack, the sole purpose was to knock services offline by saturating the OpenVPN server's UDP port with invalid requests," he said. "In the VPN business, 'downtime' is a bad word and will cause customers to look elsewhere quickly."

Enlarge/ Bandwidth for the same server as measured by data center operator Level3. Almost identical attacks also hit new TorGuard servers in Australia, Japan, and Iceland.

TorGuard responded by periodically changing the IP addresses used by the targeted nodes. But almost without fail, shortly after a new address was provisioned, it would come under attack. The ability of the attackers to rapidly target new IP addresses led Van Pelt to suspect that they were running the TorGuard service so they could keep track of the internal servers it used. Ultimately, that didn't matter. Van Pelt was able to block the assault by modifying the company's border gateway protocol. The new routes funneled the junk traffic into a virtual black hole rather than to the VPN servers. Once again, Van pelt said, operations returned to normal.

Then, near the middle of October, the service released new proxy software that made it easier for customers to use TorGuard with Vuze, uTorrent, and other BitTorrent programs. Once again, TorGuard found itself under a new tidal wave of junk traffic. This time, the DoS attack came from some two million separate end users, an indication that the attackers were now deploying one or more extremely large botnets of infected computers.

For a while, Van Pelt was able to repel the attack by hardening TorGuard's CSF Firewall, which allows for "stateful packet inspection." That measure augmented the tweaks he had done already using a "DDoS deflate" script, an Apache module known as Mod_evasive, and sysctl to fend off the smaller DoS attacks. Eventually, those measures proved futile as the new round of attacks delivered data floods as high as 15Gbps.

The distributed nature of the attack and the much larger amount of data it delivered once again knocked TorGuard offline, despite the previous tweaks Van Pelt made. It was at this point that he sought the help of anti-DDoS mitigation service CloudFlare. Almost immediately, service was restored.

Thank you for DoSing

With TorGuard back online, Van Pelt and his colleagues were on the receiving end of a rash of phishing e-mails, attempts to brute-force crack their e-mail account passwords, and repeated calls to the company's toll-free support number. When those didn't produce any results, the unsolicited food deliveries and service calls started. Throughout them all, however, the TorGuard service didn't go down. Then, finally, there was silence. The attackers seemed to give up and there have been no significant attacks since.

Van Pelt said he suspects a rival VPN service was behind the attacks. He has no conclusive proof, but he cites this tweet, which he said acknowledged in-progress DoS attacks at a time when there was no public knowledge of them. He also estimated that the attackers may have spent upwards of $7,000 in costs for booter services, SMTP servers, and botnets for hire.

"There is not a doubt in my mind that the perpetrators behind the attacks are from a rival VPN provider," he said. "I believe the targeted nature and costly budget requirements for an attack of this size rule out the notion that someone is doing it just for the 'lulz.' The person or groups involved have obvious interests invested in seeing that TorGuard's operations were interrupted time and time again."

Ultimately, Van Pelt said the attacks have been a boon for TorGuard. Including costs for anti-DoS mitigation and increased bandwidth, the company is paying only $800 more per month now than it was before. In return, he said, the service has been pushed to become much more robust than it otherwise might have been.

"If I could say two words to our attackers, it would be this: 'Thank you,'" he said. "Because of your due diligence, we have performed extensive security audits on our network and I can confidently say we are now ready for anything. Also, please note that our staff prefers healthy foods and doesn't eat pizza or Chinese takeout."

So the time honored tradition of having false deliveries sent to a location means that the "4chan party van" was involved in this? The "4chan party van" is an allegory to the FBI being on your lawn to spy on you in a "Victoria's Flower Shop" all black van with tinted windows.

My apologies on getting my memes confused. Our staff's brief understanding of the "party van" is that they are generally hoax calls to law enforcement targeting an unwilling participant. Hopefully we don't get "swatted" for misusing this term...

So the time honored tradition of having false deliveries sent to a location means that the "4chan party van" was involved in this? The "4chan party van" is an allegory to the FBI being on your lawn to spy on you in a "Victoria's Flower Shop" all black van with tinted windows.

This is a really great article and that ham fisted click baiting almost made me completely disregard everything in it.

And now we move from State-sponsored cyberwarfare to business-sponsored cyberwarfare.

To be honest, the government has made it clear that business-sponsored cyberwarfare is highly illegal. However, the governments position on State-sponsored cyberwarefare is ....Hey Look Over There --> Squirrel Terrorist!!!

I like Jason's Deli. However, they missed the true story: You can get free takeout by running a "vpn" service! I'm going to go set up a "vpn" service and talk about how awesome it is. Then, once they find out they can't hack my service (since it doesn't exist) they'll send me free takeout! That is the definition of winning!

Edit note: This is also the month of Thanksgiving, and there are a lot of homeless folks in DFW. So I'll take the free takeout and hand it out to shelters and people in need too. So I'm not completely evil, just mostly evil

I like Jason's Deli. However, they missed the true story: You can get free takeout by running a "vpn" service! I'm going to go set up a "vpn" service and talk about how awesome it is. Then, once they find out they can't hack my service (since it doesn't exist) they'll send me free takeout! That is the definition of winning!

Edit note: This is also the month of Thanksgiving, and there are a lot of homeless folks in DFW. So I'll take the free takeout and hand it out to shelters and people in need too. So I'm not completely evil, just mostly evil

well someone will have to pay for that takeout. otherwise the delivery guy will just walk away with the food.

"If I could say two words to our attackers, it would be this: 'Thank you,'" he said. "Because of your due diligence, we have performed extensive security audits on our network and I can confidently say we are now ready for anything. Also, please note that our staff prefers healthy foods and doesn't eat pizza or Chinese takeout."

So the time honored tradition of having false deliveries sent to a location means that the "4chan party van" was involved in this? The "4chan party van" is an allegory to the FBI being on your lawn to spy on you in a "Victoria's Flower Shop" all black van with tinted windows.

This is a really great article and that ham fisted click baiting almost made me completely disregard everything in it.

I came here to say this. Even the top image in the article gives a better idea of what the party van is.

So the time honored tradition of having false deliveries sent to a location means that the "4chan party van" was involved in this? The "4chan party van" is an allegory to the FBI being on your lawn to spy on you in a "Victoria's Flower Shop" all black van with tinted windows.

This is a really great article and that ham fisted click baiting almost made me completely disregard everything in it.

I came here to say this. Even the top image in the article gives a better idea of what the party van is.

"4chan party van" was the guy's quote. Yes, he misspoke and was referencing various 4chan lulz tactics rather than "being v&," but the intent is pretty clear. What's clickbaity about using his words? "4chan party van" doesn't strike me as something that would even -get- many people to click on a piece.

I predict that they will in time identify the most likely candidate for the attacks maybe even find the company that did it. I would very much like to see the company exposed and taken to mat big time as an example for others...

I predict that they will in time identify the most likely candidate for the attacks maybe even find the company that did it. I would very much like to see the company exposed and taken to mat big time as an example for others...

Definitely. If nothing else, because that kind of tactics, against a company that doesn't make its living from working with networks and security, could very well shut down business.

If your business tactics are to DDoS your competition into bankruptcy, you deserve room and board with a 250 pound, psychotic cellmate named "Twitchy".

So the time honored tradition of having false deliveries sent to a location means that the "4chan party van" was involved in this? The "4chan party van" is an allegory to the FBI being on your lawn to spy on you in a "Victoria's Flower Shop" all black van with tinted windows.

My apologies on getting my memes confused. Our staff's brief understanding of the "party van" is that they are generally hoax calls to law enforcement targeting an unwilling participant. Hopefully we don't get "swatted" for misusing this term...

"While The gentleman from TorGuard may be speculating that this attack was from another VPN provider. I'd be more inclined to speculate it was from one or more media lobbying entities, (RIAA, MPAA, MPA, IFPI, BPI etc...) with deep enough pockets to hire third parties to do the dirty work. "

There are literally no reports of anyone from the AAs doing this. Since stopping (or slowing down? I haven't hear of any lately) all the lawsuits, they do hire persons to shut down websites selling pirated media by targeting their payment processors, but that appears to be it.

I thought the 4chan party van was a bunch of sweaty fat dudes circle jerking to anime pictures of prepubescent girls who are explained to be 1,000 year old witches/vampires/demons so they don't have to feel bad about it. In a van, I mean.