FiOS customer discovers the limits of “unlimited” data: 77TB a month

Average 50TB a month—30,000 percent higher than average—and Verizon will call.

Yes, Virginia, there is a limit to what Verizon will let you do with FiOS' "unlimited" data plan. And a California man discovered that limit when he got a phone call from a Verizon representative wanting to know what, exactly, he was doing to create more than 50 terabytes of traffic on average per month—hitting a peak of 77TB in March alone.

"I have never heard of this happening to anyone," the 27-year-old Californian—who uses the screen name houkouonchi and would prefer not to be identified by name—wrote in a post on DSLreports.com entitled "LOL VZ called me about my bandwidth usage Gotta go Biz." "But I probably use more bandwidth than any FiOS customer in California, so I am not super surprised about this."

Curious about how one person could generate that kind of traffic, Ars reached out to houkouonchi and spoke with him via instant message. As it turns out, he's the ultimate outlier. His problem is more that he's violated Verizon's terms of service than his excessive bandwidth usage. An IT professional who manages a test lab for an Internet storage company, houkouonchi has been providing friends and family a personal VPN, video streaming, and peer-to-peer file service—running a rack of seven servers with 209TB of raw storage in his house.

Just another home network

I asked what exactly he does with that hardware to generate so much traffic. "Lots of stuff," houkouonchi replied. " I do some VPN stuff for people and Web/FTP/SFTP servers. A lot of friends and family stream stuff off me from my huge media collection. And I also do some P2P and Usenet stuff." Most of the storage space is taken up by videos and other media.

He's always had heavy storage requirements—in 2006, houkouonchi said he already had about 8TB of disk with no RAID. "In 2007 I went to 20x 1TB, then not too much later 20x 2TB, and then I started adding more disks and more chassis… but all the chassis that have a lot of hot-swap disk bays are rack-mount. And after having several rackmount machines, I finally decided to just buy a rack when I bought a house and got FiOS back in 2010."

This Verizon FiOS customer's home network generated 77TB of traffic in one month.

Here's what's in houkouonchi's personal data center, from top to bottom:

A 1u server acting as router and VPN server with 4 1.5TB disks.

A 1u testing server with two 1.5TB disks.

A 2u server—formerly a "colo box"—with eight 750GB disks

A 4u Solaris/ZFS backup machine with 24 1TB disks

Another 4u server—houkouonchi's main server with 24 2TB disks and two 3u storage expansion units, each with 15 3TB disks.

A 2u "Windows/miscellaneous" server with eight 1TB disks

Two 2u uninterruptible power supplies

Another 4u Solaris/ZFS server for backups with 24 1TB disks

That's just on premises. Houkouonchi also owns a 2U server running in a colocation facility with 12TB of disk on dual gigabit connections, "which I push quite a bit from as well. It runs game-servers and hosts what used to be the only LA SpeedTest.net server and a bunch of other stuff."

Saving a few bucks

Houkouonchi switched from dual 150 megabit business class connections to a 300 megabit downstream/65 megabit upstream residential plan in January (though he said he was getting 150 megabits upstream). "I only switched to residential simply because business pricing wasn't in line with residential anymore," he said. (Verizon prices the residential service at more than $200 a month for 300/65; business service for dual 150 megabit lines runs $340 a month, and 300 megabit service costs $259 a month.)

Since the switch, he's used nearly 200TB in bandwidth—an average of 50TB a month. However, traffic most months is in the 30TB range. That sort of bandwidth would cost thousands of dollars per month in most colocation facilities—with the 77TB peak month jumping into the tens of thousands based on pricing plans I looked at from a variety of hosting vendors.

Houkouonchi's network traffic—blue is outbound, green is inbound.

But his bandwidth hogging eventually drew the attention of Verizon's engineers, who monitor usage for signs of unusual patterns in traffic. This practice is to watch for both abuse of the network (such as spam and denial of service attacks, for example) and for violations of the FiOS terms of service. Those terms exclude the use of FiOS for "high volume purposes" and forbid customers to "host any type of server. Violation of this section may result in bandwidth restrictions on your Service or suspension or termination of your Service."

Houkouonchi got a call from a Verizon representative this week. "Basically he said that my bandwidth usage was excessive (like 30,000 percent higher than their average customer)," houkouonchi said. "[He] wanted to know WTF I was doing. I told him I have a full rack and run servers, and then he said, 'Well, that's against our ToS.' And he said I would need to switch to the business service or I would be disconnected in July. It wasn't a super long call."

"I don't mind upgrading to business if that's really all the problem is," houkouonchi told me. "It just surprises me they would bother going after people just to get them to pay a little bit more per month. I know that when I switched to GPON (FiOS' Gigabit-capable Passive Optical Network) about six months after a serving hub came up in my area—it took six months fighting with Verizon to get switched over—there was only one other guy on the GPON ports on the serving hub. A lot of people in my area go with Time-Warner because it's cheaper."

276 Reader Comments

"Lots of stuff," houkouonchi replied. " I do some VPN stuff for people and Web/FTP/SFTP servers. A lot of friends and family stream stuff off me from my huge media collection. And I also do some P2P and Usenet stuff." Most of the storage space is taken up by videos and other media.

As much as people complain about "trolls", this guy's an idiot. He wonders why VZ is going after him for 'a few bucks more a month'? Trolls are screwing with ordinary people, trying to extort a buck out of them -- abusing the legal system in the process. People like this are just as bad -- abusing the terms that they agreed to when they purchased a product or service and then wondering why they're asked stop or pay more?

Typical (though thankfully not totally pervasive) Gen Y attitude... thinks the internet and everything he connects to is his personal free-for-all. Gee I wonder what that "P2P Stuff" and "streaming my huge media library" bit is about. No possible way he's illegally sharing apps and movies, is there? lol

Idiot. They should back-charge him for months of business-class service given that he knowingly violated their TOS (no way he didn't know / wasn't savvy to it).

It doesn't sound like he was running a business however? Maybe they define business in an odd way, but despite his equipment all his functions are personal and not for making profit?

I'm actually curious about this - I guess Verizon can do whatever they want to do though. Doesn't matter if he was _really_ violating TOS or not?

It doesn't really matter if you're running a business if what you are using it for is business-class. I mean, I wouldn't expect to run heavy industrial equipment in a residential neighborhood, even if I'm not actually using it for industrial purposes. This is pretty similar.

In the telco world, we call this a "right-sizing" call. A good telco will "right-size" you down to a plan that's less expensive in order to keep you longer. A bad telco will let customers stay on plans way over-bloated for what they need, but will breathe down the neck of folks like this to "right-size" them up the profit chain.

I'm really surprised it took them so long to call.

And kudo's for this guy actually utilizing the service he paid for. Most folks get this "unlimited" stuff, then hardly use it. It's a waste of money for them. But, yeah, this guy was definitely abusing the ToS on this. Normally don't side with Verizon, but it sounds like they handled the phone call in a very professional manner. They could have just shut his service down and waited for him to call.

As much as people complain about "trolls", this guy's an idiot. He wonders why VZ is going after him for 'a few bucks more a month'? Trolls are screwing with ordinary people, trying to extort a buck out of them -- abusing the legal system in the process. People like this are just as bad -- abusing the terms that they agreed to when they purchased a product or service and then wondering why they're asked stop or pay more?

What he is doing is a bit extreme, but what's kinda stupid (imo) is you can't host a server. I wonder where they draw the line. So VZ FiOS customers can't have a private TS server or run private games? That's pretty lame.

It doesn't sound like he was running a business however? Maybe they define business in an odd way, but despite his equipment all his functions are personal and not for making profit?

I'm actually curious about this - I guess Verizon can do whatever they want to do though. Doesn't matter if he was _really_ violating TOS or not?

These highlights might be of interest:

Quote:

Restrictions on Use. The Service is a consumer grade service and is not designed for or intended to be used for any commercial purpose. You may not resell, re-provision or rent the Service, (either for a fee or without charge) or allow third parties to use the Service via wired, wireless or other means. For example, you may not provide Internet access to third parties through a wired or wireless connection or use the Service to facilitate public Internet access (such as through a Wi-Fi hotspot), use it for high volume purposes, or engage in similar activities that constitute such use (commercial or non-commercial). If you subscribe to a Broadband Service, you may connect multiple computers/devices within a single home to your modem and/or router to access the Service, but only through a single Verizon-issued IP address. You also may not exceed the bandwidth usage limitations that Verizon may establish from time to time for the Service, or use the Service to host any type of server. Violation of this section may result in bandwidth restrictions on your Service or suspension or termination of your Service.

Quote:

You represent that when you transmit, upload, post or submit any content, images or data using the Service you have the legal right to do so and that your use of such data or content does not violate the copyright or trademark laws or any other third party rights.

I dunno what "represent" in the second paragraph means though (english is not my native language). I assume it's not an agreement as such..

What he is doing is a bit extreme, but what's kinda stupid (imo) is you can't host a server. I wonder where they draw the line. So VZ FiOS customers can't have a private TS server or run private games? That's pretty lame.

I'm pretty sure you won't get a call for hosting a game server. What this guy is doing isn't the same thing.

Yea but its not like his servers are going away. They are just going to charge him more for the principle of him running a server. Changing residential to business doesn't change his bandwidth usage and won't lighten the load. Its a name-plate only change (Aka useless and unnecessary).

It's actually rather common to see "No servers please" on the TOS of broadband connections, though in the UK, as far as I'm aware, most 'business' connections involve requiring a leased line as opposed to more standard connections, which cost LOTS.

I would *love* to see the server thing be a bit more relaxed, though I can see how it can get silly. As this article illustrates so succinctly.

Eh I don't know about 10's of thousands/month. You can get a full rack in a DC with wide open 100Mbit commit on gig uplinks for <2000/month. His 95% percent is probably not much over 100Mbit, and even if it is it's only about $5/mbit above that.

If you are colocating a single server and paying for transferred GB's it might be higher...

more surprising is that he's maintaining such a high level of bandwidth - if this is just for "friends and family" for some VPN and storage and testing - I would expect it to be a bit more up/down.

What he is doing is a bit extreme, but what's kinda stupid (imo) is you can't host a server. I wonder where they draw the line. So VZ FiOS customers can't have a private TS server or run private games? That's pretty lame.

The question you have to ask is "what is a server other than a computer that's likely to send just as much data as it receives".

More likely, what Verizon REALLY means is server-grade loads where you're sending and receiving several TB worth of data. Because most ISPs proportion their bandwidth asymmetrically, they don't one guy hogging all the upstream bandwidth.

What he is doing is a bit extreme, but what's kinda stupid (imo) is you can't host a server. I wonder where they draw the line. So VZ FiOS customers can't have a private TS server or run private games? That's pretty lame.

The "no servers" clause is just there to justify booting abusive users. The kind of user who generates terabytes of usage per month is bound to have at least one server that they can ding him for. It's the kind of clause that everyone ignores until they need an excuse to get rid of you. They're not going to care about your server if it's just generating a couple of MB per month.

"I only switched to residential simply because business pricing wasn't in line with residential anymore," he said.

That's not any sort of excuse when he knew full well that what he was planning on doing would violate the terms of service for the account. The guy is an (known) asshat plain and simple. He's been bragging about this on DSLReports for months on how he's been gaming Verizon, and he knew full well it wouldn't last.

That said, if he were to have flipped his bandwidth numbers around, VZW probably would never have called. He uses barely anything in download bandwidth. Obvious servers are obvious.

Sean Gallagher / Sean is Ars Technica's IT Editor. A former Navy officer, systems administrator, and network systems integrator with 20 years of IT journalism experience, he lives and works in Baltimore, Maryland.