The things customer service reps say probably shouldn't be taken as gospel.

With last month's court decision to kill net neutrality, Internet users are on the lookout for signs of preferential handling of network traffic. Doing the rounds today is a blog post by David Raphael.

Raphael's boss had noticed slow Internet performance at work. Raphael had noticed the same thing at home, and both he and his office were using Verizon's FiOS service. So he performed some testing. His work used Amazon's AWS for hosting, so he used AWS for his testing. He put a file on the S3 storage service and downloaded it at home. The transfer rate was a positively feeble 40 kilobytes per second.

Further Reading

He then connected to his office and downloaded the same file again. This time? 5,000 kilobytes per second.

Though this was somewhat more respectable, it was still a long way short of the 75 megabits per second that a speedtest reported his connection as. At around 4pm each day, Raphael's home Internet performance to AWS fell precipitously.

Update: I originally wrote 500KBps. The correct speed is 5,000KBps. 5,000KBps is very respectable.

Raphael contacted Verizon's customer service, chatting online to a representative to complain about the low speeds. After some amount of to-ing and fro-ing, along with the usual pointless rebooting of routers and patching of systems, Raphael asserted to the rep that something was limiting the bandwidth and asked if Verizon was limiting bandwidth to cloud providers. The customer service rep parroted his question back at him, apparently confirming his suspicions. Raphael asked if this was also behind poor Netflix performance; again the rep said yes.

The blog post was concluded with a pair of traceroutes that Raphael thought significant. They showed the paths taken to AWS from his home connection and his work connection.

Verizon's official line is that the customer service rep was misinformed, and that it treats all traffic equally.

First, a statement of the obvious. Customer service reps are pretty much the last people who would know about such a policy, let alone be able to inform customers of it. Add to this the usual confounding factors—non-native English speakers, a desire to "resolve" the customer's problem as quickly as possible (even if this means simply agreeing to what they say just to make them go away), and insufficient knowledge to actually go off-script—and the statements that Raphael cites as evidence don't really stand up.

As tempting as it may be when frustrated that your Internet connection isn't working as well as it should be, it's simply not fair to ask that kind of question to a low-level support person, much less use a chat transcript to accuse companies of widely unpopular behavior.

ISPs keep their networks opaque

But the bigger issue is this. It's not impossible that Verizon is giving preferential treatment to traffic. The technology exists to do such things, and Verizon was the company that challenged the net neutrality mandate, so it's not as if it isn't interested in giving some traffic better treatment than other traffic. It's just that as an end-user, it's really hard to tell.

We looked last year at the complexities of the relationships and deals between online video streaming and Internet service providers. Some traffic travels across paid links; other traffic goes across free, unpaid links. Sometimes companies get into disputes over what the fair rate to pay is and don't buy as much bandwidth as they need, leading to congestion and poor performance when connecting to certain services.

Even apart from issues with connectivity between providers, there can be bandwidth issues within a company's network. A street of 50 houses might boast 75 megabits per second of optic fibre Internet per household, but that doesn't mean that there's (75 × 50) = 3.75 gigabits per second of bandwidth between those houses and the ISP's core network. Those 50 connections might be aggregated into, say, a 1 gigabit link. If everyone in the street is hammering their Internet connection to download Linux ISOs from BitTorrent, they're simply not going to get the 75Mbps that their connections notionally provide. They'll be limited to an average of 20Mbps.

Further Reading

At one time it was common in the UK for ISPs to cite this provisioning ratio; typically it was 50:1 for consumer-oriented products and 20:1 for business ones. That is to say, for every 50 megabits per second of bandwidth sold to end users, only one megabit of bandwidth on the ISP's core network was available.

Raphael's description of the testing he performed (downloading remotely, rather than via VPN) doesn't rule out this much more mundane problem. The last hop, between Verizon's core network and Raphael's home (or office) could well be the source of the slowdown. If at 4pm each afternoon the kids are back from school, watching cartoons on Netflix and YouTube, watching streaming games on Twitch, and all the other bandwidth intensive things that we know home users tend to do in the afternoons and evenings, we'd expect to see poor performance on residential connections.

Business services, which tend to have lower contention and don't tend to have the same afternoon/evening surge, will be fine, even after 4pm.

To be able to authoritatively pinpoint the sources of slowness requires a lot more knowledge of ISP networks than ISPs are willing to disclose. As end users, we simply can't see which network ports on which switches and routers are running close to capacity. We don't know which links are massively oversubscribed. We also have no insight into who is buying bandwidth from who.

This leaves us little way of distinguishing between poor performance that's a "natural" result of congestion and poor performance that's an "artificial" result of traffic shaping and prioritizing. From an end-user perspective, Verizon throttling connections to AWS to 40kbps would look essentially identical to excessive network usage causing the link between Verizon and AWS to support only an average of 40kbps per connection. It would also look very similar to local congestion causing the same average throughput.

Verizon could, if it wanted to, figure out where the slowdown was coming from. Its network equipment will expose the information the company needs to determine which links are congested and which are not. Verizon may not be altogether interested in making and publicizing that information, however. ISPs seem to regard this kind of detail as a trade secret and are reluctant to disclose a level of overselling that'd make airlines blush.

With ISPs apparently now free to prioritize network traffic in whatever ways they see fit, it's certainly wise to be on the lookout for ISPs providing differentiated services in ways that put their customers at a disadvantage. But that vigilance needs to be tempered with an understanding of how ISPs actually work and much more careful analysis of network performance. Comments from first-tier support personnel aren't enough to draw any conclusions.

Promoted Comments

Looking at the traceroutes the user provided, there aren't any obvious signs of congestion (which would usually show up as substantial variation in the three response times in each hop). If anything, the business line seems to be seeing a bit more latency variance, but there's too little data to say much, other than it looks like both paths seem to be working fairly well.

Unfortunately, it's pretty hard to really know what's going on without seeing a tcpdump log or the like, where you can get some clues. If the ISP is throttling Netflix specifically, you might see unusually large packet loss rates there, relative to other traffic. But that could also be happening on congested links upstream. It's not usually possible to figure out *where* a packet got dropped, though sometimes you can make informed guesses if there's other data about the links visible, at least if it's congestion-related.

It is generally possible to figure out what's happening, but it takes a lot more data about multiple sessions to different places from the user's endpoint to really get any sort of handle on things. Essentially you have to construct a careful set of experiments after piecing together the relevant parts of the network topology. And even there you're at the mercy of all kinds of things like multipath load-splitting, where different sessions may route over different routes than you might see with trace route.

107 Reader Comments

Very well written overview Peter. As you are wont to do, you have offered & encouraged a measured, rational & insightful approach to confronting routinely frustrating scenarios.

That said, deep within the fetid & necrotic intestines of one character central to this narrative, beats the dark heart of an unfathomable vehemence; the cadence perpetuating one who draws sustenance through its malevolent, parasitic, & sadistic exploitation of innocent masses.

HINT: David Raphael is not the name that conjures existential fear, loathing, dread; nor does he drive wholly sane persons to cry out; to pray for a slow & painful death as an alternative to continued commercial torture. "Abandon all hope, ye so cursed as to glimpse of my visage!"

(I used to work for Verizon. No way could I pass on an opportunity presented so invitingly!.)

I'm not sure how much more specific the evidence needs to get before you would believe that something is amiss with Verizon's service, Peter.

If it is to demonstrate throttling as opposed to mere contention, it would have to be very specific indeed.

In some ways, that's why being overly concerned about net neutrality is I think missing the point. The effect of Verizon deprioritizing and shaping Netflix traffic is going to be broadly similar to the effect of Verizon only having a narrow pipe to Netflix. The former would violate net neutrality; the latter wouldn't.

Quote:

In the case of the VPN vs. home connection, maybe that's just congestion between myself and the CDN that isn't present between the VPN gateway and the CDN. What about the Verizon Wireless vs. Verizon FiOS example? The most logical place to connect the backhaul for the local cell tower would be the very same building that hosts the endpoint for my FiOS connection (about 1.5 miles southwest of me.) If that's the case, then shouldn't my cellphone see the same congestion issues?

A traceroute would be informative. But I don't think it's inevitable that they use the same network infrastructure. They have different constraints (for example, mobile data usage normally has some kind of per-byte tracking and billing system, and mobile networks seem more prone to being behind large-scale NAT, for example) so I wouldn't count on them running over the same network.

Quote:

I get what you're saying. You're trying to be scientific about it. The guy's "evidence" is utter crap and it doesn't definitively tell us anything useful. There are too many confounding factors. Unfortunately, most home internet users don't have very many options with regard to tests that we can perform. My own experience tells me that there is something seriously amiss, and it has been since well before the Net Neutrality decision. It started with YouTube performance, and progressed to Netflix. I fully anticipate that Amazon will be next.

If it has been an issue since before the decision then I think that tends to lean more heavily in favour of it being something other than prioritization.

Proving distasteful throttling by ISP can't done by one customer- the entire neighborhood has to get involved. With the recent court nod-wink license to throttle, customers have got to have the right tools and organization to collect and aggregate data to confirm or refute suspicions. I for one, would be willing to get involved in such an effort.

Throttling or not, 40 kbps is simply unacceptable given the speeds advertised by ISPs. Even if consumers cannot prove traffic shaping, this should be treated as a matter of false advertising.

I seriously hope net neutrality rules are enacted at some point. ISPs are increasingly becoming content providers with their Internet+TV offerings. They own the infrastructure and a large part of the contents and have an incentive to degrade the quality of access to their competition. It's crystal clear to me that regulation is needed.

The argument about what exactly is the culprit is moot insofar as it doesn't remedy the problem. The congestion caused by the surge in bandwidth (caused by streaming movies and other footage) is reasonable but nonetheless shameful for the ISP since it has to ramp up available bandwidth according to use patterns ... which are about to change from time to time.

These antique overbooking rules of 1:X are obsoleted by movie consumption of consumers during their spare time. During the process of classical web-surfing each time the user loads a new page a short spike in data transmission is triggered. Streaming on the contrary uses the capacity to the full extent and leaves very little headroom for overbooking.

But hey ... better brace yourself for being charged for priority traffic anytime soon. The gates for that insidious practice have been opened and it's just a matter of time until some dauntless ISP takes the plunge.

It would have been much more interesting if the guy posted traceroute output from both ends.

I am kind of shocked that VZ reaches amazon via NTT. That must get expensive. Google and amazon are the top two traffic sources when I run stats at work. Both of course will peer with basically anyone for cheaper than transit if you're working on any scale beyond a few Gb/s.

Answer why VZ doesn't want a peering arrangement, and you answer a whole bunch of other questions.

Verizon could be throttling Netflix and Amazon, but there’s no actual evidence of it

What kind of headline is that?

"Unicorns could be involved in recent bank heists in Dayton, Ohio, but there's no actual evidence of it."

"Space Aliens could be behind the recent Superbowl ad from GoDaddy, but there's no actual evidence of it."

In this case the Unicorns have shown interest in making the bank heist and got a mandate allowing them to. Someone probably went through with the bank heist and the Unicorns wont say if it was them and wont help the investigation.

I am not surprised. I am more surprised anyone would find this surprising, though. Verizon, like Comcast and AT&T, all sell content and they want and will discriminate against your activities if it involves their content competitors.

I'm suspicious of Comcast doing the same with traffic between my Apple TV boxes and Apple's content (e.g. Trailers app, etc.). I try to play movie trailers in HD and they stall out, buffering comes to a halt, etc., but the exact same HD video trailers run fine on Vimeo and YouTube.

Coincidence? Maybe, but when both devices are using wired Internet with ~30 Meg downstream service and only Apple apps on the Apple TV go to Apple's network and only they are having bandwidth issues ...

Then again, the cowsumers at large (Moo!) are oblivious to things like Deep Packet Inspection and the triggering effects therein.

Actually, there is plenty of evidence, and it's not hard to prove. I pay Comcast for fifty megabit, I get USENET speeds of 51mbit when I set my server to SSL out of Asia over fifty connections. US based servers? Six, maybe.

ARS can buy 6 VPN accounts from around the US, and do an analysis of different ISP and download speeds, Netflix connections, etc - to compare tracert hops and speeds when the traffic in different types of tunnels versus natively.

Run some torrents while you are at it, do some erlang calculations, and spend at least as much time doing it as you did drinking soylent and I bet the results end up as ammunition to make this bs a federal crime.

Why wait for someone else to do it when you can garner the glory?

How does that prove throttling, as opposed to congestion?

Oh right: it doesn't.

?

Exiting the verizon system to the VPN which you can traceroute at the VPN to Netflix, then traceroute straight to Netflix simultaneously.

You could monkey with this until you have identical traceroutes minus the VPN connection, and compare your throughput to Netflix.

You could easily prove artificial 'congestion.'

Not 'court of law' prove, but certainly 'well, you delivered the food to my table and the food that you knew was coming is poisoned, and the food you thought was just a box from amazon that I had delivered through your kitchen isn't poisoned...'

From my personal experience, its near impossible to get a "Super HD" stream from Netflix during peak hour times on Verizon FIOS. Even a regular HD stream can be flaky during these periods. I have no problems with an Amazon HD streaming video though.

I'm suspicious of Comcast doing the same with traffic between my Apple TV boxes and Apple's content (e.g. Trailers app, etc.). I try to play movie trailers in HD and they stall out, buffering comes to a halt, etc., but the exact same HD video trailers run fine on Vimeo and YouTube.

Coincidence? Maybe, but when both devices are using wired Internet with ~30 Meg downstream service and only Apple apps on the Apple TV go to Apple's network and only they are having bandwidth issues ...

Actually, this could be a coincidence. The apple content delivery network is likely not at the same places as the vimeo or youtube content delivery networks, so the apple routes could be more congested than the vimeo or youtube routes.

With as many employees as Verizon (and AT&T) has, chances are good that we all know someone (or someone who knows someone) behind the scenes in a technical capacity that can give accurate answers to pointed and specific questions. Personally, i know quite a few of just such persons. I don't waste a moment bandying about with customer service reps initially when i have an issue. I first confirm my suspicions as to the cause of my issue with people "in the know", and then place that call to customer service, revealing just enough information to impart upon them the fact that they're not dealing with just another dumb ass. If you catch my drift....Verizon, AT&T, Comcast, Charter, Time-Warner.. They're all doing "sneaky things" which are really hard to prove. But if you can arm yourself with knowledge that they ARE in fact doing something in particular, you can gain the upper hand in most any situation that is giving you cause for grief. That's all i can say. It's not what you know. It's WHO you know.

1) His routes had no congestion2) The routes were nearly the same3) He was able to speed test his full 75mb/s4) He was getting 40kb/s

1+3+4 should be impossible. The first thing can comes to mind is something wrong on his home end. I would have felt better if he bypassed any NAT/Firewall that he had and tested at least 2 different computers, and preferably a naked OS with no Anti-Virus installed, like a live Linux disc.

Probably the most likely reason you'd never see such direct evidence of "throttling" is that they probably don't NEED to do that, in the sense you might be thinking of it. All they need is a certain level of not-caring.

"Switchbox 852b is reporting problems. That's the one that connects to that ISP Netflix uses - I guess it was old hardware dealing with a lot of congestion last night. We should probably replace it.""Oh, darn! That old reliable box is acting up? Well, we'll see if it was just a fluke. Besides, we don't want to add a sudden spike in the monthly hardware budget...that wouldn't look good on the review. Can you still send a 1KB ping through it?""During low-use hours? Sure.""Oh, it's fine then! Let me know how it's performing in a month."

Of course, even that shouldn't be assumed either without evidence, making this article just a little incredulous.

I have the 75/35 FiOS package and consistently get around 100/50 when I run a speedtest. I barely get SD quality when streaming Netflix even though the shows are capable of HD.

I was getting HD streams around Christmas time but I can't pinpoint the exact date the quality started turning to crap. It was definitely less than a month ago, right around the Net Neutrality ruling though.

I have no other problems with other streaming/downloading.

I've had the same problems with AT&T U-Verse. I can barely get HD streaming. It takes 10 minutes of playing before it finally switches to HD. If I enable my VPN connection, it takes mere seconds to get an HD stream. It may not be AT&T (but probably is), however, it's pretty clear that someone along my route to Netflix is doing some terrible QoS on video streaming.

I have the 75/35 FiOS package and consistently get around 100/50 when I run a speedtest. I barely get SD quality when streaming Netflix even though the shows are capable of HD.

I've got the 150/50 package, and just scored a 150/68 on SpeedTest, and yet the quality of Netflix streaming is inconsistent at best. I toggle between Super HD, SD and dropping the connection altogether.

Though I must admit that HBO Go is far worse. For the last ten days or so it's been virtually unwatchable. Yesterday evening I tried to watch Life of Pi on HBO Go, and it went into low quality mode at least a dozen times. On top of that it dropped altogether three times, and even had trouble starting back up - all the while my Internet performance was otherwise normal.

If it has been an issue since before the decision then I think that tends to lean more heavily in favour of it being something other than prioritization.

I would agree, and I think it goes back to the article of degraded peering connections. That doesn't mean, however, that it doesn't violate net neutrality or that it isn't a form of shaping. By intentionally allowing a peering connection to a specific CDN to degrade, they are effectively shaping traffic (through neglect) and hiding it behind a business dispute.

Traceroutes/pings are unreliable on as a testing tool on the internet. This is because ISPs de-prioritize such packets destined to their equipment. That is, ISP networks are built to be transport, their routers/switches aren't the destination of end-user traffic and as such, the de-prioritize ( put in lower queue ) traffic sent to the router.

Think about how traceroute works. Now, add in the fact that if the ISP has an MPLS core, traceroutes become even more unreliable. To MPLS, a traceroute has to go all the way from Label Ingress Router to the Label Egress router and back before being returned to the user. Your traceroute will hide the fact that you could be going halfway around the world (topologically) between hops.

Also, business customers are typically homed to a different physical plant that consumer customers. This could explain the better performance at his business vs. at his home...

I'm surprised nobody has questioned whether this wasn't an attempt to gin up traffic for his blog and company. The evidence is as thin for that idea as the evidence that they're throttling, but it's reasonable speculation.

1+3+4 should be impossible. The first thing can comes to mind is something wrong on his home end. I would have felt better if he bypassed any NAT/Firewall that he had and tested at least 2 different computers, and preferably a naked OS with no Anti-Virus installed, like a live Linux disc.

Those are legit test cases. I don't think people realize how much it takes to actually process network traffic. Its an entire stack of software and physical wires and hardware...if you're not thinking about that when your doing your test, your not testing correctly...

Verizon could be throttling Netflix and Amazon, but there’s no actual evidence of it

What kind of headline is that?

"Unicorns could be involved in recent bank heists in Dayton, Ohio, but there's no actual evidence of it."

"Space Aliens could be behind the recent Superbowl ad from GoDaddy, but there's no actual evidence of it."

In this case the Unicorns have shown interest in making the bank heist and got a mandate allowing them to. Someone probably went through with the bank heist and the Unicorns wont say if it was them and wont help the investigation.

I'll keep pestering a friend of mine who has FIOS to do some traceroutes. I will say that all things considered, my Comcast service has been pretty good and doesn't have any noticeable throttling to major services like Netflix. Though maybe policy is not implemented network wide. I live in a small-to-medium metro area. That might make a difference for implementation.

That said, if FIOS ever did come here I'd be nervous to use it. These accusations alone aren't enough to prove they're throttling but they're shady in so many other areas of business that it makes me a skeptic to their FIOS product. I'm not sure why I'd expect it to be better.

i get the indian kid did not know what he was saying, but he represents the company and he said that as a representative of the company. It really isn't the end users' fault that the company is so big that there isn't anyone that can answer simple questions.

if there is no traffic shaping, that is pretty easy training to pass down.

Yes he is supposed to represent the company, but any large company has trouble hiring enough qualified and competent tech support people, and it is highly unlikely that the tech support person would have the information he claimed to have.

So in this case it appears that the employee likely misrepresented the company he works for. I think we have all experienced at one time or another a tech support person that didn't know what they were talking about that didn't let that stop them from providing an explanation that was completely wrong.

As much as I dislike VZW, I can't imagine that such a large ISP would throttle such large content providers without telling anyone.

The most likely scenario would be VZW would tell Netflix about their plans to do X unless they are paid. Then Netflix would bring it to the press.

Business may work slowly, it may be unkind, small transgressions might take a while to come to the surface and be acknowledged and officially remedied. In general such a high profile move won't be done without an official communication.

And if they are officially planning such a move, it's going to take a while for the whole process to play out. So please don't act so offended if once the decision is made, the board room doesn't tweet their intentions within the hour.

Step 1 is simply requiring ISPs to reveal their policies on throttling, shaping traffic, QoS, etc. A free market can operate only when the customers can make valid comparisons between products. If the quality of service is a trade secret, it's preposterous to pretend market forces can do anything about it.

Once they are open, then they can be held to account for whether their actual behavior matches their promised behavior.

i get the indian kid did not know what he was saying, but he represents the company and he said that as a representative of the company. It really isn't the end users' fault that the company is so big that there isn't anyone that can answer simple questions.

if there is no traffic shaping, that is pretty easy training to pass down.

The other point people are missing (or conveniently ignoring) is that no employee of a large company is expected to know everything about that company and it is completely unreasonable to expect otherwise. In case you've never dealt with technical support before, here's how it works: there are tiers of support. People on the first tier typically don't have a lot of training or information beyond very basic stuff because their job isn't to answer every single question and concern you have. Their job is crowd control. A company like Verizon gets hundreds of calls per hour--at the front lines you need people who can dispense with the easy and obvious stuff that's a waste of a trained technician's time ("I don't remember my password" type questions). A job listing I found online read "Each rep handles 40-58 calls per day". Highly trained technicians are expensive and harder to find, so if the front line was all highly trained, queue times would be a lot higher and less people would get through.

That's where the other tiers come in. If a front line rep runs into an issue that the script can't address, they are supposed to escalate to a higher tier. The technician's fault here is attempting to answer a question he wasn't qualified to answer. He should have said something to the affect of "I can't answer that, sir, but I can transfer you to my supervisor (or another call group or however they're structured) who should be able to help you". Unfortunately, front line techs are often motivated (usually by the company itself) to process as many calls as possible, and it's faster to just give an answer and get them off the phone than to transfer them.

TL;DR: the issue is not with reps being ill-informed, the issue is that the rep answered questions that were outside his role and had no place answering. Front-line call center reps are not technical people and are only there to process routine calls and route non-routine calls and questions to someone with more technical competencies.

I have 80/40 FIOS and I haven't noticed anything suspicious yet. I can stream Netflix without problems and I was actually trying out AWS/S3 and I was disappointed that I was getting "only" 10Mbps uploads. Downloads are plenty fast too. But ended up not liking AWS so I don't use it any more.

I have the same problem in the last couple of weeks. I usually get 25Mbps (Comcast business) but when Netflix is on at night, the transmission rate drops like a rock. I couldn't even finish a show last night and it is very frustrating. I am definitely keeping an eye on it. If Comcast is messing with me, I would switch to AT&T. Albeit at a lower rate, I would be happier.

He didn't demonstrate, this, actually. Singular ping times are a really terrible way of demonstrating that the network isn't congested. For all we know, the normal, uncongested ping times on those hops should be half of what he showed. The information he presented simply isn't sufficient to make any determination.

Quick correction - when remoted through the office connection, he saw 5000kB/s, not 500kB/s - so the download vpn'd over his office network is 100x faster, not 10x.

Also, the traceroutes from the house and the office both go through the exact same path once outside verizon's network. I would be curious to see if he is hitting the same first few hops going from house -> office as he is going house -> amazon, which would indicate whether it is in fact just congestion on some hops within verizon's network.

He didn't demonstrate, this, actually. Singular ping times are a really terrible way of demonstrating that the network isn't congested. For all we know, the normal, uncongested ping times on those hops should be half of what he showed. The information he presented simply isn't sufficient to make any determination.

He is lacking the traceroute between his house and his office. *IF* that uses the same hops within verizon's network as the traceroute from his house to amazon, then that would indicate there is no congestion.

If it has been an issue since before the decision then I think that tends to lean more heavily in favour of it being something other than prioritization.

I would agree, and I think it goes back to the article of degraded peering connections. That doesn't mean, however, that it doesn't violate net neutrality or that it isn't a form of shaping. By intentionally allowing a peering connection to a specific CDN to degrade, they are effectively shaping traffic (through neglect) and hiding it behind a business dispute.

By that logic, intentionally allowing a peering connection to improve would also be shaping traffic. Peering agreements are business decisions and the disputes are more over who foots the bill for the new 10Gbps port on the switch than anything else.

Netflix wants Verizon to peer directly with their CDN because currently their packets have to pass through the peering connection between Verizon and one or more ISPs (in NYC, CenturyLink/Qwest), which may not be robust, especially during peak hours. Peering directly with Verizon would allow connections to be routed directly to Netflix, easing congestion. The problem is that Netflix wants to peer for free and Verizon wants to be compensated, and both sides have (at least partially) valid arguments.

Netflix argues that the reason for the disproportionate downloads vs uploads is because Verizon's customers are requesting large amounts of data, so Verizon should pay the costs associated with peering with them. This is only partially correct because Verizon peers with their ISP (CenturyLink/Quest in NYC), and congestion is only a problem at certain times of day. If anything, that peering connection should be improved, but that depends on the contract Verizon has with that ISP, not Netflix.

Verizon's argument is that Netflix CDN isn't an ISP, which it isn't, so they shouldn't have to peer with them for free. Technically they are correct: my web host can't peer directly with Verizon for free so why should any non-ISP? Preferentially giving Netflix free access would not even be in the spirit of net neutrality—either everyone should pay or no one should, and the latter is unsustainable. That said, we do pay Verizon with the expectation that they will do whatever is necessary and proper to provide the level of service we expect and the price they are demanding might be a bit unreasonable.

Seems clear to me that the 40kb/s is not due to kids watching cartoons, or penguins downloading linux, if you at the same time are getting 75Mb/s scores on speed tests. Clearly some form of traffic bias takes place here. So it's safe to say that the problem is not at the endpoint. This does not exclude similar problems further up the stream though, but it seems to me that this article is biased or makes too light of the issue.

Verizon's argument is that Netflix CDN isn't an ISP, which it isn't, so they shouldn't have to peer with them for free. Technically they are correct: my web host can't peer directly with Verizon for free so why should any non-ISP? Preferentially giving Netflix free access would not even be in the spirit of net neutrality—either everyone should pay or no one should, and the latter is unsustainable. That said, we do pay Verizon with the expectation that they will do whatever is necessary and proper to provide the level of service we expect and the price they are demanding might be a bit unreasonable.

When you put it this way, it seems Netflix should buy or start an ISP...

"As tempting as it may be when frustrated that your Internet connection isn't working as well as it should be, it's simply not fair to ask that kind of question to a low-level support person, much less use a chat transcript to accuse companies of widely unpopular behavior."

Not fair??! How is it not fair??!

Verizon chooses who they hire as tech support. They train them. They use them as the point of contact for customers who have issues using their services.

If Verizon is negligent in this.. and hire unqualified people, and don't train them properly.. who else is to blame but Verizon?

No, it's totally fair to expect to be given correct information by tech support.

"Is Verizon now limiting bandwidth to cloud providers like Amazon's AWS services?""Yes it is limited bandwidth to cloud providers"

"And this is why my netflix quality is also bad now""Yes, exactly"

I call that a smoking gun.

I find it amusing that you believe that they train them. These are more likely to be plebs at the bottom of the income food chain, who are given a sheet of sample answers and tests to keep the customer happy and on their way. Do you actually believe that they are trained and competent at doing tech support?

This is bit unfair. Naturally, the most qualified people are not those you will get when you call Support. You get someone who was hired for social and telephone skills more than anything else, and their job is often to get a problem statement, and do some basic script-based troubleshooting and resolve the problem if they can. Stuff they cannot resolve gets escalated up the chain through progressively better qualified and more experienced people. Things are organised this way so that you don't get your most experienced people wasting time on the phone solving problems that a router reboot will fix. They are far too expensive to be doing this.

A question like "do you throttle Netflix" would be difficult for the guy on the phone to answer indeed, and even if they did have the answer, they would more than likely be encouraged to withhold it.

Verizon's argument is that Netflix CDN isn't an ISP, which it isn't, so they shouldn't have to peer with them for free. Technically they are correct: my web host can't peer directly with Verizon for free so why should any non-ISP? Preferentially giving Netflix free access would not even be in the spirit of net neutrality—either everyone should pay or no one should, and the latter is unsustainable. That said, we do pay Verizon with the expectation that they will do whatever is necessary and proper to provide the level of service we expect and the price they are demanding might be a bit unreasonable.

When you put it this way, it seems Netflix should buy or start an ISP...

Just what we need, more vertical integration.

No. It's not vertical integration. It's protecting your interest. They don't need to buy or start an ISP. They can invest (or buy a stake) in an ISP with policies they consider good, and in this way, both help to ensure their continued survival, and have additional sources of income (more assets).