AT&T was fastest in data but completed the fewest voice calls.

AT&T defeated rivals Verizon Wireless and Sprint at yesterday's Super Bowl almost as soundly as the Seattle Seahawks bested the Denver Broncos, according to a test of cellular data performance.

AT&T provided average throughput of 5.3Mbps down and 2.9Mbps up, while Verizon and Sprint couldn't even manage half that according to tests by Nexgen Wireless. It wasn't all good news for AT&T, though, as its network completed a lower percentage of voice calls than either Verizon or Sprint.

The tests yesterday were conducted in six sequences before, during, and at the end of the game in various seating areas and levels of the stadium, both indoors and outdoors. Each sequence took about 28 minutes and included 10 calls of about 20 seconds each; three calls of about two minutes each; playback of a two-minute YouTube video; uploads of 20MB and 10MB files; and 100 ping tests. While the networks varied in percentage of calls completed, none of the networks dropped any calls after they were completed.

Average data speeds and ping latency are shown in the following chart, along with percentage of voice calls completed:

"AT&T was the clear winner with a big advantage on their data network over Verizon and Sprint," Nexgen Wireless said when announcing the results. "Sprint won in the Voice category but with a much more narrow margin than what was seen in data. Verizon placed second in both categories. The results showed that AT&T was the best prepared for the traffic they encountered at this event."

T-Mobile US wasn't included in the test because its "file structure wasn't compatible with the Accuver data collection software," a Nexgen spokesperson told us. The company expects to be able to evaluate all four major US carriers in future tests. The carriers were aware of the tests but didn't have any influence over them, according to Nexgen.

Nexgen, a maker of network analytics products, plans to continue conducting these tests at major events in big US cities throughout the year.

Verizon led the buildout of cellular and Wi-Fi infrastructure at MetLife Stadium in New Jersey, but all four major carriers upgraded their Distributed Antenna Systems in anticipation of the Super Bowl. To keep data usage from overwhelming the network, the NFL blocked streaming of the game inside the stadium.

AT&T reported that its customers used 624GB of data at the stadium, "the highest data usage we’ve ever seen from a one-day sporting event we’ve measured."

After publication of this article, Verizon pointed us to an announcement of its own which showed that its customers used 1.9TB of data in the Super Bowl stadium. That's about three times more data served than AT&T, which could well explain the speed differences between the two networks. Still, an individual user would have had a better experience on the AT&T network when uploading and downloading data.

One thing that wasn't clear from Verizon's announcement is whether the 1.9TB includes Wi-Fi in addition to cellular data. A Verizon spokesperson told us that it's just cellular data, saying, "The 1.9 terabytes is on the Verizon Network only, no Wi-Fi included."

44 Reader Comments

Unless I'm missing something, isn't this just a typical engineering trade off where ATT favored data speeds over voice calls, Sprint took the opposite approach and Verizon was somewhere in the middle? That's assuming the same number of active users on each network which, without the carriers providing data, is an unlikely assumption IMO.

I know it's pie in the sky, but I would love if they had included a random prepaid MVNO on each network in the tests as well. Just to see if there is any credence to the rumor that MVNO customers are second class citizens.

Unless I'm missing something, isn't this just a typical engineering trade off where ATT favored data speeds over voice calls, Sprint took the opposite approach and Verizon was somewhere in the middle?

I'm on Sprint and would rather have voice calls and 1Mbps unlimited than 100Mbps with a cap. Am using over 10GB/month sometimes much more, that would be expensive on ATT/VZW. Granted, I'm in a completed Sprint area. Typically get 6-20Mbps, which is fine as no smartphone app needs more than a few Mbps.

Nobody uses AT&T/T-Mobile in the metro area because they suck. GSM based stuff sucks in NYC just because the ways buildings are built. So Voice service has always been lower.

And that Verizon is pretty much king in region in terms of wireless because they've had the stronghold on tower locations way before data even came into play. So they have high subscriber usage.

Sprint is down in the dumps because well its really really cheap in terms of data cost so everyone jumps onboard so they get near zero bandwidth but decent voice. Their 3G network is overloaded and their 4G is spotty to say the least.

What you don't see in the study. How many people in the area used which service simultaneously.

Nobody uses AT&T/T-Mobile in the metro area because they suck. GSM based stuff sucks in NYC just because the ways buildings are built. So Voice service has always been lower.

And that Verizon is pretty much king in region in terms of wireless because they've had the stronghold on tower locations way before data even came into play.

Sprint is down in the dumps because well its really really cheap in terms of data cost so everyone jumps onboard so they get near zero bandwidth but decent voice. Their 3G network is overloaded and their 4G is spotty to say the least.

ALL tests are region based. Sheesh. You say "the metro area" like I should know where that is :-P

I'll say that in the San Francisco Bay Area, that AT&T slaughters Verizon for data speeds. I have an iPhone 5s on AT&T, and an iPad Air with Verizon LTE coverage. My iPhone regularly gets 2x-3x the speeds of the iPad. This was the case with my previous generation devices as well. I think in general AT&T has come a long way with their network buildout here in the past couple years. And this is mostly down on the peninsula; in San Francisco itself there are still painful times where I'll get 5 bars and effectively no data because the network is overloaded.

But this is going to vary greatly by region. Some regions will favor one carrier, others will favor another. In general AT&T and Verizon are pretty far ahead performance-wise over Sprint, and coverage-wise over T-Mobile. There's a site that does a regular annual survey of about 100 metro areas that's worth a read so you can decide which is fastest in the area that's important to you.

A super bowl test is interesting though as it effectively represents a "worst case" scenario with EXTREMELY high density. Now you'd think this would be an event the telcos would want to prepare for: If you're on say Verizon and can't get data, but all your friends or seat-neighbors are on AT&T and have plenty of signal, it could sway your opinion come renewal time. Plus, maybe Ars Technica will run an article that makes you look like a fool ;-)

Personally, I will say that as a Sprint user back in the late 90's I was at Laguna Seca for a weekend and had NO SIGNAL the entire time. A total dead zone. But everyone around me had plenty of phone service on other networks (I think Cingular at the time?). I did change phones shortly thereafter and have not considered Sprint again due to the poor quality/coverage of their network the last time I was a customer. It's likely that a lot has changed in the past 15-18 years but I'm still bitter ;-)

Nobody uses AT&T/T-Mobile in the metro area because they suck. GSM based stuff sucks in NYC just because the ways buildings are built. So Voice service has always been lower.

And that Verizon is pretty much king in region in terms of wireless because they've had the stronghold on tower locations way before data even came into play. So they have high subscriber usage.

Sprint is down in the dumps because well its really really cheap in terms of data cost so everyone jumps onboard so they get near zero bandwidth but decent voice. Their 3G network is overloaded and their 4G is spotty to say the least.

What you don't see in the study. How many people in the area used which service simultaneously.

At a minimum, those are completely unverified assertions and more likely fall into the "patently false" category. I know a lot of people in this region. Both AT&T and Verizon are very well represented in the customer base. I oversee our company's mobile phone policy and the split is pretty even.

Personally, I use T-Mobile and its coverage is excellent in the NYC area. I'm almost always on LTE. I was at the game too and LTE coverage was consistent. I was able to do a couple of Facetime calls with no performance issues, and my Internet speed was what I've come to expect from T-Mobile's LTE network. I actually wish I had done a speed test while I was there but was too shocked at Denver's self-inflicted wounds during the game.

It seems that AT&T *should* have provided the fastest data at the game.

Assuming that the LTE technologies are equivalent between the carriers (same approx. speed/capacity), then AT&T would have a speed advantage during heavy usage with customers who only had 3G connectivity or were dropped down to 3G due to congestion.

(AT&T's "3G" network uses a technology called HSPA, which allows (theoretical) download speeds of 21Mbps, while Verizon's 3G technology is called EvDo (Rev A) which allows for a (theoretical) download speed of 3.1Mbps. (Sprint *may* be using EvDo (Rev B), which (in theory) can support speeds up to 14.1Mbps))

What they didn't measure was how many of each carries phones couldn't get any data service, or was so slow it was unusable.

Verizon delivered ~3x more data total than AT&T, which could be 3 x the total number of handsets. I'd rather a few Mb tops on Verizon than my AT&T phone having no data.

Unfortunately without data from the carriers about number of subscribers it's just speculation on why VZW did 3x the data. One thing we do know is that VZW won't provision a 4G/LTE tower without a minimum of 10Gbps of backhaul. No idea about Sprint or AT&T.

Unfortunately without data from the carriers about number of subscribers it's just speculation on why VZW did 3x the data. One thing we do know is that VZW won't provision a 4G/LTE tower without a minimum of 10Gbps of backhaul. No idea about Sprint or AT&T.

True but since AT&T's data was faster on average, they obviously had a bigger backhaul.

Every carrier was maxed out, otherwise people would have seen ~50Mbps.

Having faster data on average means they had more backhaul available per customer on AT&T.

I'm with Nexgen Wireless, we conducted the tests. The reason we don't normally test MVNO's is that it's not a true representation of the network performance since they utilize the tier 1 carrier's network.

I know it's pie in the sky, but I would love if they had included a random prepaid MVNO on each network in the tests as well. Just to see if there is any credence to the rumor that MVNO customers are second class citizens.

I'm sure blocking streaming out of the stadium had everything to do with ensuring there was plenty of bandwidth available and nothing at all to do with the NFL's lucrative exclusionary television broadcast contracts.

Nexgen Wireless' testing was concentrated on the Super Bowl performance. Our testing is micro level focused as we tend to conduct our studies at marquee venues and major events. Given that the carriers spend months of planning for these major events, it's important to assess how they have prepared for the influx of traffic on their network.

Nobody uses AT&T/T-Mobile in the metro area because they suck. GSM based stuff sucks in NYC just because the ways buildings are built. So Voice service has always been lower.

And that Verizon is pretty much king in region in terms of wireless because they've had the stronghold on tower locations way before data even came into play. So they have high subscriber usage.

Sprint is down in the dumps because well its really really cheap in terms of data cost so everyone jumps onboard so they get near zero bandwidth but decent voice. Their 3G network is overloaded and their 4G is spotty to say the least.

What you don't see in the study. How many people in the area used which service simultaneously.

I know it's pie in the sky, but I would love if they had included a random prepaid MVNO on each network in the tests as well. Just to see if there is any credence to the rumor that MVNO customers are second class citizens.

I think it has to do with the roaming agreements be excluded than anything else.

I am just glad that we are finally developing high density cell service. I would really like to get reception in buildings.

(AT&T's "3G" network uses a technology called HSPA, which allows (theoretical) download speeds of 21Mbps, while Verizon's 3G technology is called EvDo (Rev A) which allows for a (theoretical) download speed of 3.1Mbps. (Sprint *may* be using EvDo (Rev B), which (in theory) can support speeds up to 14.1Mbps))

Sprint's only using Rev A which maxes out at ~3.1mbps. It's a very good point to make here, and something that the Verizon fans were making fun of when VZW began rolling out their network. "Who cares if T-Mobile and ATT roll out HSPA, nothing's as awesome as LTE!". ATT and T-Mobile stated pretty clearly that a benefit to the 3gpp track is that there's a smooth transition from LTE speeds to HSPA+ speeds. The drop from LTE to EvDO is practically jarring in comparison. There's quite a bit of value for T-Mobile and ATT customers in having a 42mbps and 21mbps HSPA+ network to fall back on when LTE data is overloaded. Moreover, they have 2 separate voice networks to use: UMTS and GSM. If UMTS is overloaded, customers can drop down to the base GSM network. All in all, this means that - all things being equal (ie spectrum, backhaul, and load) - the ATT and T-Mobile networks will theoretically perform better for more users than the VZW and Sprint ones can.

T-Mobile US wasn't included in the test because its "file structure wasn't compatible with the Accuver data collection software," a Nexgen spokesperson told us.

Something doesn't add up here. If Nexgen is performing the test itself with Galaxy S3 phones that have its software installed, how in the world is it not capable of testing T-Mobile US? Something is rotten here...

T-Mobile US wasn't included in the test because its "file structure wasn't compatible with the Accuver data collection software," a Nexgen spokesperson told us.

Something doesn't add up here. If Nexgen is performing the test itself with Galaxy S3 phones that have its software installed, how in the world is it not capable of testing T-Mobile US? Something is rotten here...

Sometimes shit just does not work, this was one of those times. The best conspiracy is an unproveable(sic) one.

I talk so little. I use the least amount of minutes on our family plan, all the time.

I'm the same way -- it makes finding a reasonably-priced plan or carrier a pain. I ended up sharing an account with my mother just so I wouldn't be paying for 100 minutes each month and only using 5-10 of them. I've heard that communicating via text or online & not via regular calls is increasingly common with people in their mid-30s & younger, so hopefully more carriers will finally change their plans to match.

Nobody uses AT&T/T-Mobile in the metro area because they suck. GSM based stuff sucks in NYC just because the ways buildings are built. So Voice service has always been lower.

And that Verizon is pretty much king in region in terms of wireless because they've had the stronghold on tower locations way before data even came into play. So they have high subscriber usage.

Sprint is down in the dumps because well its really really cheap in terms of data cost so everyone jumps onboard so they get near zero bandwidth but decent voice. Their 3G network is overloaded and their 4G is spotty to say the least.

I agree, overall the information's not terribly useful for those of us outside that area. IMHO things like Open Signal Map are more useful, both when compared to the test and to the carrier's official coverage maps.

Up in my part of the Bay Area, Verizon has great coverage & speed, AT&T & T-Mobile aren't far behind, and then Sprint has a *really* weak/limited signal. My Ting (Sprint MVNO) phone loses the signal if I go inside many strip-mall stores like CVS even when people on other networks can get 3-4 bars. When my mother rode to/from Oregon to visit family for New Year's, she couldn't get any signal from Sprint/Ting in-state once she left the Bay Area.

Unless I'm missing something, isn't this just a typical engineering trade off where ATT favored data speeds over voice calls, Sprint took the opposite approach and Verizon was somewhere in the middle?

I'm on Sprint and would rather have voice calls and 1Mbps unlimited than 100Mbps with a cap. Am using over 10GB/month sometimes much more, that would be expensive on ATT/VZW. Granted, I'm in a completed Sprint area. Typically get 6-20Mbps, which is fine as no smartphone app needs more than a few Mbps.

Just so I understand correctly. You are saying that you are a person that prefers quantity over quality?

Why is this even News. Carrier setup dedicated network for event...erm soooo...

If Vodaphone or O2 setup a dedicated network for my work i could get good service as well. This surely is what they are meant to do give good service. If their service is sooo bad that this is news then maybe y'all need to talk to your congressmen - or whatever you American's have.

Sounds like typical Sprint. Except where I live (major city in Texas), I can't even place phone calls some days because the network is so overloaded in the downtown area that I live in. It is atrocious.

I'm with Nexgen Wireless, we conducted the tests. The reason we don't normally test MVNO's is that it's not a true representation of the network performance since they utilize the tier 1 carrier's network.

I know it's pie in the sky, but I would love if they had included a random prepaid MVNO on each network in the tests as well. Just to see if there is any credence to the rumor that MVNO customers are second class citizens.

Just a question about the test parameters, maybe you would know the answer. When we do data call testing we generally have a timeout point, so that if the data call fails to start traffic within a certain time limit we mark that call as a failure and for the purpose of average throughput that contributes a 0Mbps to the average.

In your testing do you use something like this to penalize spotty connectivity? Or is the average listed the average over successful data calls (and then it seems like a separate metric for failed data calls is needed.)

True but since AT&T's data was faster on average, they obviously had a bigger backhaul.

That's assuming the bottleneck was the backhaul, and not the wireless spectrum itself.

An oversubscribed backhaul makes it slower for everyone, oversubscribed channels get to a point where phones can't do anything.

I'm betting it was the latter.

I would bet my house on the bottleneck being the OTA(over the air) channel. Designing a network layout to minimize interference but maximize the number of nodes is really difficult.

The network layout and frequency reuse strategy is really important in a venue like this, you can expect hundreds of simultaneous data requests, which would quickly saturate a single channel of any RAT (radio access technology). So the network architects will use multiple disjoint channels to maximize frequency use, they will use directional antennas to split up the venue spatially allowing for frequencies to be reused on separate nodes in different parts of the stadium. And they will maximize the number of concurrently active RATs in the stadium (because each RAT has a different band generally) so that fall back is available to cover as many users as possible.

Overall I think this last point is possibly where ATT won, HSPA fallback from LTE is still pretty good data rates and ATT has good spectrum for multiple channels of both LTE and H.

Unless I'm missing something, isn't this just a typical engineering trade off where ATT favored data speeds over voice calls, Sprint took the opposite approach and Verizon was somewhere in the middle?

I'm on Sprint and would rather have voice calls and 1Mbps unlimited than 100Mbps with a cap. Am using over 10GB/month sometimes much more, that would be expensive on ATT/VZW. Granted, I'm in a completed Sprint area. Typically get 6-20Mbps, which is fine as no smartphone app needs more than a few Mbps.

I guess this explains why sprint have customers , because people be bad at understanding what they are buying.

Yes we have this six course meal that is $89,99! Or we have this sprint bag of dog poo for $49.95.