95Mbps may be the NBN’s real-world limit

Note: This article has been updated with comments from NBN Co since it was first published.

news Early trials of National Broadband Network fibre connections through several different Internet service providers have shown speeds topping out at 95Mbps, five percent slower than the NBN’s overall theoretical limit of 100Mbps.

In a post on his blog yesterday, Exetel chief executive John Linton, whose company was one of the first to go through NBN Co’s on-boarding process and be ready to provide services over the network, revealed that the first dozen or so of Exetel customers had now had their NBN services connected and were able to use their connections.

“A surprisingly large percentage of the trial orders are for 100Mbps/40Mbps services, and of those we have contacted/have contacted us, the service is delivering around 95Mbps down and 32Mbps up, with a 10ms ping time,” he said. “As the NBN Co network would currently be as under-utilised as it’s ever going to be those results seem likely to be the maximum achievable in the near term.”

The news almost precisely mimics the results achieved by PCRange chief executive Raaj Menon, who moved suburbs in Adelaide in order to make sure he could get the NBN early.

In late June, after his NBN fibre service was connected, Menon noted in a statement issued by his service provider, Internode, that he was achieving speeds of 95.5Mbps downstream and 35.5Mbps upstream, with ping times of about 3ms. In a separate blog post, Menon added that he ran a variety of speed tests, average around 94 to 95Mbps down and 35Mbps up. “It is close enough to the advertised speeds,” he noted.

A NBN spokesperson responded to the issue by noting that the company was building a network designed to achieve downlink speeds of up to 100Mbps at the wholesale Layer 2 (Ethernet) level.

“Most speed tests operate at the Layer 3 (Internet Protocol) level and the extra packet overhead will typically result in lower measurement speeds (depending on the type of traffic),” they said. “We point out that the speeds actually experienced by end-users will depend on a number of factors including, in part, protocol factors, the retail broadband plan they choose, their ISP’s network capacity, the number of simultaneous users on the service, the server’s equipment and configuration, their equipment and their in-premises connection.”

In a recent blog post, Internode chief executive Simon Hackett noted that the ISP’s experiences in Tasmania showed that each point of interconnect region located around Australia would need at least 200Mbps of capacity supplied to it (which NBN Co refers to as the ‘Connectivity Virtual Circuit’) in order to ensure most customers would be able to access the maximum speeds under their NBN connections when they wanted to use them.

In short, that capacity is shared between customers connecting to a certain point of interconnect.

Other typical technical limitations on network links include factors such as the speed of serving data from storage, or other, non-NBN links in between the data being downloaded and a customers’ own PC. Inefficiencies in a customers’ own router can come into play.

opinion/analysis
While this sort of theoretical limit on NBN speeds isn’t really unexpected (I mean look at ADSL — ‘theoretically’ it allows up to 24Mbps, but few actually get those speeds), it is interesting to see such similar results across ISPs. Is there some kind of 95Mbps bottleneck in the NBN infrastructure which is impossible to push past? It seems there could be many factors to consider. In the meantime, of course, I personally — and I suspect almost everyone else — would be quite happy to get “only” 95Mbps speeds … at least until the gigabit upgrade eventually comes along ;)

And yes of course it’s overhead. Have you tried sending a letter through the post without the envelope with the address on it? The post office won’t have a clue where to send your letter to. The envelope itself is an overhead.

There’s a difference between theory and real world. Some of it hinges on data throughput vs bandwidth. With NBNCo running a layer 2 network, there would be some management traffic running throughout, but whether that’s enough to limit connections by 5 Mbit is only something that NBNCo and a whole raft of sniffing would be able to confirm. I wouldn’t expect NBNCo to build a complete out of band seperate network for all their network management traffic without adding significantly to the hardware costs, so it’s most likely (as only NBNCo can confirm or deny this) that they are running their management traffic on the same hardware, but in a seperate VLAN.

However, the good news, is that said management traffic wouldn’t increase significantly even if the amount of customers increase (although happy to be corrected by those more knowledgable in MPLS than little old me).

But this doesn’t make any sense. Look at Verizon’s FIOS network in the US. Verizon overprovisions so that customers almost always get 110-115% of advertised speeds. It’s really not that hard to do on a fiber network. Given that the CVC charge is completely arbitrary to begin with, NBNCo should alter the charges and/or rebate so that overprovisioning is encouraged. Make sure everyone is super satisfied and the network will become even more popular.

That doesn’t automatically mean that NBNCo will though. Hopefully they’ll offer an explanation.

It’s not difficult to do, but given the nature of the NBN, that is, a build to 93% of the population of Australia initially, perhaps they’ve tried to scrape as close to the line as possible to keep costs down (something that Quigley has always stated that he was aware of, and endeavouring to meet).

Hanging for NBNCo’s answer, if it’s forthcoming, and as I said below, it may not even be the NBN that’s the issue.

The real cost is in the labor, not the bandwidth. And for the record I don’t think it’s an “issue”. 95% is still really good. It’s just a nitpick and I think a really great PR move for NBNCo would be to alter the CVC charges and rebates to encourage ISP overprovisioning to make sure everyone is extremely happy with their connections.

Of course Verizon don’t sell their FTTH product to other ISP’s either, it is their own infrastructure in limited areas sold as a package with their phone, internet and Pay TV packages.

As a vertically integrated monopoly they have complete control over all aspects of the links at all physical layers and the servers, to compare it with the wholesale only NBN in Australia is ludicrous.

Once the lines are in the ground the cost of bandwidth is negligible. It doesn’t cost more to send pulses of light or electricity to a rural area than it does an urban area. The question is whether NBNCo wants to encourage overprovisioning by the ISPs, which it can do by altering the CVC pricing structure. It’s not a big deal, but I would think it’d be great PR.

You’ve touched on some questions that still need to be asked. How do we know the NBN is the problem here? If it’s a simple FTP download, it could be the ISP’s infrastructure that’s the bottleneck, not the NBN.

Anything above layer 2 is suspect, as that is all that the NBN is. It’s impossible for an end point user to determine whether the NBN is a bottleneck, as you’d have to sniff at both ends of the NBN infrastructure (i.e. customer end node, and NBN exit point to ISP infrastructure) to determine whether it is actually their network that’s the issue.

You’ve touched on some questions that still need to be asked. How do we know the NBN is the problem here? If it’s a simple FTP download, it could be the ISP’s infrastructure that’s the bottleneck, not the NBN.

Anything above layer 2 is suspect, as that is all that the NBN is. It’s impossible for an end point user to determine whether the NBN is a bottleneck, as you’d have to sniff at both ends of the NBN infrastructure (i.e. customer end node, and NBN exit point to ISP infrastructure) to determine whether it is actually their network that’s the issue.

The problem with any network speed is “where do I measure it”? Speedtest sites measure it based upon delivered payload. There is packet headers and synchronization signals to consider.

NBN could be measuring the raw amount of information on the network, or, slightly better, payload of the tagged packets. Ever way there will be a small, but consistent, overhead. And unfortunately the more bandwidth you push, the more obvious this overhead becomes.

NBNCo could do an overhead adjustment where they offer 105% or 110% of advertised speeds, but if I were them I wouldn’t bother.

They should overprovision though. The feeling of satisfaction gained from seeing your connection faster than you thought it would be is unmatched. Not to mention the massive PR boost with people going around saying “I’m getting *more* than I paid for. Wow.”

A speed test produces a result by downloading “data” and dividing it by the time taken. This gives an *average* download speed for the entire chunk of data.

One very, very important factor is that this data is only delivered inside an ethernet frame, which contains a IP packet, which contains a transport segment, which contains “data”, which itself may be further encapsulated. (OSI network model).

Technically the result is called “goodput” – thats the average data delivery rate, less the various overheads, such as the IP packet headers.

NBN deliver 100Mbps ethernet – thats a 100Mbps Layer 2 service. Customers have no way of testing that. They can perhaps test Layer 3 throughput if they can control both ends (which they dont) if they have sufficiently complex testing tools – which, again, they typically dont. So the best thing they can usually do is rely on speedtest.net which relies on a test produced inside a web browser. FTP is a better choice.

There is also TCP slowstart, where a TCP session – used in speed tests – ramps up its throughput until it reaches congestion, then it backs off and tries again, eventually leveling out at the point *just below* the congestion point.

(The congestion point is where either latency increases across the link or pack drops start to occur).

I’ve left your comment up as I think you raise some very interesting technical points. However, I will note that it is our policy on Delimiter to be polite and that describing an article as “appalling” in the first case is not very polite.

Personally, I am pretty thick-skinned; you’d have to be to write articles as controversial as I do, about topics which are as contentious.

However, I am determined to maintain a level of ‘politeness’ on Delimiter to ensure readers are not scared away. At times I have received a lot of feedback from people that they feel scared to comment on Delimiter for fear of having their head bitten off.

I want to ensure Delimiter does not descend to the level of other sites and become a cesspit of bile. This is not 4chan ;)

I think Brad’s point was that the author’s lack of knowledge led to the article painting the NBN as vaguely deficient, when really this is an excellent result, almost the best it could be. “NBN Far More Efficient Than ADSL” might be a more appropriate headline.

I thought Brad pointed out the article’s weaknesses quite well. “Appalling” might be considered a strong word (though hardly 4chan level), but I suspect he is a little tired of the misconceptions that un-researched journalism leads to, especially in controversial topics.

“The Line Rate sets the maximum bound on the information-carrying capacity of the link. Customers are advised that they should be familiar with the inherent limitations of Ethernet in relation to the impact of framing overhead and asynchronous operation on bandwidth efficiency, and accommodate this within any NFAS capacity allocation.”

This is totally normal when normal framing overheads at multiple levels are factored in.

The services concerned are working fine.

When you buy a ‘1000GB’ hard drive and format it, you don’t see ‘1000GB’, you see a lower number, that factors in normal formatting overheads. No emergency, nothing wrong.

On ADSL services, the overheads there mean that throughput in terms of payload (after ATM cell overheads, TCP and IP overheads, and PPP framing overheads) comes out around 83-86% of the ‘theoretical’ number. This is something we observed ten years ago and it remains the case today (e.g. a ‘1500’ ADSL connection delivers around 1300 kilobits of payload rate after those overheads).

Summary: It would be great if you could follow this article up with a well researched piece about how this is a normal and reasonable performance expectation on the NBN when all normal Ethernet framing, PPP framing, IP header overhead and TCP header overheads are factored into normal data transfers. Customer expectations are important to properly manage – and framing the 100M NBN as somehow being deficient in delivering only a 5% aggregate framing overhead in total (compared to a demonstrated overall 15% or so overhead in ADSL networks) seems highly unfair in terms of how it paints the NBN.

Think how differently you could have painted this, if you had said, for instance:

“NBN proven to be far more efficient than ADSL networks, and with dramatically lower latency to boot!”

I don’t think this was unexpected … as I noted in the article: “While this sort of theoretical limit on NBN speeds isn’t really unexpected”. I also don’t think it’s a political issue or anything like that. I just think it’s interesting to see what the NBN can do in real-world conditions :)

I’m aware of network overheads and the different layers of the network stack etc. However, so far discussion about these haven’t been part of the discussion about overall NBN speeds. This article was an attempt to tease some of that out, as has happened in the comments. I’m quite happy that it has :)

I wasn’t trolling, and I find your comment that I may be “undermining” myself as a tech journo impolite. I was reporting and commenting on an interesting aspect of the NBN’s performance, which I think is a valid thing to do :)

“However, despite the pricing uncertainty, Exetel will send out invitation emails to its customers in early stage NBN rollout zones in NSW and Victoria, offering them a free NBN fibre plan install with no charges for using it until September 30th this year. The Exetel trial will allow customers to keep their existing ADSL service and use both the ADSL and the new fibre network side by side, while continuing to pay only for the ADSL service.

“On or before 30th September they select which service they would like to keep,” wrote Linton. “If they don’t want to continue with the fibre service then they are not charged for it to be removed and they simply go back to using their ADSL service at their contracted price per month.”

please note I have received the following comment from NBN Co on this matter:

“NBN Co is building a network designed to achieve downlink speeds of up to 100Mbps at the wholesale Layer 2 (Ethernet) level. Most speed tests operate at the Layer 3 (Internet Protocol) level and the extra packet overhead will typically result in lower measurement speeds (depending on the type of traffic). We point out that the speeds actually experienced by end-users will depend on a number of factors including, in part, protocol factors, the retail broadband plan they choose, their ISP’s network capacity, the number of simultaneous users on the service, the server’s equipment and configuration, their equipment and their in-premises connection.”

Who are these idiots who write these articles, please guys go and ask a TAFE student studying their CCNA and they will be able to help you write an accurate, non biased article….hang on, I was assuming you were trying to be fair and reasonable….my mistake.

100mb Ethernet will NEVER achieve 100meg/sec. As a lot of the above posters have noted, there are so many factors. I’d wager that even on a LAN between 2 PCs you will never achieve 100/sec with 10/100 Nics.

The reality is that most of the content is in the USA and the Round Trip times mean that a protocol like TCP will never achieve anything like 100/sec. You just cannot make light travel faster than it already does.

TCP was designed to be a reliable, connection oriented protocol, and it was put together years ago. TCP Windowing goes some of the way to help, but the reality is that as soon as you loose 1 packet you take a 50% performance hit immediately.

Very interesting re Bob @ Midway Point. If that’s true, it shows the potential difference the right CPE can make. 7Mbps isn’t something to laugh at in terms of a difference between peak speeds. That’s half of what most of us are getting on ADSL total right now.

Yes, there are good CPE and bad CPE. But this conversation has *nothing* to do with CPE.

The take-away message here is the speedtest.net is inaccurate. There is no possible way that he actually achieved 102Mbps.

There was a Today Tonight episode that showed Bob’s Cisco 881 router. It has 100Mbps ethernet ports – both on uplink and downlink. Also, clearly, based on lots of test results and NBN’s own tested specifications, they do 100Mbps of ethernet, not 100Mbps of IP traffic.

NBN do not over provision. I’ve tested that in multiple NBN areas and they simply do not.

Yes, there are good CPE and bad CPE. But this conversation has *nothing* to do with CPE.

The take-away message here is the speedtest.net is inaccurate. There is no possible way that he actually achieved 102Mbps.

There was a Today Tonight episode that showed Bob’s Cisco 881 router. It has 100Mbps ethernet ports – both on uplink and downlink. Also, clearly, based on lots of test results and NBN’s own tested specifications, they do 100Mbps of ethernet, not 100Mbps of IP traffic.

NBN do not over provision. I’ve tested that in multiple NBN areas and they simply do not.

So not only do you not provide info in regards to all the issues of benchmarking (header overheads, consumer equipment, dropped packets etc), which is somewhat understandable, you also forget to actually provide some adsl2+ numbers for analysis. (E.g. my 24Mbps connection currently syncs at 14Mbps yet speedtest just said I had 12.5Mbps).

However my biggest concern, was that you did not make clear that 100Mbps is /NOT/ the top offering by NBNco! NBNco have quite clearly outlined plans up to 1Gbps in their documentation. So any such nonsense about ’95 Mbps may be NBN’s real world limit’ is clearly garbage.

So yeah, if your going to step into a volatile debate, and make strong claims without doing much research your going to get some flack. I don’t think you did it intentionally, but hope you would seriously reconsider writing something like this again without more info, or a big disclaimer.

So this is what happens when journalists with no technical background comment on technical matters. Next time you feel like writing an article about something you dont understand, do some goddamn research.

I don’t disagree with the basis of the article, but the headline looks like it has been generated by something The Australian would use. We all know that there are certain segments of Australian political/media circles who are trying to sink the NBN and you are doing the job for them!

It doesn’t matter than the technical details about overhead etc. are in the article itself – the people mentioned above doesn’t understand and doesn’t care.

Another thing to note is in the Ethernet specification the is a requirement for a minimum idle time between the transmission of each Ethernet frame and this eats in to your usable bandwidth. The spec is set at 96 bit times, which works out to be roughly 12 bytes for Fast Ethernet IIRC

OMG. One needs to learn Networking 101 before they can make such opinions.

As has been mentioned several times. There is a reason that only 95mbps was achieved. This is not due to over subscription it is due to the way networks are built. More specifically, this is a fundamental element of IP networking.

Every packet sent on the wire is constructed of 3 parts.

|Ethernet MAC|-|IP header|-|Data payload|

As you can see, there are 3 parts to a packet (I am grossly simplifying this)

The MAC portal takes a bit of data, the IP portion also takes a bit and lastly, the Data portion is the user data.

ALL speed test programs out there measure speed based upon how fast they receive the “DATA” portion.

Given that they do not measure how fast the entire packet including MAC and IP header is being received at, you are actually only ever going to get a slightly less speed than the link speed.

If you want a practical demonstration, download IPerf and perform a test between two workstations/computers on the same network switch and you will find that you will only achieve at maximum 95mbps (or around there)….In fact Windows is sooo badly built that most tests with IPerf will result in measurements of around 80mbps.

This is all because the operating system performs packet de-encapsulation (strips away the MAC and IP portion of the packet) before it gives it to the application. As such, you may see 100mbps of traffic entering your computer, but the program will only ever report something less than that.

The only way to test true speed of the link is to use very very expensive Layer 2 or even Layer 1 equipment. This stuff from companies like Fluke will actually measure the speed at which the ones and zeros appear on the interface. It doesn’t measure how fast the data come in. It actually measures the everything up to and including the all headers in the packet (mac and ip headers)

If you are accessing the NBN with a router that only supports 100Mbps ethernet, of course you are not going to pull 100Mbps from the net! There are LAN overheads as well. On a router that supports 1000Mbps ethernet then you shouldnt have any problems at all.

Comments are closed.

Book now available

Written by Delimiter Publisher Renai LeMay, The Frustrated State is the first in-depth book examining of how Australia’s political sector is systematically mismanaging technological change and crushing hopes that our nation will ever take its rightful place globally as a digital powerhouse and home of innovation.

Welcome! We were an energetic and engaged community of Australians who worked with or who were interested in technology -- all sorts of IT professionals, IT managers, CIOs, tech policy-makers and tech enthusiasts.