FedEx's file-transfer capacity versus the Internet

Today on XKCD's "What If...?", Randall Monroe runs the numbers of when and whether the Internet's throughput will ever exceed FedEx's sneakernet file-transfer capacity (one interesting note here: why not treat FedEx's trucks and planes full of hard-drives and SD cards as part of the Internet? After all, you book your FedEx pickup over TCP/IP, track it over TCP/IP, and pay for it over TCP/IP).

Cisco estimates that total internet traffic currently averages 167 terabits per second. FedEx has a fleet of 654 aircraft with a lift capacity of 26.5 million pounds daily. A solid-state laptop drive weighs about 78 grams and can hold up to a terabyte.

That means FedEx is capable of transferring 150 exabytes of data per day, or 14 petabits per second—almost a hundred times the current throughput of the internet.

If you don’t care about cost, this ten-kilogram shoebox can hold a lot of internet

We can improve the data density even further by using MicroSD cards:

Those thumbnail-sized flakes have a storage density of up to 160 terabytes per kilogram, which means a FedEx fleet loaded with MicroSD cards could transfer about 177 petabits per second, or two zettabytes per day—a thousand times the internet’s current traffic level. (The infrastructure would be interesting—Google would need to build huge warehouses to hold a massive card-processing operation.)

So the interesting thing here is the implicit critique of cloud computers. Leave aside the fact that a cloud computer is like a home computer, except that you're only allowed to use it if the phone company says so.

Instead, consider for a moment whether streaming -- especially wireless streaming -- of media that you're likely to play more than once makes economic or technological sense. Your hard drive brims with capacity. It costs nothing to use (after you've paid for it once, and leaving aside the electrical bill). It is vastly faster than any wide-area network link. Contrast with wireless bandwidth: there's only one RF spectrum, and you have to share it with everyone within range of your device.

Increasing the hard drive in your laptop does nothing to the storage capacity of my laptop, but increasing your demands on the wireless spectrum comes at the expense of my use of that same spectrum.

On the other hand, it's easy to see why telcos would love the idea that every play of "your" media involves another billable event. Media companies, too -- it's that prized, elusive urinary-tract-infection business model at work, where media flows in painful, expensive drips instead of healthy, powerful gushes.

The progeny of this hellish marriage is the non-neutral Internet connection where a telco offers to spy on, and slow down, its users' connections -- but selectively, so that the media from a "preferred partner" comes in more quickly, and doesn't count against a bandwidth cap. This is especially virulent where telcos are entertainment companies -- Comcast and Rogers and so on, all champing to freeze out services like Netflicks by metering its bandwidth, but freeflagging downloads from their in-house competing services.

35 Responses to “FedEx's file-transfer capacity versus the Internet”

A quick glance shows that the act of transferring all the data on and off of the drives is missing from the equation. This is the most time consuming part of process and leaving it out really skews the results.

You never need to access all the info at once. Accessing what info you need from a local hard drive is incredibly faster than accessing that from the cloud. Transferring 1TB from one hard drive to another takes what, an hour? How long would it take you to upload 1TB to a cloud storage drive?

With a really good consummer SSD hard drive you can maybe get 400MB/s write speed, with good-ol’ platter-spinning hard drives or micro-SD cards you’ll be lucky if you get a 1/4 of that : way faster than your home ISP, but if you got the money you can already buy a faster pipe to the net.

Again you are totally ignoring the real world. How are these hard drives hooked up and interfaced? How are they packed and taken to the airport? There are tons of real world issues that are totally ignored.

Exactly. Especially with the example of the milk jug full of 25,000 micro SDs. How do you find the one correct micro SD that contains the file you want? And then you have to put it in a card reader and do the actual file transfer. You need an excellent indexing and card storage system.

It wouldn’t be prohibitively difficult to make a system that turns all those cards into a giant RAID array. In a properly designed system you wouldn’t need to keep track of which card goes where. But then you’d just have to have thousands of card readers on the host machine. Of course, that would require multiple USB controllers because of the addressing limitations of USB.

If only FedEx would publish an API and put a user-defined QR code on the waybill, we could make them a transparent part of the fabric! Get rid of the human element. This would be a great way to back up data centers to locations not susceptible to earthquake, hurricane, etc. and would be a very viable way to create a community-driven global digital delivery network for indie films from studio directly to theaters and homes. I started tongue in cheek but actually it makes a lot of sense! You could have your local 7/11 (which already handles packages and has fiber, net payment kiosks and networked registers where I live) act as intermediate shipping points and copy stations. Just reserve a copy and you can pick up a hard drive or envelope of sd cards within hours after it has arrived at your local hub. You could even work a payment system out with 7/11 either through them, or any of a number of e-payment services (you print a receipt from the net kiosk and then pay at the register). Heck iTunes sells debit cars there next to the shampoo and eyeliner too, why not everyone else? Hmm I’m thinking this is an interesting idea.

Funny you say that because for infrequent users, a brick and mortar blockbuster pay-per-use model is cheaper than the more “advanced” online subscription based provider. User consumption pattern is a big determining factor.

I’m saying there is no reason to clog up satellites and undersea cables when delay is acceptable, or you have a local cache. There is a use-case for high latency, high volume data distribution such as 4K video.

I’m not living in another age. Though I have been lucky enough to be part of some experiments along these lines, such as doing the first live streaming of motion capture data between California and Japan with Silicon Graphics, and getting involved in Carl Malamud’s Internet World Expo which built a 45Mbps Internet Railroad around the world in 1996.

More recently (at the past few years of the Tokyo International Film Festival) I have seen demonstrations of live global 4K video, from Cinegrid.org (see http://cinegrid.kmd.keio.ac.jp/english.html ) which for example enables live connections between studio and filming location, or CAVE connections for education. Obviously these are fabulously resource hungry and only usable for limited use-cases. Whereas the world’s P2P bandwidth or Fedex bandwidth offer alternatives. And how does Hollywood distribute their movies globally? I believe by fedexing a hard drive. Sure, streaming is okay. Unless you are the Australian government with a soda straw for a pipe, or want to actually send a high enough quality file to play in a theater or other big screen.

I’m saying it is prohibitive for any one producer to start mailing hard disks to every country in the world, but if there was an easy to board Internet railroad, or you could think of it as chucking your hard disk into that fabled station wagon as it drives by, just get your content to a single node and would be replicated by fedex and friends and that would be a fabulously enabling global resource.

Think of it another way: Usenet is over 10TB/day. http://www.newsdemon.com/average-feed-size.php

Now consider automatic cascading updates – a petabyte sized repository backing a huge, constantly updated library of 10,000 blue-ray quality (50GB) video titles. That’s 500TB. You don’t have to view every one. But it might be a good use for a town library too, wouldn’t it? And if half of it is commercial content, a small fee could pay for the servers. The media (64GB SD cards) is selling for $50 on Amazon. So it makes sense to think of what kind of an application could make use of postal bandwidth (doesn’t need to be fedex) and I think this might be one of them.

As another data point I was approached last year by a documentary film director who had been tasked at building a very low-cost cultural library of documentary films. His idea was to make a library of USB sticks that fold into a credit card sized piece of plastic, onto which you could affix a printed label like a record cover. A small bookcase would be enough to display a hundred cards – each full of documentaries on a given subject area – which could be copied freely by visitors or viewed there. It seems that postal bandwidth is the only way to maintain an offering like this, and I’m thinking of helping implement it. It would probably be for free or close to it, but could be a useful resource.

Fedex does publish an API – thousands of businesses use it to ship through them – and the tracking number is encoded as a standard bar code that’s inherently machine readable. No need for a QR code when there’s already a bar code on there identifying the package and tracking its location.

Reminds me of an information systems lecture I had many many years ago involving calculating the data rate of sending a transport aircraft across the Atlantic carrying floppys. Was actually surprisingly high in the hypothetical situation we were given at the time versus traditional internet traffic at the time.

I guess the main thing was trying to make sure we weren’t too hung up on data and information always meaning electronics or photonics.

Once a year or so we have to move vast chunks of data between servers at work, allowing us to retire old machines and get users working on the new ones. Our smallest machines these days are in the 16TB range, the newest ones are over 100TB.

At any rate, migrating this data is no fun. We have bonded gigabit connections all over the place and never get to use all of it because hard drives are just too damn slow at reading out data. Anecdotally, the best transfer speeds I see are about 2TB copied per day. If either end of the copy is doing other work, rates rapidly go down from there.

Gonna be a good day when we stop relying on spinning sheets of metal to store our data.. we might actually be able to saturate the network then!

The only issue here is that there are legal ways to rip my DVD movies but there is none for BluRay movies–that I am aware of. Last I checked it was still illegal to rip them, even though the process is technically exactly the same as the DVD movies.

Please, someone enlighten me on if there has been any development in this area. I would so gladly spend the next month ripping my movies and buying 4TB hard drives if it were legal.

Technically yes, but the trials that I have been able to find have ruled in favor of the person and not the MPAA. So I feel fairly safe with that, seeing as I still have my entire DVD collection in safe storage in the attic and I only watch them on my own devices.

From what I understand, BluRay discs use another type of encryption that is protected under something more than DVD’s. That is why it is still a bigger NO-NO

Many years ago, when Netflix was in the DVD-by-mail rental business rather than the download business, somebody estimated that their bandwidth was about 1/3 of the total Internet’s. 4 GB/day is about 300 kbps for an individual user, and Netflix was shipping a lot of DVDs.

Tokyo. Though I could be mixing it up with FamilyMart or Lawson their arch-rivals. 7/11 has awesome 7 and i Holdings Bank atms which can withdraw from credit cards even. They also (unless I am thinking FamilyMart) have Internet-connected copying machines so you can upload PDFs and print in A3 color – or use a USB stick. The kiosk machines let you access a ton of services including international phone card purchase, e-money, software purchases, and reservations for concerts and events. You can also pay your electric and gas bills at these places and at least some (am/pm) let you ship boxes via popular couriers. Not available in the U.S.?