Forecast | The digital storage revolution: Future of Computers P2

Most of you reading this probably remember the humble floppy disk and it’s solid 1.44 MB of disk space. Some of you were probably jealous of that one friend when he whipped out the first USB thumb drive, with its monstrous EIGHT MB of space, during a school project. Nowadays, the magic is gone and we’ve become jaded. One terabyte of memory comes standard in most 2015 desktops—and for all you digital hoarders out there, Kingston even sells one terabyte USB drives now.

Our obsession with storage grows year over year as we consume and create more digital content, whether it’s a school report, travel photo, your band’s mixtape, or a GoPro video of you skiing down Whistler. But just like the microchips we spoke about in the first part of our Future of Computers series, digital data storage is driving straight into a wall.

Over the course of this series chapter, we’ll explore the innovations set to dominate the data storage industry over the coming decades; we’ll discuss the rise of cloud storage and why you may never again need to own a USB in five years time; and we’ll explain how all these trends will come together to redefine how you use computers by the early 2020s.

Data Storage Innovations in the Pipeline

It’s physics really: There are limits to the capacity we can squeeze into and the performance we can squeeze out of traditional hard disks using existing designs and manufacturing processes. But fear not, science is coming to the rescue. The following list is a brief glimpse into the near- and long-term innovations hard disk drive manufacturers will use to satisfy our storage-hungry society. (FYI, this part is a bit dense, so you’re totally welcome to skip over it if you’re not in the mood to geek out.)

Solid state hard drives. Replacing the traditional hard disk drive noted above is the solid state hard drive (SSD). Unlike HDDs, SSDs don’t have any spinning disks—in fact, they don’t have any moving parts at all. This allows SSDs to operate far faster, at smaller sizes, and with more durability than their predecessor. SSDs are already on the market, but still far more expensive than HDDs; however, with their price falling faster than HDDs, their sales could overtake HDDs by the mid 2020s

Flash memory goes 3D. Hard disks can be compared to your long-term memory, whereas flash is more akin to your short-term memory. And just like your brain, a computer needs both types of storage to function. Flash memory, commonly in the form of random access memory (RAM) chips, is running into the same physical constraints hard disks are facing. Worse, the tinier transistors become inside RAM, the worse they perform over time—the transistors get harder to erase and write accurately, eventually hitting a performance wall that forces their replacement with fresh RAM sticks.

In light of this, companies are beginning to build the next generation of flash memory. There are four main contenders competing for that mantle. In order of existing industry support and investment:

3D NAND. Companies like Intel, Samsung, Micron, Hynix, and Taiwan Semiconductor are pushing for the wide-scale adoption of 3D NAND, which stacks transistors into three dimensions inside a chip.

Phase Change Memory (PCM). The tech behind PCMs basically heats and cools chalcogenide glass, shifting it between crystallized to non-crystallized states, each with their unique electrical resistances representing the binary 0 and 1.

Spin-Transfer Torque Random-Access Memory (STT-RAM). A powerful Frankenstein that combines the capacity of DRAM with the speed of SRAM, along with improved non-volatility and near unlimited endurance.

Regardless of which option wins out, all of these new forms of flash memory will offer more memory capacity, speed, endurance and power efficiency.

Software-Defined Storage Infrastructure (SDS). It’s not just storage hardware that’s seeing innovation, but the software that runs it is also undergoing exciting development. SDS is used mostly in large company computer networks or cloud storage services where data is stored centrally and accessed through individual, connected devices. It basically takes the total amount of data storage capacity in a network and separates it among the various services and devices that run on the network. Better SDS systems are being coded all the time to more efficiently use existing (instead of new) storage hardware.

Will We Even Need Storage in the Future?

Okay, so storage tech is going to improve a whole lot over the next decade. But the thing we have to consider is, what difference does that make anyway?

The average person will never use up the terabyte of storage space now available in the latest desktop computer models. And in another two to four years, most smart phones will have enough storage space to horde a year’s worth of pictures and videos without having to spring clean your device. Sure, there’s a minority of people out there who like to horde massive amounts of movies and pictures on their computers, but for the rest of us, there are a number of trends reducing our need for excessive, privately-owned disk storage space.

Streaming services. Once upon a time, our music collections involved collecting records, then cassettes, then CDs. In the 90s, songs became digitized into MP3s to be hoarded by the thousands (first through torrents, then more and more through digital stores like iTunes). Now, instead of having to store and organize a music collection on your home computer or phone, you can stream an infinite number of songs and listen to them anywhere through services like Spotify, Songza, and Apple Music.

This progression first reduced the physical space music took up at home, then the digital space in your computer. Now it can all be replaced by an external service that provides you with cheap and convenient, anywhere/anytime access to all the music you could want. Of course, most of you reading this probably still have a few CDs lying around, most will still have a solid collection of MP3s on their computer, but the next generation of computer users won’t waste their time filling their computers with music they can access freely online.

Obviously, copy everything I just said about music and apply it to film and television (hello, Netflix!) and the personal storage savings keep growing.

Social media. With music, film, and TV shows clogging up less and less of our personal computers, the next largest form of digital content is personal pictures and videos. Again, we used to produce pictures and videos physically, ultimately to collect dust in our attics. Then our pictures and videos went digital, only to again collect dust in the nether reaches of our computers. And that’s the issue: Most of the pictures and videos we take, we don’t look at them daily—we take pictures and videos, then look at them months or years later.

Luckily, soon after our memories became digitized, social media happened.

All of a sudden, sites like Flickr and Facebook gave us the ability to share an infinite number of pictures with a network of people we care about, while also storing those pictures in a self-organizing folder system or timeline. While this social element, coupled with miniature, high-end phone cameras, greatly increased the number of pictures and video produced by the average person, it also reduced our habit of storing photos on our private computers, encouraging us to store them online, privately or publicly.

Cloud and collaboration services. Given the last two points, only the humble text document (and a few other niche data types) remains. These docs, compared to the multimedia we just discussed, are usually so small that storing them on your computer will never be a problem.

However, in our increasingly mobile world, there’s a growing demand to access docs on the go. And here again, the same progression we discussed with music is happening here—where first we transported docs using floppy disks, CDs, and USBs, now we use more convenient and consumer-oriented cloud storage services, like Google Drive and Dropbox, which store our docs at an external data center for us to access securely online.

Services like these allow us to access and share our docs anywhere, anytime, on any device or operating system. In fact, recent brands of laptops, like Google’s Chromebook, are designed to only function while connected to the web; they contain very little disk space and store its owner’s content entirely in the cloud.

To be fair, using streaming services, social media, and cloud services doesn’t necessarily mean we will move everything to the cloud—some things we prefer to keep overly private and secure—but these services have cut, and will continue to cut, the total amount of physical data storage space we need to own year over year.

The Future of Business Is in the Clouds

By 2017, 73 per cent of the world’s data will be housed in the cloud, while generating over 20 exabytes of personal cloud traffic. (You can check out this New Jersey Institute of Technology infographic of see how pervasive cloud services have become and how fast they’re set to grow.) But just as our increasingly digital culture is pushing us to move our data to the cloud, similar pressures are pushing businesses of all sizes to do the same.

Outsourcing costs. While it’s true the cost of storing data falls year over year, the electricity, maintenance, security, and labour costs of maintaining massive data centers to store said data can still be quite hefty. With stock market pressures forcing public corporations to cut operating costs wherever possible, these businesses are turning to large cloud storage services to store the majority of their company’s data.

Juggernauts like Microsoft Azure, Amazon Web Services, and Google Cloud are all expecting to see huge growth in their businesses, as more and more companies outsource their digital storage needs to companies that already specialize in the latest in cost-effective data storage tech and practices.

Big data. Just as computers consistently grow exponentially more powerful (so far), so too does the amount of data our global society generates year over year. We’re entering the age of big data where everything is measured, everything is stored, and nothing really ever gets deleted.

This mountain of data presents both a problem and an opportunity. The problem is the physical cost of storing ever larger amounts of data, accelerating the abovementioned push to move data to the cloud. Meanwhile, the opportunity lies in using advanced software to discover profitable patterns inside that data mountain

For example, Google uses its mountain of search engine data to not only offer you the best answers to your everyday questions, but to serve you ads tailored to your interests. Uber uses its mountain of traffic and driver data to generate a profit off of underserved commuters. Select police departments worldwide are testing out new software to track various traffic, video, and social media feeds to not only track down criminals, but predict when and where crime is likely to occur to preemptively stop it, Minority Report-style.

Software as a Service (SaaS). Aside from outsourcing the costs of storing big data, more and more business services are being offered exclusively over the web. For example, companies use online services like Salesforce.com to manage all their sales and customer relationship management needs, thereby storing all of their most valuable client data in Salesforce’s data centers.

Similar services have been created to manage a company’s internal communications, email delivery, human resources, logistics, and more—allowing companies to outsource any business function that’s not their core competency to low-cost providers accessible solely via the cloud. Essentially, this trend is pushing businesses from a centralized to decentralized model of operations that’s usually more efficient and cost effective.

The End of Files, Folders, and Online Security?

Given everything you’ve read so far, it’s clear that storing your digital content will gradually shift more and more into the cloud. This will make the Internet even more of a necessity than it already is, and by the late 2020s, could prompt various world governments to reclassify access to the Internet (and the cloud storage it facilitates) as a human right, thereby treating the Internet more like a utility than a privately funded and managed service.

That said, all this talk about cloud storage hides the fact that it will also usher in a unique set of challenges our society is still ill equipped to handle.

Internet dependency. The obvious disadvantage to storing your digital goods in the cloud is they aren’t accessible unless you can connect to the Internet. This can be an issue during blackouts or travel to less developed regions of the world. But frankly, with the growth of broadband and Wi-Fi networks, this danger is becoming less of an issue with each passing year.

Format conversion. As various consumer and enterprise oriented cloud services proliferate, the market will become even more saturated with online startups looking to differentiate themselves from the pack and lock in customers. One way companies will do this is by ditching the standard file and folder format we’re all accustomed to. In their place, we’ll begin to see new, more task-specific digital organization systems based around projects, tasks, and lists.

Outsourcing costs. If you have your entire life stored online, security becomes kind of a big deal. With data breaches hitting the news every few months and personal, corporate, and government data getting hacked, stolen, and sold to the highest black market bidder, it’s clear digital security is a field that will see larger amounts of funding over the coming years.

In fact, the world of tomorrow won’t be able to function without secure encryption—far too much of the world’s economic system, public utilities, corporate and personal data is now accessible only. That’s why the security issues of today will become far less common over the coming decades. The use of biometric encryption (for individuals), CryptDB, and quantum and lattice-based cryptography (for organizations) will become mainstream.

Data ownership. Our digital identity is slowly becoming as important as our real life identity, and further into the future, they will be one and the same. This makes the abovementioned security issue all the more relevant. It also introduces the matter of legal ownership over our digital identities. Do the profile and images you upload to Facebook belong to you or Facebook? The song playlist you curated on Spotify, is it Spotify’s property or yours? Do you remain in control of all those important documents you upload to cloud storage services like Dropbox?

Intuitively, you would say yes. After all, you invested the time and effort to create and organize the digital content these services are storing for you. And yet, if you read the fine print for many of these services, they maintain the right to block or delete your profile, distribute your content for profit, or share said content with the authorities without you knowing. Questions, and then regulations, around digital ownership rights will begin to see increased traction by the mid 2020s once these services become too important to the average voter’s digital identity to leave in publicly traded hands.

The Death of the Desktop Mouse

Tomorrow’s microchips and hard drives will open up new possibilities in how much data we’ll one day store and compute. But none of that really matters unless we also understand all the new ways we’ll soon engage and control computers to produce and consume amazing digital content. We’ll cover all this and more in the third and fourth installments of our Future of Computers series.