Exabyte, the new megabyte

Page Tools

I started in the computer industry in 1976 as a mainframe
computer operator. A large part of my job consisted of mounting and
unmounting heavy 3330 disk packs on an IBM System/370.

These disk packs each had a capacity of 200 megabytes. If a
particular job the computer was running needed a data set that was
not loaded, we had to remove an unused pack and replace it with the
correct one. Day and night we loaded and unloaded disk packs,
ensuring the mainframe always had the right data available.

In those days, 30 years ago, data storage was so expensive, it
was treated as a very valuable commodity. Each of the disk drives
cost nearly $200,000, so it was cheaper to hire a large staff of
computer operators to load and unload disk packs than it was to buy
more storage.

The first hard disk for microcomputers was released in 1980, by
which time I had left computer operations and was selling Apple IIs
for a living. I remember my first sight of a Seagate ST506 disk
drive. It held five megabytes, was the size of a shoebox, and cost
a couple of thousand dollars. We were in awe of its capacity. It
could hold as much as 40 Apple II floppy disks, all available at
the same time.

To use that first Seagate, you had to separate the drive into 40
logical floppies, because that was how Apple's disk operating
system recognised it.

How times have changed. Seagate is still in business, and is
still one of the leaders in data storage. Last month I got my hands
on one of its new Pocket Hard Drives, which connects to my laptop's
USB port and fits neatly into my hand. It cost me a couple of
hundred dollars and has a capacity of five gigabytes.

Let's put this in perspective. Taking into account inflation,
the Pocket Hard Drive costs about one-50th of the price of the 1980
ST506, its capacity is about 1000 times more and it weighs a
fraction as much.

Price-performance metrics are improving in every area of IT, but
such figures are astonishing.

Seagate also makes a 400 GB USB-attachable external disk drive
for about $600. The entry-level disk drive on a laptop PC is now 80
GB.

A gigabyte is about a billion bytes. A byte is a character of
information, like a number or letter. Given that the average word
(including the spaces between them) is about six characters, and
that a decent novel is about 100,000 words, quick calculation
suggests one gigabyte will hold more than 1600 novels.

The figures at the high end are equally astonishing. Large
corporate data centres now typically have many terabytes of storage
(a terabyte is 1000 gigabytes).

Do some more quick sums. Each major Australian bank has a data
centre of 20 terabytes or more, which equates to holding a megabyte
of information, on average, for every man, woman and child in
Australia.

Much of the demand for storage is being driven, at least at the
domestic level, by digital storage of photos and movies. That
explains why a terabyte in the home will be the norm within a few
years (a normal movie, properly recorded, requires a couple of
gigabytes). But why so much storage on mainframe computers?

There's an old saying in computing that data expands to fill the
space available. No matter how much disk capacity we have, we fill
it. At the company level, data storage has become so inexpensive
that it is easier to make multiple and redundant copies of data
than it is to properly manage it.

Companies such as IBM and EMC sell storage management software
to help big companies efficiently store large amounts of data, but
often it's cheaper and easier just to buy more disk storage.

EMC, the leading high-end storage vendor, promotes a strategy it
calls "information life-cycle management" to encourage people to
put the right data in the right place, the modern-day equivalent of
having a computer operator load and unload disk packs as they are
needed.

But still our storage needs grow. Researcher IDC now measures
annual shipments in "exabytes" (1 million terabytes or, to be more
precise, 1 quadrillion bytes or 1,152,921,504,606,846,976
bytes).

A graph of the cost per megabyte now hardly registers.

As homes and businesses embrace multimedia, the demand, and
supply, of disk storage will continue to grow.

Here's a scary thought: at the moment, most of the world's
telephone calls are ephemeral - as soon as the words are spoken,
they are lost. But storage capacity is now so great, and cost so
low, that it is becoming feasible to store all telephone
conversations on computer disk.

With the increased requirements for audit trails for compliance,
the real and artificial imperatives of the war on terrorism, and
the increased sophistication of data mining software that can
analyse digitised voice patterns, can this be far away?

Be assured that it will happen, and when it does, data storage
and data management will enter a new era.