Posted
by
msmash
on Wednesday August 02, 2017 @12:00PM
from the breakthrough dept.

IBM and Sony have developed a new magnetic tape system capable of storing 201 gigabits of data per square inch, for a max theoretical capacity of 330 terabytes in a single palm-sized cartridge. From a report: To achieve such a dramatic increase in areal density, Sony and IBM tackled different parts of the problem: Sony developed a new type of tape that has a higher density of magnetic recording sites, and IBM Research worked on new heads and signal processing tech to actually read and extract data from those nanometre-long patches of magnetism. Sony's new tape is underpinned by two novel technologies: an improved built-in lubricant layer, which keeps it running smoothly through the machine, and a new type of magnetic layer. Usually, a tape's magnetic layer is applied in liquid form, kind of like paint -- which is one of the reasons that magnetic tape is so cheap and easy to produce in huge quantities. In this case, Sony has instead used sputter deposition, a mature technique that has been used by the semiconductor and hard drive industries for decades to lay down thin films.

Putting a few of their numbers together seems to imply a read bandwidth of 24 Tbps (330 terabytes = 330*8 terabits/tape, tape length = 1098 meters => 2.4 terabits/meter, tape speed = 10 m/s => 24 Tb/s). Which to me is way more interesting than the total storage size. Of course I have no idea what you could actually feed that into, it far exceeds CPU memory bandwidth, let alone SCSI or PCIE. Do these things not actually run at full speed for more than a fraction of a second?

Surprisingly, no performance discussion in the first link and I can't get access to the IEEE published paper without paying. 24Tb/s would be impressive. Also, what is the expected MTBF? Better, worse, or the same as a regular tape?

While it would certainly be impressive if it delivered those numbers I strongly suspect that the final product will use a narrow set of heads and require multiple passes to fully access the tape. According to Wikipedia LTO 7 [wikipedia.org] for example requires 112 passes to fully read or write a tape. Also, even if full width heads could be economically built it would be troublesome to sink or source data at that rate, and as far as I'm aware tape drives take a relatively long time to seek and restart streaming after a bu

Holographic storage has been vaporware for nearly 20 years. Inphase was talking about huge storage discs back when Windows XP came out.

Last I heard, they managed to deliver only a tiny percentage of that capacity (under 10%?), with very slow speeds, but with a ludicrous price and zero availability.

Meanwhile:-tapes cost as much per TB as external drives, are slower, require drives that cost over $1000 and complicated software...-Blu-ray discs cost as much per TB as external drives, but are write-once, requir

Can we get a consumer-level tape drive, something that works with USB 3.x, costs around $1000, with media costing $10-25 a cartridge, with an actual archival life, and some basic features like AES-256 encryption and compression in hardware? Having WORM media would be nice as well.

This would solve a ton of storage/backup issues. A lot of people don't have the network connection to make cloud backups (much less complete disaster recoveries) feasible. People may not trust the cloud. Regulations may prohibit use of a remote backup provider. Or, someone just likes having the peace of mind of physical control of their data on media which might last past the end of this decade.

There are tons of high capacity optical offerings available (Sony has some)... but other than specialty markets, they are not worth it. What would be nice would be a tape or even an optical drive for backups.

Optical wouldn't be bad either. 400 CD jukeboxes are a solved problem, so having a BD-like format with 2-3 terabytes per disk would solve a lot of backup problems, especially with WORM capability to fend off ransomware.

If you think optical is any good whatsoever for archiving, you have never wasted time on it, No two optical drives can even agree on whether a written media can be read. Even the original drive is likely to reject a disk it wrote itself the day before. It is garbage technology on which I wasted all kinds of time and money, only to find it to have a reliability of zero. You can't have recording surfaces exposed to fingerprints and dust like that if you want something to depend on.

If you burn at the absolute slowest speed your drive supports, use quality media (m-disc, or archival grade optical... although I've just bought cheap stuff and kept it cool and dark and it has lasted...)

The biggest things to ensuring long term stability of optical archival media is keeping it out of the sun, burning it at a slow rate of speed, and keeping it cool and protected. If you do these 4 things it will last you 20+ years (and yes I have CD-Rs that have now lasted since the late 90s! Y'know back bef

The only real option for most normal people is to just build a backup server with another array to hold the backups. Even ignoring media archives, personal data with photos, videos, etc. can easily approach 1TB. I just upgraded my main server, and used the old one with some of the old 2TB drives in 3-way mirrors as the backups. I'll likely move it offsite now that it's synced up.

An affordable tape system would be great, but I have yet to find one that has a reasonable cost with better reliability than my HDD arrays, even though they require a little more maintenance. Disk will always be faster for random I/O, but tape would be great for a lot of stuff if the cost were somewhat reasonable.

And $1000 won't do it. People won't buy it. A few people on/. might, but your average person? No way in hell. Not with 4TB HDDs under $100. $200 drives with $20 tapes might work, but they would need to be in the TB range, native. I don't see that happening, though I'd love to be wrong.

And $1000 won't do it. People won't buy it. A few people on/. might, but your average person? No way in hell. Not with 4TB HDDs under $100. $200 drives with $20 tapes might work, but they would need to be in the TB range, native. I don't see that happening, though I'd love to be wrong.

Agreed as to price range. And you're not wrong; it's not going to happen. It's friggin' IBM and Sony. That's practically synonymous with overpriced crap. IBM will only want to sell them with $10,000/year service contracts, and Sony will only want to sell them for $10/gig. They want to pay back the R&D after one sale. No way these drives will ever be reasonable for even medium sized enterprises, nevermind consumers.

I too simply built a NAS. FreeNAS makes it so easy these days, and ZFS snapshots gi

reasonable cost with better reliability than my HDD arrays, even though they require a little more maintenance.

HDD is only reliable if it's spinning and in an array that's scrubbing once a month. Otherwise a single disk fails at p=0.02 per year, which makes it impossible to bulid a large reliable offline library. LTO is enough more reliable that, using the same number of parity tapes, it can be scrubbed much less often while achieving a given size & reliability compared to a disk array.

This is an important archival characteristic. It takes a lot of infrastructure to keep a disk array scrubbing: electricity,

The question is what do you want to do and how much space you need. If you're looking for an onsite backup, then for $1000 you can build/buy a RAID server that will handle terabytes of files. If you want offsite backup, you can also leverage that server to feed HDDs that backup and then send offsite. If you are looking for archival storage, that's another solution.

The problem with optical has been is that most consumer grade discs are basically crap. They don't last long and they don't hold a lot. There are

That doesn't backup the 20TB of Linux ISO's I have stored away on my NAS/SAN. I need reliable portable, external, removable backups that I can shuffle to another location and restore when the array inevitably fails.I bought a Dell T4000 (48 tapes) with dual LTO-4 for $500. Ridiculous overkill for my data but you know what, it'll back everything up and I can shuffle it to another location in a fireproof box that I don't have to trust Amazon, Google, or CloudBackupVendorX to restore my data from. Everything

Longer archival life for the media is irrelevant if the drive becomes obsolete, as is the case with today's tapes. LTO, for example, only guarantees backwards compatibility for two generations, which come every 2-3 years. The typical consumer isn't going to put in the time to migrate archives to the current format before old drives stop being available.

If you think $1000 is a reasonable investment, might as well buy a new $100 hard drive every two years. Interface compatibility will be vastly easier to ensu

Can we get a consumer-level tape drive, something that works with USB 3.x, costs around $1000, with media costing $10-25 a cartridge, with an actual archival life, and some basic features like AES-256 encryption and compression in hardware? Having WORM media would be nice as well.

This would solve a ton of storage/backup issues. A lot of people don't have the network connection to make cloud backups (much less complete disaster recoveries) feasible. People may not trust the cloud. Regulations may prohibit use of a remote backup provider. Or, someone just likes having the peace of mind of physical control of their data on media which might last past the end of this decade.

There are tons of high capacity optical offerings available (Sony has some)... but other than specialty markets, they are not worth it. What would be nice would be a tape or even an optical drive for backups.

Optical wouldn't be bad either. 400 CD jukeboxes are a solved problem, so having a BD-like format with 2-3 terabytes per disk would solve a lot of backup problems, especially with WORM capability to fend off ransomware.

Can we get a consumer-level tape drive,... and some basic features like AES-256 encryption...

This is an accident waiting to happen. Every vendor implements encryption as they see fit, usually storing keys on either an EEPROM, flash, or battery-backed RAM -- or if you're lucky, some unused or segregated part of the tape (might waste a few blocks but that's OK). It sounds great at first, until the product in question stops working / dies / goes EoL. You were using Vendor X's drive, but now you've found Vendor Y sells a drive that supports your OMG-2113 tape form factor... except none of their cont

We need proper backup software, first. All the major backup companies are moving to cloud-only storage, and OSes do everything possible to prevent backups due to piracy concerns (locking you out of root, only backing up your home folder, etc.)

Consider that for many years the most common way to diagnose problems on PCs was to do a factory reset (whether it was labeled as such or not). Manufacturers don't give a damn about your data and will happily wipe it all out if it saves them 30 seconds on a tech supp

This. It isn't like backup software is anything new. The best backup software I've seen would be Retrospect, because it did compression, deduplication, encryption, and it allowed one to move backed up data to new media, verify the data, expire old backups, move selected sets to WORM media, and so on. It was a great way to archive files because you could just put the files into it, hit "archive", and pretty much forget about it.

Then, there are programs like NetBackup and TSM/Spectrum Protect, which have h

The big problem with tape - forget price, forget capacity, forget software - is that it's sensitive to dust. You can get high capacity for relatively low cost, but the tape drives and library need to be in a climate controlled, dust controlled data centre. As soon as dust gets inside the tape drive in significant quantities, it's pretty much game over for the drive, and the data on any tapes that are loaded before the problem is identified and fixed.

Back in the '90's and early 2000's I fell for the 'backup tape' meme three times. Every single time it ended the same way: Buy backup tapes. Put important stuff on them. Put them away. Some time later, you need something off the tape. Whoops, tape drive isn't working right anymore, not even with a new blank tape! Get a new tape drive. Whoops! Not compatible with your old tapes. Or, Whoops! It just won't read your old tapes, even though it's supposedly compatible! Throw out ALL THE TAPES because they're now useless junk and START OVER FROM SCRATCH. Repeat this THREE TIMES.

Verbatim's QIC-eXtra was the worst scam... bundled backup software that barely worked, and bundled recovery software with so many bugs, you basically *had* to buy their expensive premium version to have any hope of recovering more than a few random files... and even then, it broke with every new version of Win9x, and wasn't compatible with NT unless you bought their even MORE expensive "enterprise" edition. And some thirdparty backup tools would happily create backups on QIC-Xtra tapes, then crash after res

What I *really* want: a 100-disc carousel-type jukebox (like the ones they made for audio CDs in the late 90s), with BD-R optics instead and a non-proprietary API that encourages support by open-source backup software. 25-gig BD-R discs are fairly cheap now, and a 100-disc carousel could give you 2.5TB of ransomware-proof WORM storage.

It seems like a disk controller could have a write-once option jumper, and a 3TB disk would cost a hell of a lot less than your fancy carousel. There's a reason those things went away.

I was going to write a post nearly identical to yours. After about 3 generations of backup tape technology and not ONCE getting anything useful off a backup tape, I quit tape forever. Now I just try to keep my stuff under 1 TB (er, 2 TB, er, 3 TB) and image to a backup drive of the same capacity.

I was going to write a post nearly identical to yours. After about 3 generations of backup tape technology and not ONCE getting anything useful off a backup tape, I quit tape forever. Now I just try to keep my stuff under 1 TB (er, 2 TB, er, 3 TB) and image to a backup drive of the same capacity.

Yeah I keep my data down to about 2TB of just the most important (or difficult to reproduce) items. Then I use a tool to sync the data to external drives that are removed. I've got 3. One that stays in the fireproof safe and gets updated once every few months, one that stays in the safe and (theoretically) gets updated and rotated every 2 weeks. And one that stays in my desk drawer that gets synced every other week in rotation with the drive in the safe. In practice I usually just end up syncing every

Yup, and if you want to have a perfectly peaceful mind, just encrypt these drives An encrypted drive left at a friend's place or at your workplace is just a great way to protect yourself from fire and stuff while not having to pay for a paranoid and expensive vault.
Just do not forget to backup the encryption key too, separately;)

This is something I really have to commend the optical disk makers for. The newest Bluray drives can still read DVDs and CDs. Heck, most of them can still burn CD-Rs.

My master's thesis is stored on a Zip disk in some box in my garage. So is the Zip drive, but it didn't work when I switched my archival storage to CD-Rs a couple decades ago. OTOH those early CD-Rs are mostly still readable.

If you really want to keep your data safe, I would say prefer hard disks or dvd's over tape due to the fragility of tape. I had some good experiences with tape recovery; the drive was industrial quality, but that was only a year back in an era when hard drives were on the cusp of becoming cheap.

Hopefully, for the first time in years, we'll have a backup medium with the capacity to do a true full backup of everything... that's also fast enough to finish the job in less time than it takes to render the backup almost moot, with enough capacity to include plenty of forward error correction.

Of course, if it satisfies the capacity, speed, and reliability criteria, it'll probably cost at least ten times as much as the computer it's being used to back up, and will thus effectively not exist for non-enterp

Before you get excited about this announcement, note that IBM and Sony has a history of announcing tape drive vaporware. Here is from wiki:

"In 2011, Fujifilm and IBM announced that they had been able to record 29.5 billion bits per square inch with magnetic tape media developed using the BaFe particles and nanotechnologies, allowing drives with true (uncompressed) tape capacity of 35 TB.[18][19] The technology was not expected to be commercially available for at least a decade.

In 2014, Sony and IBM announced that they had been able to record 148 gigabits per square inch with magnetic tape media developed using a new vacuum thin-film forming technology able to form extremely fine crystal particles, allowing true tape capacity of 185 TB.[20][21]"