Sunday, October 25, 2009

I Always Need More Storage

The following was a reply to another photographer about hard disk storage for his raw photo files on the MacNN forums. I got carried away, so I thought I might as well post it here as well. So if you've been thinking about saving your photo files:

To the original poster, my wife is a nature photographer and we've been fighting the HD capacity problems for several years. Back in 2006'ish, I convinced her to run mirrored disks, but it wasn't real Raid 1, just manually duplicating everything (with a pair of DVD's burned for backup as well), but since the mirroring wasn't automatic, she'd get in the habit of going back to one of the mirrored drives and working her photoshop magic there, but not keeping the updates duplicated. My one attempt to run a real RAID 1 on firewire drives using the Mac's built-in software was less than successful due to wife's curse of the semi-alive cables that were constantly tangled and unreliable.

Next step, I purchased a Buffalo TeraStation with 4 500 drives that provided a RAID 5 solution over the home network. From the start, it was unsuccessful for her workflow because even with gigabit ethernet, she wasn't getting the speed she needed to browse her photo library via Bridge. It became a side repository for photos that were less used.

Next, as the numbers of drives became unmanageable, we went with a Drobo. With a FW800 connection, she was immediately happy and began the process of moving her substantial library of photos over to it. I began buying 1.5T drives to increase the capacity in her 4 drive Drobo until it was clear that we would need more. There are 2T drives out there, but the cost per GB is still too high to use them. Drobo uses RAID-like technology, but with their own easy to use software. To add more capacity, just remove a drive and put a bigger drive in its place. The Drobo Desktop software gives you hints and warnings. If it's green, everything is fine. If yellow, it needs more capacity but still works and still protects files against a HD failure. If red, you're in danger if a HD failure occurs. When we unplugged a drive (hot swap) it went from yellow to red but still allowed her to use her files normally and then spent some hours integrating the new drive into the set, at which time it went back to green.

This month, I purchased the 8-drive DroboPro and moved the four drives as a set to the new box (and after a moment's panic when I had inserted the drives upside down and it couldn't recognize them) it came up with the library running fine in its new home. I plugged in a couple of 1T's and a couple of 500G's to fill out the slots and it made more capacity. Mary Ann promptly began moving still more of her library from individual drives to the new redundant storage. I can see that we'll need more 1.5T drives soon and I can use the smaller ones in the old Drobo chassis for an on-the-road Drobo for our frequent road trips.

Last night, about midnight, Mary Ann had connected the last one of her old 2006 drives and started a copy session that would have completed her move of photo libraries to the new Drobo Pro overnight. Just then, one of those sneaky, vile, cables wrapped itself around her leg and as she got up from her desk, the 2006 HD went sailing across the room, still blindly copying files, until it slammed hard on the ground and began making *horrible* sounds.

I was called in and spent an hour removing the Seagate drive from its chassis and confirming that it wasn't a fan making that noise, but it was indeed the spindle inside the drive. She wasn't going to be getting any of those photo file back. Luckily, this was one of those 'manual' mirrors we had started back then, so it appears that almost everything was available from other sources, so the trauma is reduced. But still, we're looking to see what kind of off-site storage we can come up with to protect nearly 10T of photo files.