Every day I experience life in the world of healthcare IT, supporting 3000 doctors, 18000 faculty, and 3 million patients. In this blog I record my experiences with infrastructure, applications, policies, management, and governance as well as muse on such topics such as reducing our carbon footprint, standardizing data in healthcare, and living life to its fullest.

Monday, January 12, 2009

The Home CIO Gets in Trouble

For Christmas, I gave my wife a Macbook Pro 15" with 4 gigabytes of RAM and 320 gigs of storage, running Adobe Creative Suite 4. A great machine.

Whenever I get a new device for the family, I apply all updates, install printer drivers, and move all data from the old machine to the new machine.

In this case, I moved 60 gigabytes from my wife's old machine to this new machine - all her documents, all her powerpoints, all her photoshop work, and all her 5000 song iTunes library. I migrated her digital life.

My daughter has an iMac 20" with 320 gigabytes of storage and I have a Macbook Air with 80 gigabytes. We use Time Machine and an external USB hard drive for backup, but it's only a 280 gigabyte drive, without enough room to backup my wife's system. I planned to buy a Western Digital My Book Studio Edition II 2 Terabyte Dual-Drive Storage System (900 Gigabytes usable with RAID 1) this week to support all our backup needs. In the meantime, I was confident that the new Macbook Pro would keep my wife's data safe.

The likelihood of a new Macbook Pro having a catastrophic hard drive failure is near zero.

Last Friday morning, my wife opened the lid of her Macbook and a Flashing Question Mark appeared instead of the usual OS X screen. The consequence of this bad event is that my wife lost her entire digital existence and had no backup.

I desperately tried everything suggested by Apple - booting from the DVD, doing a disk repair with the Disk utility, reseting Parameter RAM, reinstalling OS X etc. but the hard disk could not be mounted.

I consulted with my best Mac engineers at BIDMC and Harvard.

They recommended Disk Warrior. It's been running all weekend without success.

If that fails, then Drive Savers will rebuild the drive by inserting the platters into a new drive.

The good folks at Apple have been very supportive and have offered to replace my Mac book at no charge.

Along the way, I've learned many important lessons

1. All hard drives can fail. Back them up.2. Hard drives fail most often when they're very new or very old. A failure within the first week of operation is not uncommon.3. If you need to copy your iTunes library back from your iPod to iTunes, it is possible despite the digital rights management design which attempts to make this difficult. Here are the simple instructions to do this.

This week I'll get my wife a new Macbook, get a backup drive with built in RAID 1 protection, restore her iTunes from her iPod and hopefully recover her documents/powerpoints/photoshop work.

John, the cost of an external 500gb drive at Costco is $120, which is firewire/USB2. You can liberally sprinkle these around the house PRN.

I have an old mac mini in my audio rack that has a train of these (1 per mac) which is running file sharing and all machines then time-machine to those. Has been a lifesaver. Over gigabit ethernet is very fast (after the initial one). This is also more useful than a RAID on the local machine, since the chance of a drive failure is low, but the chance of software corruption or user failure is high and constant (particularly with spouses!) time machine protects against these problems.

If you REALLY want to have protection, backup via time machine onto Drobo arrays (that's what we do over in DCI at Beacon St) hooked onto the external array. We paid ~$1000 for a 4TB RAID-5 with "green" enterprise grade drives (then bought 2 of these) which are hooked to our Xserve via FW800. They have the added nice feature of dynamic rebuild so when 2TB drives ship at good prices and you are running low on space, you simply replace one at a time and it will upgrade itself up to a 8TB array (and so on up to 16TB).

I'll second the Drobo recommendation. The ability to dynamically (and easily!) grow the volumes in the Drobo to meet growing storage needs is invaluable.

Also- if you would like to rescue more than just the physical tracks off an iPod, I highly recommend Senuti. It is the only utility I'm aware of that can pull playlists as well as useful metadata (ratings, play counts, last played) off of an iPod.

Why not hire one of the services that will recover your data for you? These services often go to extreme lengths to recover the data - as far as disassembling the hard disk in a clean room and remounting it on a new rack of spindles.

A digital existance is a terrible thing to waste - it might cost a few hundred (few thousand?) bucks, but maybe its worth it. (I dont work for these people)

Be sure to try Data Rescue II, I have always been able to pull data off the failed hard drive, less a mechanical issue causing the failure. In most cases, we all know know that the directory fails not the actual data. As you already know, Data Rescue II does a sector or block read and retrieve regardless of any corrupt directory, bad blocks and sectors etc. Truly a lifesaver product, it's really too bad we don't have something like Data Rescue II for our lost memories...