lolpix:I use SpiderOak. It's a so-called zero-knowledge encrypted back up and sync service that keeps historical versions of all your files. I have numerous other backup systems, as that's kind of my job, but if you only have one back up, I recommend SpiderOak.

They look like a good company. But, if you have only one backup, I'd still recommend NOT using a small-business cloud service. Back in the post-2001 tech recession, several early cloud storage companies literally disappeared overnight, when creditors came to the facility with a court order and confiscated all the hardware and furnishings. Some customers never saw their data again.

SPIDEROAK CANNOT GUARANTEE UNINTERRUPTED SERVICE, SERVICE AT ANY PARTICULAR TIME, OR INTEGRITY OF DATA, INFORMATION OR CONTENT STORED OR TRANSMITTED VIA THE INTERNET. SPIDEROAK WILL NOT BE LIABLE FOR ANY UNAUTHORIZED ACCESS TO, OR ANY CORRUPTION, ERASURE, THEFT, DESTRUCTION, ALTERATION OR INADVERTENT DISCLOSURE OF, DATA, INFORMATION OR CONTENT TRANSMITTED, RECEIVED OR STORED ON ITS SYSTEM CAUSED BY CIRCUMSTANCES OUTSIDE OF SPIDEROAK'S CONTROL.

christ1:My PhD adviser told the story of one of his former students, probably 20 years ago. The guy spends 4 years or so doing his PhD and gives the rough draft of his dissertation (hardcopy) to his adviser. Then his computer crashes and apparently he loses everything. Instead of just retyping his dissertation (or has someone do it for him), he gets a job and never completes his PhD.

At that point, I would have paid someone $10.00/hour (or whatever the going rate is) to just type it up again for me.

That's the advantage of doing a Ph.D. in mathematics. If you lose your dissertation you can just type it in again. It doesn't rest on years worth of measurements that you can lose.

Signal processing is similar: your dissertation does require data, but if you lose it, you can recompute it. If I had lost my entire drive, it would probably have taken a week of frantic work to recode the algorithms, reproduce my data and retype the whole 150ish pages of my dissertation.

In fact, I actually did lose some of my dissertation, not due to a system crash but just lousy file management. One chapter was tucked away in a random tarball that I mistakenly deleted to make some space. I freaked out at first, and then I realized that since I published that chapter, it was actually on the shelf at the engineering library. I was able to get the bibliography (the one thing I didn't want to re-do) and then retype the rest.

Beowoolfie: They look like a good company. But, if you have only one backup, I'd still recommend NOT using a small-business cloud service.

I understand your point. But, to put things in perspective, some the world's largest institutions disappeared after the economic collapse of 2008. Being large doesn't necessarily mean a company is more secure. And it's also important to consider that the odds of losing your primary copy and your off-site backup service being shut down simultaneously are remote enough that if you only need to maintain one backup copy, they're probably negligible.

We had a pact in my lab: in the event of fire, someone would break out the window and throw any lab notebooks to the street below. Funny how we didn't really think about getting the hell out of the lab. Guess the grad experience warps you.

Coastalgrl:1)In theory, I could purchase a couple drives at a time and move things from the 5 hard drives (4 portable, 1 internal) to the RAID.

2) In order to interface with the RAID, I would need a separate OS or does it come with some minimal interface program and just access it like a giant HD?

1) Not necessarily. Basically, once you make a RAID array, you can't remake the RAID array without deleting the data and reformatting the drives AFAIK. Now if you went the FreeNAS ZFS 'build-your-own-box' route, ZFS has all sorts of cool stuff to help you extend your arrays and basically do everything that the crazy file system nerds have been wishing for for the last few decades, and it's basically magic. (It also has like no OS support outside of FreeNAS and I believe BSD, so YMMV)

2) OS-wise, the box itself will have its own OS, which will either be FreeNAS or some custom OS from the OEM if you go the pre-built route.

From your personal computer, you can then just treat the FreeNAS drive(s) like you'd treat any other network share. Right now, my NAS is mounted into my Windows 7 laptop as the Z: drive and as long as I have a stable network connection, I'd be able to treat it like any other (high-latency, extremely slow) drive.

/Also, keep in mind that those numbers are ballpark. Basically, the more drives you add to a single box, the more expensive it gets to attach the next drive, especially once you get above 6 drives. So a 2 drive prebuilt NAS might be $3-400 for just the box, but an 8-drive prebuilt NAS might be $1000+ for just the box.

BumpInTheNight:Coastalgrl: Thanks guys for a summary of the different storage options out there

Just starting my doctoral research and have roughly 10 TB of data from previous work. The sheer volume means I'm looking at multiple offsite storage options.

If anyone has any favorites let me know. Also leading up consolidating 25 years of research to develop in house database for research group.

10TB? Jesus christ, hopefully it compresses down to something more reasonable at least? That much would command at least two NASes (little computers devoted to hard drives). There's some online companies that seems to love storing huge data, here's a list http://online-storage-service-review.toptenreviews.com/ I think the trouble with those would be the sheer upload time it'd take to put the backup there in the first place. My connection's upload speed for instance is 1Mbit/sec which is roughly 120KB/sec which would take 23 thousand hours to upload unless I mathed it out wrong.

She should use 7-zip to compress it into pieces and first put the thirds on different storage sites, then take the time to upload all parts to all three.

That's what happens when you grow up in the era of reliable hard drives. Unlike us old fogies who grew up listening to the grind of the motors of a 5.25" floppy (which we had 3 back ups of each).

Chunk chunk bbbbbrrrrraaaaaappppp chunk chunk brrrrrrraaaaaappppp.... the sound of backup #3 of thesis work on floppy drive. Oh well, switch to copy number 2, buy another $10 5.25" floppy and make that the new back up disk.

My favorite backup story was when my old boss, PhD engineer from MIT (and a complete ass) asked me to verify results that the (small) company had been using in it's advertising for a couple of years. It did not take me more than a few hours to realize the guy who originally achieved the results (not the sharpest spoon in the drawer) had compared our software to a debuggable version of the competitors software hence achieving amazing speed ups that no customer ever saw after buying our software. I thought anyone involved in that debacle (e.g. my boss) should have resigned on the spot but my work was immediately questioned by the boss and his boss and I was told to find the old data. It turned out that the companies software development process was shiat (thanks to my boss) and the only archive of said results was an old back up tape that was produced by equipment so old that there was only one working drive in the entire company in an office on the west coast (we were in Boston) and that drive had not been used for over a year. After overnight the tape across the country we (my boss, his boss, several other players) teleconferenced in as the company IT head fed the tape into the drive. After a moment he said "Oh shiat it just ate the tape." I could not stop laughing. Surprisingly I was not fired but after spending months blaming others that were no longer with the company my boss left the company for a higher paying job. Btw, if you can bs your way through a PhD from MIT you can be the biggest tool in the world and still land a great job.

So what's the cheapest way to backup large quantities of data that you don't want transmitted over the internet? I backup in triplicate, and one copy is disconnected from my system, but a fire/tornado/flood/Godzilla could easily wipe out everything at once. The combination of slow internet (relative to drive speeds) and paranoia (not wanting my data physically available to others, even though it's all encrypted) makes online backups less than ideal for me.

For my thesis and dissertation, part of the problem was keeping old copies of them organized, because I liked to have old copies of my work in case did something incredibly dumb (it happened, but I forget how) and had to start from a previously saved point. Stories like this are generally repeated ad nauseum through grad school as warnings, so I guess thousands of grad students owe this guy a thanks for the reminder.

Bisu:So what's the cheapest way to backup large quantities of data that you don't want transmitted over the internet? I backup in triplicate, and one copy is disconnected from my system, but a fire/tornado/flood/Godzilla could easily wipe out everything at once. The combination of slow internet (relative to drive speeds) and paranoia (not wanting my data physically available to others, even though it's all encrypted) makes online backups less than ideal for me.

Do you get eight hours of sleep a night?

Start the online backup last thing before you go to bed.

/if you don't get eight hours of sleep a night, you should//unless you really intend to kill yourself before you reach 30, it will catch up to you

Bisu:So what's the cheapest way to backup large quantities of data that you don't want transmitted over the internet? I backup in triplicate, and one copy is disconnected from my system, but a fire/tornado/flood/Godzilla could easily wipe out everything at once. The combination of slow internet (relative to drive speeds) and paranoia (not wanting my data physically available to others, even though it's all encrypted) makes online backups less than ideal for me.

Short Version:

Find someone who lives in another state that you visit every few months (Family Celebrations are a good one). Keep 1 at home, 1 portable, and 1 at their place. Whenever you go visit, physically take the box/drive with you and back it up while you're visiting.

As weird as it sounds, the people I know who don't do backups, or don't keep multiple copies of stuff around, are invariably people who have never had the joyous experience of never having lost large files before; or better still, never had to sit for a week cursing and praying before that awful message: WARNING: Reformatting the drive will remove all data on the drive! Do you want to proceed? Y/N? before finally hitting...Yes...

Most of us get to do this sometime early in our careers, and learn from it; but as computers have become more reliable, I've encountered many people who have never had this delightful experience until something like this happens.

working toward multiple engineering degrees, working a shiat job at an office supply store. The only reason I still work there is because I get cheap products. Hide a drive or two behind the counter, forget about them for a month or two and presto, bargain storage. For my arduino/c/chemistry/matlab/solidworks/verilog projects. I can't get it through my head how this person didn't back up their data. How do you not have the information on at least 3 drives which are geographically separate from each other? My current files are on my laptop, 16gb drive x 2, RedTwistGeo(8gb drive), and a host of smaller drives which hold individual class information/data/projects. I am crap for getting them organized, but everything damn well has a backup.

/extra backups are for my collection of GoT episodes.//senior project will be interesting... chemical/electrical/mechanical project///evil genius laughter////*weeps because my college doesn't have a chemical engineering program*

lolpix:Beowoolfie: They look like a good company. But, if you have only one backup, I'd still recommend NOT using a small-business cloud service.

I understand your point. But, to put things in perspective, some the world's largest institutions disappeared after the economic collapse of 2008. Being large doesn't necessarily mean a company is more secure. And it's also important to consider that the odds of losing your primary copy and your off-site backup service being shut down simultaneously are remote enough that if you only need to maintain one backup copy, they're probably negligible.

Large, publicly-traded corporations don't normally vanish overnight; they give months of warning. I'd call them more secure from that perspective. At that time, I was providing independent computer support to homes and small businesses. One of my regular customers got burned by one of these services (don't remember which). She was using them for archival storage, and all her previous years went *poof*. Granted, that's different from using them to back up your active files, but that's why I remember it.

After a 25-year career in computing, I'm just paranoid about backups. I've never seen anyplace else where Murphy's Law stays so busy (but then I've never worked with missile launches or nuclear power).

I was there when both halves of a mirrored RAID failed within 15 minutes of one another.

When all 3 redundant backup tapes of the hospital's patient database had unreadable spots (due to the tape drive quietly damaging them after they were written and verified, during the farkin' rewind! I curse you forever, drive 281.), I was the guy who had to reassemble the damned thing, working from a dozen, foot-thick hex dumps of readable parts of the tapes That was my longest workday ever, at just over 40 straight hours. I was sick for a week after.

I was watching as the new, multi-million-dollar enterprise backup system crashed its supposedly failsafe catalog file in its first month after a solid year of testing. Glad I was just watching rather than involved. When the catalog crashed, the system lost track of 2/3rds of a petabyte of backup data. Management didn't like having to start over with fresh, full backups for over 10,000 servers. Time is money, etc. The cool part, though, was it was the first time I heard anyone actually say "petabyte" in a practical, rather than theoretical situation.

And there are more stories where those came from. Somebody out there really hates backups, and he scares me. :)

As weird as it sounds, the people I know who don't do backups, or don't keep multiple copies of stuff around, are invariably people who have never had the joyous experience of never having lost large files before; or better still, never had to sit for a week cursing and praying before that awful message: WARNING: Reformatting the drive will remove all data on the drive! Do you want to proceed? Y/N? before finally hitting...Yes...

Most of us get to do this sometime early in our careers, and learn from it; but as computers have become more reliable, I've encountered many people who have never had this delightful experience until something like this happens.

I had the good fortune to learn this lesson very early. Back in the day I used one of the earliest word processors around that for some reason had a DELETE DEATH key that when inadvertently pressed would delete the entire document--no Y/N or Are you sure?--and with no restore option. Eventually I physically disabled the key with tape. And also learned to back up frequently.

As weird as it sounds, the people I know who don't do backups, or don't keep multiple copies of stuff around, are invariably people who have never had the joyous experience of never having lost large files before; or better still, never had to sit for a week cursing and praying before that awful message: WARNING: Reformatting the drive will remove all data on the drive! Do you want to proceed? Y/N? before finally hitting...Yes...

Most of us get to do this sometime early in our careers, and learn from it; but as computers have become more reliable, I've encountered many people who have never had this delightful experience until something like this happens.

I became a hero to a female neighbor because of this. We were socializing and she was extolling the convenience of keeping all of her stuff (writings, financial records) on her laptop. I asked her about backup, and she said "but the laptop has a 5 year warranty".

I gave her a crash course in why you backup, and taught her how to burn stuff on CDs and to use internet storage personally (I believe in 3 levels of backup: another computer in the house, removable media kept out of the house, and the internet).

About 6 months later laptop goes for a swim in the ocean (long story)... was recovered but useless. With her backups (which she did on some regular basis) she didn't lose anything of importance. So now I can do no wrong in her eyes.

Back-up data stories!!Last year, I went away for a weekend with my laptop, so I could finish writing a review.I was backing up on a USB drive - no internet access, so I couldn't access Dropbox.Had my bag stolen on the way home - with the laptop and USB.Yeah.On the bright-side, review mark 2 was most excellent. Furthermore, it was submitted on time. Sure, I didn't sleep for the next week, but submitted on time.

Also, I had so many copies of my thesis, I wasn't sure which version was finally sent for publication. I'm still not sure.

I feel for her. My HD just crashed a couple of weeks back. Machine locked up. I rebooted and it just sat at the loading screen. I tried safe mode and again, it got to the windows loading screen and stopped. I pulled the drive and mounted it as an external to another laptop, and to my horror it showed empty. I tried a data recovery program, it got half way through the scan and then *click click click*. I hadn't backed up my HD in 9 months. 10GB of anime porn just gone forever.

Worst thing was that just three days prior I was thinking "I hadn't backed up in a while, I really need to do that soon"

Beowoolfie: I was there when both halves of a mirrored RAID failed within 15 minutes of one another.

In grad school we had a RAID fail due to a driver bug. It corrupted all the data on the departmental server. A Masters' student lost her thesis work. Fortunately she had just submitted it for defense, but she wasn't able to publish later. (The prof wouldn't publish any study he didn't have the source code to replicate.) It all worked out in the end, because she took a job in industry and didn't need the publication, but it sucked.

qualtrough:I had the good fortune to learn this lesson very early. Back in the day I used one of the earliest word processors around that for some reason had a DELETE DEATH key that when inadvertently pressed would delete the entire document--no Y/N or Are you sure?--and with no restore option. Eventually I physically disabled the key with tape. And also learned to back up frequently.

In high school, I wrote papers using FontMaster 128 for the Commodore 128. I turned out that a joystick plugged into port 2 would lock up the program if someone jiggled it or pressed the fire button at the right time. It took me a while to figure this out, because my dad had bought a tiny keyboard-attached thumb joystick, and I would either bump it by accident, or fiddle with it when thinking about what to type. Until I figured out the problem, I learned to save religiously. Once I found the bug, I immediately reverted to my usual level of complete disregard for loss of data.

However, there were also love letters and tomes of fiction so awful that decorum prohibits me from mentioning them here. I can't even quote any of it for your amusement without wanting to stab my left eye with a fork.

Most of my life before turning 30 was like that. I want a do-over, dammit.