Posted
by
Cliff
on Monday July 31, 2006 @07:55PM
from the penguins-need-backups-too dept.

technocraft asks: "I've been searching for an online service to backup data from my Linux file server and have come up with nothing. For many users, Carbonite looks to be a great solution: Affordable, with 'unlimited' capacity. Unfortunately for me, you can only backup from Windows XP and explicitly NOT from external drives or network mapped drives (like my file server)."
Is anyone aware of an online backup services without these restrictions?

We're customers of the parent organisation, JohnCompanies [johncompanies.com], and I can't recommend them highly enough. The thing that makes them stand out in my opinion is the support, you don't get clueless newbies reading from an FAQ like you do with every other hosting provider I've ever encountered.

I've been looking for an excuse to try out rsync.net, it seems like an interesting service, for example it offers WebDAV access, which is built into Windows, OS X and KDE.

Yeah, I know I sound like a fanboy or astroturfer, but I don't care, it's really rare to find a company that actually gets it so right when there are so many incompetents around, especially in the hosting business.

Wouldn't it be a lot cheaper just to just have one of your employees carry a DLT tape home every week? What's the obsession with backing up over the 'net? It isn't cost effective by any stretch of the imagination. In fact, this is seven times the cost per gig of making a DLT backup, and a DLT backup costs you that much -once-, not monthly.

The best network backup, if you really have to have such a thing, is to set up a second site in another state.

Having an employee take the tape home is the worst backup solution ever. There are far too many risks involved. What do you do if the employee loses the tape? Who then has your accounting data?What about the quality of the tape? Employee puts tape into her purse/bag/briefcase with all the other garbage in there, not a good place for a tape. How hard would it be to get the tape from an employee if someone wanted to? Employee takes tape, leaves it in car when they pick up groceries, opps, someone stole

Taking a quick look on Froogle, the tape drive alone will cost about $900, and 80gb tapes cost about $70 each. You can buy hard drives for less, and have an employee take a hard drive home each week. That's not cheap by any measure. Tapes aren't very reliable if you reuse them. They'll appear to backup ok, but most will be bad after several uses. If you have a terabyte to back up, you're going to need about a dozen tapes a night, no matter how little has changed. Tapes are old, expensive, dying technology.

I realize this wouldn't be suitable for large companies but for my home
network, I just use a regular webhosting provider for system backups. I use
one-and-one which is $4.95/mo for 50gb of space, duplicity on the Linux side
which is able to gpg encrypt the volumes and incrementally backup the
systems, transfering the volumes over FTP to the webhost. The backup
directory is configured so as not to be publicly accessible but it's gpg
encrypted in any case.

rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification ti

While I agree that tapes are an old, dying technology, there is no need to resort to hyperbole.

They'll appear to backup ok, but most will be bad after several uses. If you have a terabyte to back up, you're going to need about a dozen tapes a night, no matter how little has changed.

If you don't verify your backups (even on disk or network backup) then you deserve to lose your data. And since when does anyone do a full nightly backup of terabytes of data? You do quarterly full backups with monthly,

Taking a quick look on Froogle, the tape drive alone will cost about $900, and 80gb tapes cost about $70 each.

If you have serious backup requirements, LTO-3 tape drives are about $6000, the tapes are about $80 each, and they hold (in our experience) close to 1TB with compression turned on. You'll need a fast machine to prevent shoe-shining the tape, as you want to drive ~40MBytes/sec into them for best performance. The tapes last for practical purposes indefinitely --- it's a close cousin of DLT, and

That sounds really cool, except I'm surprised at how high the price is. Between photos and home movies, I have about 100GB of data I'd really like to get backed up offsite, and it grows by about 2GB per month. I have some other stuff I want offsite as well, but it's very small.

So, according to their pricing scheme, it would cost me $126 per month to store my data, and the price would increase by about $4 each month as my data grows. Storing my data for a year would cost me nearly $1,800. That probably

My reaction to rsync.net was similar. I'd really love to try them out, but I can't come up with any really good ways to justify that kind of expense.If you have multiple locations available for your use -- and if you're a home user, who doesn't? (parents, friends, etc.) -- it's a lot cheaper and you can get a lot more flexibility if you take an old PC, put a bunch of drives in it, and set it up somewhere. Then just have your systems replicate to it at night. (Yes, it's not bidirectional if you just use vani

Description: remote incremental backup
rdiff-backup backs up one directory to another, possibly over a network. The
target directory ends up a copy of the source directory, but extra reverse
diffs are stored in a special subdirectory of that target directory, so you can
still recover files lost some time ago. The idea is to combine the best
features of a mirror and an incremental ba

This is quite true, and every current implementation of rsync that I've run across (not that I'm exactly a scholar on the topic or anything, but I've used it on Mac OS X and a bunch of Linux distros) uses SSH as the default shell when the address is specified with a single colon ('rsync -avz ~/Documents jdoe@foobar.com:/var/backup/'), although I suppose its probably best practice to specify the shell explicitly in a script.So just to restate what I should have said: the only thing you'd need to have running

Our company has about 200gb of important, frequently updated data that we need backed up nightly. $260 a month for peace of mind for a business sounds pretty cheap.I've checked out probably a dozen backup services, and most were around 5x as expensive. These guys [backup.com] for example, charge $65/month for only 10gb. Another called me to give a quote and when I told them that another company quoted $300/month (before I found rsync.net), they laughed a bit and said they couldn't go anywhere near that cheap. Granted, t

I'm surprised nobody mentioned Mozy. Ok, they only support Windows for now but they have a MacOS X and probably a Linux client coming soon. Mozy offers 2GB free or 30GB for $5/month. Here is my shameless plug with my referral code: https://mozy.com/?code=KXYE5N [mozy.com]. This will give you an extra 256MB of free space. If you don't want to use my link, just go to mozy.com.Radicode

Our IT department actually found one through TIDF [tidf.com]. They found the "best fit" for us for free, and quoted us a price that was slighly higher than the one found on the website. So out IT guys got the name from TIDF then went to the other firm, told them they were referred by TIDF and signed up. (It was net mass [netmass.com] btw - it is pretty pricey, but the support is great and we don't back up much data).

Me? I've only personally used mozy [mozy.com] and while I liked it, it fails your requirement of linux compatibility (whi

I use Carbonite, and I've backed up over 100GB so far. My cost is $5 per month. The cost with rsync would be $126 per month. How can there be such a massive discrepancy? I'm sure the rsync service is probably more robust, but it costs 25 times as much? Something is not right here.

If a Pizza Hut pizza costs you $10, a quality pizza from a local place might cost you $15. But if rsync was running the local place, you'd have to pay $250 for that pizza. Momma mia!

I think the pricing on this is way too high if you have more then a couple of GB to store. Consider that for $60/month you could get 160 GB here [cari.net], and you get to use a full fledged server for anything else you want. It could be a "hot site" for a lot of the services you run on your home server, or more likely, you'd let it be the primary site and let your home server be the backup.

I use Blacksun [blacksun.ca] which doesn't have the "unlimited" storage you had before, but they are very affordable and offer rsync, ssh, sftp, and the regular linux services as well as the typical dragndrop interface clients. Very nice, and their tech support is helpful and quick to respond!

I developed and maintained [and, sadly, had no say in the design] the Saskatchewan Air Show web site for 2 years until the accident that occurred in 2005, then the organization decided to give up and well, to my knowledge there isn't going to be an Air Show ever again... unless someone is willing tp pay off their debt.

Anyway, back on track. They were originally hosted by Blacksun... and I have to say we had nothing but problems. When I firs

Do you have any more information regarding your situation? For example, why does it have to be online storage? Is there a reason tape/optical/hard drive doesn't work for you? Also, how much data are we talking about? If it is hundreds of GB, then online may not be the best solution; the time to restore your data may be prohibitive. In line with that same question, what kind of bandwidth are we talking about? DSL is great for a couple of GB, but you need a lot more than that if you have a lot of data.

Thanks for the questions.I'm looking for easy, reliable offsite backup functionality. I'm a photographer with a lot of photos. I use a Linux box with a raid array for my primary storage. This is backed up to DVD piece meal. I've also been using an external usb drive as more a nearline backup device.

In terms of bandwidth and speed, I expect my data to be slowly backed up over time. Probably will need a few weeks to get everything uploaded. I'm assuming the backup client will use idle time to transfer. Once t

This is going to be expensive... Even with the 50% discount at rsync, that's $450/mo... *ponders other ideas*
Perhaps a dedicated server somewhere isn't such a bad idea. Could possibly share it with a few friends.

If most of that isn't changing, you can archive it to removable hard disks. Get two removable hard disks, keep one at home, the other in a secure place, and swap them from time to time. Keep your recent (since last swap) work in its own folder, and back only that folder up online.

Or you can set up a backup server on the other side of your house and hope both servers aren't taken out by the same disaster.

OK. I think there are a lot of solutions that would work, and there are quite a few that people have mentioned.

I think the biggest issue for you to consider is how much security you want for your data. I don't just mean encrypted and properly administered, but also the likelihood that the company will be around for the next few years, and prices won't go through the roof.

Consider Carbonite. They claim unlimited storage for only $5/month, but I'm not sure how they plan to do this and maintain a profit

Regarding your question on Carbonite:
"Carbonite does not currently back up files that are larger than 2GB in size, removable hard drives (e.g., USB drives) or mapped network drives."
"Carbonite will back up all the supported files on your hard drive whether you have 1GB, 10GB or more. As a practical matter, the speed of today's DSL and cable Internet services will make it very slow to back up more than, say, 40-50GB of data. Our Terms of Use will allow us to deny service to abusers."
Given those terms,

If you are willing to lose a few days/weeks of data, you could use an external hard drive (or an internal drive with an external enclosure), and just store them at different locations. Visit your parents once a month? Drop off a drive and pick up the old one. You could also keep one in a safe deposit box and switch it out every few weeks. A 500GB external drive can be found for $350. Definitely not as convenient as the server solution I mentioned earlier, but less overhead, but more latency in the back

Start with a good enclosure like the BYTECC ME-835U2F ($45). It has an internal power-supply using a standard A/C power cord so there are no funky wall warts or AC adapters to lose. It has both Firewire and USB so you can pick your poison. It's an aluminum shell so it will probably hold up well. Plus it has a 2nd internal fan which is designed to keep the hard drive cooler (most external units omit fans or only have a small 40mm fan in the back).

Query: How can you possibly trust any third party not to take liberties with personal or business information entrusted to their care?
Answer: I wouldn't trust with my real email address, much less my data... I would encrypt the data before backing it up.

Good question. Also, one of the things you might be concerned about is the remote compromise of your machines. If you have a 7-day daily backup stored in separate directories on a remote server, but they're *all* writable via rsync, and you have a cron job on your (e.g.) webserver that backs up to the remote server, what stops someone from corrupting both your webserver and all the remote backups?

The backup directory on the remote host can be owned by user1 and group-owned by user2, with the following permissions: u=r-x,g=rwx,o=r-tA backup is copied to the remote server as user2. Once the backup is complete, change its owner to user1. user2 will then no longer be able to read, modify or delete the backup file.

If you are confident with the syntax of sudoers(5) then sudo(8) is a good way to do this. If not then it can be done with a cron job, as long as you ensure the job runs once the backup is compl

Don't forget to include the costs your ISP will charge you for the bandwidth usage... or if you need to restore 10GB over a T1, I hope you didn't need that data that quickly:)It's not the online backups that sound like your concern but "outsourcing backups", otherwise you'd backup to a RAIDed pair of disks or backup to tape, both which will be cheaper than $onlineserver + $bandwidth costs. If the tech (could be you) loses the tape, breaks the tape, corrupts the filesystem that is the target for the backup

are cheap... probably under $150 for a 100+GB server-grade drive + USB/FW shell. Get two and do differential backups to alternating drives daily - make sure that one's off-site at night in case of a fire where you can't get the drive out. Much cheaper than bending over monthly for an online backup service where your data's in the hands of god-only-knows-who. If you're using an online service, I hope that you pre-encrypt the data with your own keys before handing it over to their software and keep your ke

Much cheaper than bending over monthly for an online backup service where your data's in the hands of god-only-knows-who.Sounds like somebody should let someone else handle their investment banking. You're not thinking this math through very clearly. Have you, perhaps, bought a hybrid to save money?

Your calculation is correct -- IF you've got no more than about 4GB worth of data worthy of being properly backed up, then the online-solution is the better one.

Which means that for 95% of the target audience (i.e. people with a high techknowledge and interest) the online-solution is going to suck major.

Even if I backed up *only* my digital photos and own written documents (i.e ignored the movies and oggs since they can be recreated from the CDs (though at a cost of 100 hours of work or something), even

I work for these guys, vaults in 3 redundant datacenters, 10 years doing remote backup for almost EVERY operating system out there and, most importantly, a thiry day free (as in beer) trial just to check out what we can do. Not to mention we're one of, if not the biggest, player in the continental US when it comes to remote data storage.

Jungle Disk [jungledisk.com] is a cross-platform front-end for Amazon S3 that supports Windows, Mac, and Linux. You only pay the Amazon fees ($0.15/gig/month). On Linux you can mount it directly using DavFS then backup using any software you want (rsync, etc). It supports encryption and caching as well.

You might want to check out S3/Jungledisk [jungledisk.com]. S3 by amazon is relatively cheap (15 cents/GB + transfer) and Jungledisk acts as WebDAV middleware. The middleware is still rather basic, but it works for backups, and passes the "are they going to be around in a couple of years" test. All data is encrypted by the Jungledisk middleware.

Although it might sound a little like spamming, the company I work for has a decent solution they just recently launched. I personally haven't tried it out myself, but all the current clients we have that use it love the service. You might give it a look:
www.indy-backup.com

I provide consulting services to Divinsa. They use BackupPC to provide online backups for unix machines such as Linux, Mac, and FreeBSD. Dual datacenters with the data replicated between them. Check them out at http://www.divinsa.com./ [www.divinsa.com]

www.dreamhost.com [dreamhost.com] Disk: 20GB/month, increases by 800MB/month; Bandwidth: 1TB/month, increases by 32GB/month. Price: $7.95/month. The longer you're a customer, the more space and bandwidth you get. Includes shell accounts, etc. On top of that, ssh/sftp/rsync bandwidth doesn't even count against your account usage. I use Duplicity (rdiff-backup+gzip+gnupg) to backup several gigs of data on a regular basis, and it's great.