Linux Backup Solutions

I've been looking for the best Linux backup system, and also reading lots of HN comments.

Instead of putting pros and cons of every backup system I'll just list some deal-breakers which would disqualify them.

Also I would like that you, the HN community, would add more deal breakers for these or other backup systems if you know some more and at the same time, if you have data to disprove some of the deal-breakers listed here (benchmarks, info about something being true for older releases but is fixed on newer releases), please share it so that I can edit this list accordingly.

@sammcj Why would you dismiss Amanda? It's actually pretty awesome and well seasoned, the enterprise version only gets you appliances for noobs who can't operate outside of a point and click paradigm and crap themselves when they see a terminal. Config management + Amanda (especially ZRM for LVM snapshot based backups of MySQL clusters) == profit.

@mikerev - I don't think that having a UI of sorts of a backup system makes it for 'noobs' - backups can quickly drain time from your team if they're not quick and easy to manage.

Some times a UI/GUI/WebUI is the most effective tool for a job (but usually not) - I think backup is an area that benefits from a good UI (especially if it's backend is also easily configurable using automation tools or standard structures such as yaml).

I've been using Amanda for the last three years with approx 350~ servers, I really haven't been impressed.

It has a lot of management overhead and that's a problem if you don't have time for a full time backup administrator.

It mainly comprises of using tar for backups which is pretty inflexible by modern standards.

The enterprise web interface is OK but it's had so many bugs it's not funny.

Backups are very slow.

Restores are slow and painful to manage.

I haven't found it to be great when trying to integrate with puppet / automation frameworks.

For Snebu, would it be sufficient to use something like a LUKS encrypted disk volume as the backup storage device? I'd really rather leave encryption to the experts, instead of rolling my own (even if I use a well tested library). Also proper encryption (with unique salt for each file) would break deduplication.

Of course this isn't ideal for remote backups, but for that I think the best plan would be to back up to a local device (encrypted at the FS layer), then once I get the replication code in place, an encrypting replication module can be used to send offsite (Amazon, rsync.net, google cloud, etc) or to tape.

Edit: For encryption to a remote device -- run Snebu locally, and set it up to have its vault on a network file system that does local encryption. You'd want to keep the sqlite catalog DB local for performance reasons, then write out a compressed copy of that to your backend storage when the backup is finished.

As of early 2015, Mondo Rescue v3.2.x seems to have managed to cope with the uncontrolled morass of 'systemd' dependencies (and systemd's inscrutable, binary config files and so on). I haven't tried it (yet). Before systemd, Mondo Rescue was a a remarkably powerful and easy-to-use (full-system-image) backup (and disaster recovery from bootable device/CD/DVD) tool. It should be worth a try ;-)

BackupPC up to v3 doesn't encrypt the pool of the deduplicated files (FDE can save you from off-line access anyway), but you can easily configure an archive host where you put encrypted tarballs (you can hook in with scripts at any[?] stage of the backup). e.g. use an on-site BackupPC setup which puts the pool on a encrypted partition/container/folder/etc. and use off-site storage as archive hosts.

Are you aware that your definition of your use case "the best Linux backup" opens the flood gates for bike shedding comments?

My advice: start with a (superficially) easy solution and try it out; read or at least skim and tag the mailing list of that software often.

@drkarl, is lack of encryption the only deal breaker for Snebu? Or is it the primary deal breaker? (You may want to add "file level, not block level deduplication", as this impacts backing up VM images, although an add-on that specifically address VMs is in the works). If encryption is the main issue, I've put together a plan to address this without compromising some of the other features (such as minimal client-side requirements) -- I should be able to code it up this weekend.

Backupninja didn't support Attic the last time I checked. Also not sure whether Attic does block based de-duplication or it's just file level. The former would make it a killer - that's one good strong point Tarsnap has that is missing in many backup clients.

I think we will never find the "best" backup tool as everyone has different needs. For example, I do not want to compress backups as they are very big and takes a lot of time and resources, while for others, compression is a must.

My crontribution are Fwbackups which is an easy tool to backup both locally and remotelly (compressing if needed)

Bera Backup another open source tool to backup files/folders but also configurations (crontabs, users, system config...) to easily replicate a system.

Amanda (stated above) is probably the best tool, very powerful but also more complex than the others...

We're still pretty keen to replace Amanda backup with something, we've had so many problems with it again recently, it'd be great if there was a simple web interface that provided functionality similar to that of ninjahelper but also gave a breakdown of backup timelines and offered restore options.

I had some experience with backup tools and I came finally to these tools:Zpaq, attic, Dar and Duplicity (Deja-Dup) (Obnam, Zbackup and bup was considerable but I dismess them after a while). The way bup handles deduplication is very interesting but I didn't test it since it had no encryption at the time.

As far as I remember:
The best regarding Simplicity and Ease of Use: Deja-Dup
feature rich: attic
and Fast and compression/deduplication effecient: Zpaq

Currently I'm just using Zpaq, Deja-Dup(Duplicity) and one archive with attic. Attic is slow in comparison to Dar and Zpaq here are my benchmark (I found it finally):
attic on HugeRepo with 120+GB takes ~5.5Hr and 100 GB final deduplicated backup
attic on MediumRepo with near 26GB takes ~1Hr and 13GB final deduplicated backup
attic on TinyRepo with near 4GB takes 19min and 1.94GB final deduplicated backup
zpaq on MediumRepo with 26GB takes 1300Sec and 12.2GB
zpaq on TinyRepo with near 4GB takes 170Sec and 1.7GB
(I'm not sure about parameters of each command)

Thanks @ThomasWaldmann, I made it to the end of the comments just to look for updates. As now it's 2017 a lof of the solutions mentioned here are very out of date and their lack of development is prevalent. (Almost half are no longer being maintained, it seems.) Some like attic haven't had any new commits in years, unfortunately.

Any more updates are appreciated. (Comments about stable software not needing commits and such notwithstanding, the focus is to trust in something that will continue to work)

Thanks for the census @drkarl. For our Windows workstations, we settled on Duplicati, which uses a block-based deduplication algorithm to allow incremental backups to local, remote, or cloud object storage indefinitely. So the first backup is the big one, but all backups are incremental from there. It is open source and runs on the major unices. We are experimenting with it under Linux, and it does feel a bit weird running a C# program under Linux, but until someone writes an open source program with similar abilities to hashbackup, it seems to be the only game in town.