Synchronization and backup programs

This page lists and compares applications that synchronize data between two or more locations, and those that build on top of such functionality to make incremental copies of important data for backup purposes. Because of their relationship, the two groups share several traits that justify describing them in the same article.

Backup overview

Having backups of important data is a necessary measure to take, since human and machine processing errors are very likely to generate corruption as time passes, and also the physical media where the data is stored is inevitably destined to fail. In order to choose the best program for one's own needs, the following aspects should be considered:

The features expected from the backup solution, e.g. compression, encryption, handles renames, etc.

The planned method to restore backups if needed.

Data synchronization

These applications simply keep directories synchronized between multiple locations/machines, in a "mirror" fashion. Nonetheless, most of them still allow storing and reverting to old revisions of modified or deleted files.

Name: the application name, linking to the ArchWiki article or the official website.

Installation: a link to the package.

Implementation: the programming language, library, or utility that the application is based on.

Delta transfer: only the modified parts of files are transferred.

Encrypted transfer: data is encrypted by default when transferred over the network.

FS metadata: file system permissions and attributes are synchronized.

Resumable: the synchronization can be resumed if interrupted.

Handles renames: moved/renamed files are detected and not stored or transferred twice. It typically means that a checksum of files or its chunks is computed.

Version control: the old version of files are backed up (reverse incremental backup).

Change propagation: specifies in how many directions changes can be propagated.

unidirectional means one-way synchronization of two locations,

bidirectional means two-way synchronization of two locations and

multidirectional means full synchronization of more than two locations.

Conflict resolution: the application handles file conflicts, either automatically or interactively, i.e. it does not silently discard conflicting files. This attribute does not apply to applications that only propagate changes in one direction.

FS monitoring: the application listens to file system events to trigger the synchronization.

CLI: the application is command-line driven, i.e. it is scriptable.

Other interfaces: the application has the specified user interfaces, e.g. GUI, TUI, or web-based.

License: the license of the server and client applications.

Other platforms: supported operating systems other than Linux.

Specificity: brief notes about special features that notably set the application apart from the others.

Incremental backups

Applications that can do incremental backups remember and take into account what data has been backed up during the last run (so-called "diffs") and eliminate the need to have duplicates of unchanged data. Restoring the data to a certain point in time would require locating the last full backup and all the incremental backups from then to the moment when it is supposed to be restored. This sort of backup is useful for those who do it very often.

Installation: a link to the main ArchWiki article, if existing, or directly to the package pages.

Implementation: the programming language, library, or utility that the application is based on.

Compressed storage: compression is used for storage.

Encrypted storage: encryption is used for storage.

Delta transfer: only the modified parts of files are transferred.

Encrypted transfer: data is encrypted by default when transferred over a network.

FS metadata: file system permissions and attributes are backed up.

Easy access: the backup is stored plainly in the file system, or is mountable as such.

Resumable: the backup can be resumed without restarting it if interrupted.

Handles renames: moved/renamed files are detected and not stored or transferred twice; it typically means that a checksum is computed for files or chunks thereof.

CLI: the application is command-line driven, i.e. it is scriptable.

Other interfaces: the application has the specified user interfaces, e.g. GUI, TUI, or web-based.

Licence: the licence of the server and client applications.

Other platforms: supported operating systems other than Linux.

Specificity: brief notes about special features that notably set the application apart from the others.

Single machine

These applications are aimed at backing up data from the machine they are installed on, although the backup destination can be located on an external machine or storage media.

Chunk-based increments

If a file is modified, these applications store only its changed parts at the next snapshot. Compared to #File-based increments applications, these are more space-efficient, especially when large files receive small modifications; on the other hand, the archived snapshots have to be opened with the backup application that created them, since the files have to be reconstructed from the stored binary diffs.

File-based increments

If a file is modified, these applications store its new version entirely at the next snapshot. Compared to #Chunk-based increments applications, these are less space-efficient, especially when large files receive small modifications; on the other hand, often the archived snapshots can be opened without the need to have the backup application installed.

Network oriented

These applications have been designed to centralize the backup of several machines connected to a network, through a server-client model. In general they are more complicated to deploy, compared to #Single machine solutions.

Cloud storage

Third-party services

Notes: Remove the extra bullet points: the Wikipedia comparison table, and the respective webpages and Wikipedia articles do a better job at describing them. (Discuss in Talk:Synchronization and backup programs#)

Tip:cryptomatorAUR is an open-source, multi-platform program designed to add client-side transparent encryption on cloud-shared files.

Amazon S3 — Amazon Simple Storage Service (Amazon S3), provides developers and IT teams with secure, durable, highly-scalable object storage. Amazon S3 is easy to use, with a simple web service interface to store and retrieve any amount of data from anywhere on the web. With Amazon S3, you pay only for the storage you actually use. There is no minimum fee and no setup cost.

rclone — Rclone is a command line program to sync files and directories to and from Google Drive, Amazon S3, Openstack Swift / Rackspace cloud files / Memset Memstore, Dropbox, Google Cloud Storage and The local filesystem.