This article discusses common techniques available in Arch Linux for cryptographically protecting a logical part of a storage disk (folder, partition, whole disk, ...), so that all data that is written to it is automatically encrypted, and decrypted on-the-fly when read again.

"Storage disks" in this context can be your computer's hard drive(s), external devices like USB sticks or DVD's, as well as virtual storage disks like loop-back devices or cloud storage (as long as Arch Linux can address it as a block device or filesystem).

Why use encryption?

Disk encryption ensures that files are always stored on disk in an encrypted form. The files only become available to the operating system and applications in readable form while the system is running and unlocked by a trusted user. Reading the encrypted sectors without permission will return garbled random-looking data instead of the actual files.

For example, this can prevent unauthorized viewing of the data when the computer or hard-disk is:

located in a place to which non-trusted people might gain access while you're away

lost or stolen, as with laptops, netbooks or external storage devices

in the repair shop

discarded after its end-of-life

In addition, disk encryption can also be used to add some security against unauthorized attempts to tamper with your operating system. For example, the installation of keyloggers or Trojan horses by attackers who can gain physical access to the system while you're away.

Warning: Disk encryption does not protect your data from all threats.

Including the following:

Attackers who can break into your system (e.g. over the Internet) while it is running and after you've already unlocked and mounted the encrypted parts of the disk.

Attackers who are able to gain physical access to the computer while (or very shortly after) it is running, and have the resources to perform a cold boot attack.

A government entity, which not only has the resources to easily pull off the above attacks, but also may simply force you to give up your keys/passphrases using various techniques of coercion. In most non-democratic countries around the world, as well as in the USA and UK, it is legal for law enforcement agencies to do so if they have suspicions that you might be hiding something of interest.

A very strong disk encryption setup (e.g. full system encryption with no plaintext boot partition and authenticity checking) is required to stand a chance against professional attackers, who are able to tamper with your system before you use it. And even then it is doubtful whether it can really prevent all types of tampering (e.g. hardware keyloggers). The best remedy might be hardware-based full disk encryption (e.g. Trusted Computing).

Data encryption vs system encryption

Template:WikipediaData encryption, defined as encrypting only the user's data itself (often located within the /home directory, or on removable media like a data DVD), is the simplest and least intrusive use of disk encryption, but has some significant drawbacks.In modern computing systems, there are many background processes that may cache/store information about user data or parts of the data itself in non-encrypted areas of the hard drive, like:

Preventing unauthorized physical access to private data that may cached by the system.

Disadvantages:

unlocking/locking of the encrypted parts of the disk can no longer coincide with user login/logout, because now the unlocking already needs to happen before or during boot

In practice, there's not always a clear line between data encryption and system encryption, and many different compromises and customized setups are possible.

In any case, disk encryption should only be viewed as an adjunct to the existing security mechanisms of the operating system - focused on securing offline physical access, while relying on other parts of the system to provide things like network security and user-based access control.

Available methods

All disk encryption methods operate in such a way that even though the disk actually holds encrypted data, the operating system and applications "see" it as the corresponding normal readable data as long as the cryptographic container (i.e. the logical part of the disk that holds the encrypted data) has been "unlocked" and mounted.

For this to happen, some "secret information" (usually in the form of a keyfile and/or passphrase) needs to be supplied by the user, from which the actual encryption key can be derived (and stored in the kernel keyring for the duration of the session).

The available disk encryption methods can be separated into two types by their layer of operation:

Stacked filesystem encryption

Stacked filesystem encryption solutions are implemented as a layer that stacks on top of an existing filesystem, causing all files written to an encryption-enabled folder to be encrypted on-the-fly before the underlying filesystem writes them to disk, and decrypted whenever the filesystem reads them from disk. This way, the files are stored in the host filesystem in encrypted form (meaning that their contents, and usually also their file/folder names, are replaced by random-looking data of roughly the same length), but other than that they still exist in that filesystem as they would without encryption, as normal files / symlinks / hardlinks / etc.

The way it is implemented, is that to unlock the folder storing the raw encrypted files in the host filesystem ("lower directory"), it is mounted (using a special stacked pseudo-filesystem) onto itself or optionally a different location ("upper directory"), where the same files then appear in readable form - until it is unmounted again, or the system is turned off.

Block device encryption

Block device encryption methods, on the other hand, operate below the filesystem layer and make sure that everything written to a certain block device (i.e. a whole disk, or a partition, or a file acting as a virtual loop-back device) is encrypted. This means that while the block device is offline, its whole content looks like a large blob of random data, with no way of determining what kind of filesystem and data it contains. Accessing the data happens, again, by mounting the protected container (in this case the block device) to an arbitrary location in a special way.

The following "block device encryption" solutions are available in Arch Linux:

dm-crypt is the standard device-mapper encryption functionality provided by the Linux kernel. It can be used directly by those who like to have full control over all aspects of partition and key management.

LUKS is an additional convenience layer which stores all of the needed setup information for dm-crypt on the disk itself and abstracts partition and key management in an attempt to improve ease of use.

Preparation

Choosing a setup

Which disk encryption setup is appropriate for you will depend on your goals (please read #Why_use_encryption? above) and system parameters.
Among other things, you will need to answer the following questions:

Many many other combinations are of course possible. You should carefully plan what kind of setup will be appropriate for your system.

Choosing a strong passphrase

When relying on a passphrase, it must be complex enough to not be easy to guess or break using brute-force attacks. The tenets of strong passphrases are based on length and randomness.
Refer to The passphrase FAQ for a detailed discussion, and especially consider the Diceware Passphrase method.

Another aspect of the strength of the passphrase is that it must not be easily recoverable from other places.
If you use the same passphrase for disk encryption as you use for your login password (useful e.g. to auto-mount the encrypted partition or folder on login), make sure that /etc/shadow either also ends up on an encrypted partition, or uses a strong hash algorithm (i.e. sha512/bcrypt, not md5) for the stored password hash (see SHA_password_hashes for more info).

Preparing the disk

Before setting up disk encryption on a (part of a) disk, consider securely wiping it first. This consists of overwriting the entire drive or partition with a stream of zero bytes or random bytes, and is done for one or both of the following reasons:

prevent recovery of previously stored data

Disk encryption doesn't change the fact that individual sectors are only overwritten on demand, when the file system creates or modifies the data those particular sectors hold (see #How_the_encryption_works below). Sectors which the filesystem considers "not currently used" are not touched, and may still contain remnants of data from previous filesystems. The only way to make sure that all data which you previously stored on the drive can not be recovered, is to manually erase it.
For this purpose it does not matter whether zero bytes or random bytes are used (although wiping with zero bytes will be much faster).

prevent disclosure of usage patterns on the encrypted drive

Ideally, the whole encrypted part of the disk should be indistinguishable from uniformly random data. This way, no unauthorized person can know which and how many sectors actually contain encrypted data - which may be a desirable goal in itself (as part of true confidentiality), and also serves as an additional barrier against attackers trying to break the encryption.
For this purpose, wiping the disk using high-quality random data is crucial.

The second reason only makes sense in combination with block device encryptions, because in the case of stacked filesystem encryption the encrypted data is easily identifiable anyways (in the form of distinct encrypted files in the host filesystem). Also note that even if you only intend to encrypt a particular folder, you will have to erase the whole partition if you want to get rid of files that were previously stored in that folder in unencrypted form. If there are other folders on the same partition, you will have to back them up and move them back afterwards.

Once you have decided which kind of disk erasure you want to perform, refer to the Securely_wipe_disk article for technical instructions.

Tip: In deciding which method to use for secure erasure of a hard disk drive, remember that this will not need to be performed more than once for as long as the drive is used as an encrypted drive.

How the encryption works

This section is intended as a high-level introduction to the concepts and processes which are at the heart of usual disk encryption setups.

It does not go into technical or mathematical details (consult the appropriate literature for that), but should provide a system administrator with a rough understanding of how different setup choices (especially regarding key management) can affect usability and security.

Basic principle

For the purposes of disk encryption, each blockdevice (or individual file in the case of stacked filesystem encryption) is divided into sectors of equal lenght, for example 512 bytes (4,096 bits). The encryption/decryption then happens on a per-sector basis, so the n'th sector of the blockdevice/file on disk will store the encrypted version of the n'th sector of the original data.

Whenever the operating system or an application requests a certain fragment of data from the blockdevice/file, the whole sector (or sectors) that contains the data will be read from disk, decrypted on-the-fly, and temporarily stored in memory:

Similarly, on each write operation, all sectors that are affected must be re-encrypted complelety (while the rest of the sectors remain untouched).

Keys, keyfiles and passphrases

In order to be able to de/encrypt data, the disk encryption system needs to know the unique secret "key" associated with it. This is a randomly generated byte string of a certain length, for example 32 bytes (256 bits).

Whenever the encrypted block device or folder in question is to be mounted, its corresponding key (called henceforth its "master key") must be retrieved - usually from one of the following locations:

stored in a plaintext keyfile

Simply storing the master key in a file (in readable form) is the simplest option. The file - called a "keyfile" - can be placed on a USB stick that you keep in a secure location and only connect to the computer when you want to mount the encrypted parts of the disk (e.g. during boot or login).

stored in passphrase-protected form in a keyfile or on the disk itself

The master key (and thus the encrypted data) can be protected with a secret passphrase, which you will have to remember and enter each time you want to mount the encrypted block device or folder.

A common setup is to apply so-called "key stretching" to the passphrase (via a "key derivation function"), and use the resulting enhanced passphrase as the mount key for decrypting the actual master key (which has been previously stored in encrypted form):

The key derivation function (e.g. PBKDF2 or scrypt) is deliberately slow (it applies many iterations of a hash function, e.g. 1000 iterations of HMAC-SHA-512), so that brute-force attacks to find the passphrase are rendered infeasible. For the normal use-case of an authorized user, it will only need to be calculated once per session, so the small slowdown is not a problem.It also takes an additional blob of data, the so-called "salt", as an argument - this is randomly generated once during set-up of the disk encryption and stored unprotected as part of the cryptographic metadata. Because it will be a different value for each setup, this makes it infeasible for attackers to speed up brute-force attacks using precomputed tables for the key derivation function.

The encrypted master key can be stored on disk together with the encrypted data. This way, the confidentiality of the encrypted data depends completely on the secret passphrase.

Additional security can be attained by instead storing the encrypted master key in a keyfile on e.g. a USB stick. This provides two-factor authentication: Accessing the encrypted data now requires something only you know (the passphrase), and additionally something only you have (the keyfile).

Another way of achieving two-factor authentication is to augment the above key retrieval scheme to mathematically "combine" the passphrase with byte data read from one or more external files (located on a USB stick or similar), before passing it to the key derivation function.The files in question can be anything, e.g. normal JPEG images, which can be beneficial for #Plausible Deniability. They are still called "keyfiles" in this context, though.

randomly generated on-the-fly for each session

In some cases, e.g. when encrypting swap space or a /tmp partition, it is not necessary to keep a persistent master key at all. A new throwaway key can be randomly generated for each session, without requiring any user interaction. This means that once unmounted, all files written to the partition in question can never be decrypted again by anyone - which in those particular use-cases is perfectly fine.

After is has been derived, the master key is securely stored in memory (e.g. in a kernel keyring), for as long as the encrypted block device or folder is mounted.

It is usually not used for de/encrypting the disk data directly, though.
For example, in the case of stacked filesystem encryption, each file can be automatically assigned its own encryption key. Whenever the file is to be read/modified, this file key first needs to be decrypted using the main key, before it can itself be used to de/encrypt the file contents:

Ciphers and modes of operation

The actual algorithm used for translating between pieces of unencrypted and encrypted data (so-called "plaintext" and "ciphertext") which correspond to each other with respect to a given encryption key, is called a "cipher".

Disk encryption employs "block ciphers", which operate on fixed-length blocks of data, e.g. 16 bytes (128 bits). At the time of this writing, the predominantly used ones are:

developed as successor of Blowfish, but has not attained as much widespread usage

Encrypting/decrypting a sector (see above) is achieved by dividing it into small blocks matching the cipher's block-size, and following a certain rule-set (a so-called "mode of operation") for how to consecutively apply the cipher to the individual blocks.

Simply applying it to each block separately without modification (dubbed the "electronic codebook (ECB)" mode) would not be secure, because if the same 16 bytes of plaintext always produce the same 16 bytes of ciphertext, an attacker could easily recognize patterns in the ciphertext that is stored on disk.

The most basic (and common) mode of operation used in practice is "cipher-block chaining (CBC)". When encrypting a sector with this mode, each block of plaintext data is combined in a mathematical way with the ciphertext of the previous block, before encrypting it using the cipher. For the first block, since it has no previous ciphertext to use, a special pre-generated data block stored with the sector's cryptographic metadata and called an "initialization vector (IV)" is used:

One thing worth noting is the generation of the unique initialization vector for each sector. The simplest choice is to calculate it in a predictable fashion from a readily available value such as the sector number. However, this might allow an attacker with repeated access to the system to perform a so-called watermarking attack. To prevent that, a method called "Encrypted salt-sector initialization vector (ESSIV)" can be used to generate the initialization vectors in a way that makes them look completely random to a potential attacker.

There are also a number of other, more complicated modes of operation available for disk encryption, which already provide built-in security agains such attacks.
Some can also additionally guarantee authenticity (see below) of the encrypted data.