Friday, September 20. 2013

Automatically transferring (syncing) files between multiple computers is easy these days. Dropbox, owncloud or bitpocket to name a few. You can imagine I use the latter (if you want a recommendation)1.

In any case you want to encrypt what you send to be stored in "the cloud" even if it is just for a short time. There are many options how to encrypt the "in flight" data. Symmetric ciphers are probably the safest and most widely researched cryptography these days and easier to use than asymmetric key pairs in this context as well.

Encryption is notoriously hard to implement correctly and worthless when the implementation is flawed. So I looked at gpg, a well known reference implementation, and was amazed that it can neither use a proper keyfile for symmetric encryption (you can just supply a passphrase via --passphrase-file) nor does it handle multiple files on the command line consistently.
You can use --multifile (wondering...why does a command need that at all?) with --decrypt and --encrypt (asymmetric public/private key pair encryption) but not with --symmetric (symmetric shared key encryption). Duh!

With a bit of scripting around the gpg shortcomings, you end up with crypt_gpg that can nicely encrypt or decrypt multiple files (symmetric cipher) in one go.

Dropbox is closed source so it cannot be assessed for its security. Owncloud needs a thorough code review before I would dare to run it on my systems. ↩

#!/bin/bash# crypt_gpg v0.1# Copyright (c) 2013 Daniel Lange, http://daniel-lange.com.# Released into the public domain. NO LIABILITY ACCEPTED WHATSOEVER. USE AT YOUR OWN RISK.

Now the file pointed to by CRYPT_KEY is your keyfile, your shared secret that the PCs you want to sync need to have available to be able to decode the .gpg files.

Gpg only reads the first line from the keyfile (as it considers it to be a passphrase). Thus we need to make sure it does not contain \n (line feed) and for the Mac and Windows folks \r (carriage return). The tr command inside the crypt_gpg script above takes care of that but it should be good practice to make sure your keyfile also works around those gpg limitations. I also added \000 (NUL) to the exception list. C code, you know...

So, to generate your keyfile:

First: Ensure your system has enough entropy available for generating such a key.
cat /proc/sys/kernel/random/entropy_avail needs to return >2000 bits or you need to fix your entropy source first. Do this by installing haveged (apt-get install haveged on Debian and Ubuntu) or running guchaos as a one time measure.

Second: Generate a 1024 byte keyfile (128 bytes would be more than enough if you run low on entropy or processing time matters).
It is again critically important to strip line ending and NUL characters from the raw key material (as gpg would consider them to indicate the end of the passphrase).

Obviously you can make as many keyfiles as you wish for different setups.

Third: transfer the keyfile to all PCs that ought to sync via a secure channel (ssh is fine). The server (dropbox, owncloud or your own ssh, web or ftp server) should not have a copy of the keyfile. This way, even if the .gpg files are compromised, they are impossible to decode with any known means. Notice: Everybody who has access to the keyfile can decode the .gpg files lying around on storage media somewhere, so keep it safe. It is your secret encryption key.

Fourth: Every time you now run crypt_gpg myfile.doc anotherfile.txt /path/to/yetanotherfile.ods this will result in *.gpg files in the same directories as the original files.
Make sure to only add *.gpg files to your sync profile(s) and run a manual synchronization to see the *.gpg files transfer.

Fifth: On the receiving end(s) run crypt_gpg --decrypt *.gpg /path/to/*.gpg after a synchronization has taken place and have the original files reconstructed.

Very nice. It would be also cool to have a script which is synchronizing a whole local folder with a cloud folder on fly with encryption. Maybe this could be also done with scripting by checking the last modified date. Are there any solutions out there already that can do that? I'd love to do it with gpg though. I'll give it a try and post my solution here.

This is trivial with eCryptfs. You basically sync the encrypted (.ecryptfs/) folder and "the cloud" basically only sees and syncs the encrypted files.
Obviously that does not allow for things like conflict detection. The last one always wins.

https://github.com/DanHerbert/CloudSync is a wrapper around duplicity which in turn uses gnupg.
You may be able to extend this into two-way sync.

I'd prefer the eCryptfs option (if I were to use "the cloud" for any personal data at all) but you may have different preferences so give it a try.

If you need something more capable / user friendly, try Seafile or Pydio.

E-Mail addresses will not be displayed and will only be used for E-Mail notifications.

To prevent automated Bots from commentspamming, please enter the string you see in the image below in the appropriate input box. Your comment will only be submitted if the strings match. Please ensure that your browser supports and accepts cookies, or your comment cannot be verified correctly.

Enter the string from the spam-prevention image above:

Remember Information?

Submitted comments will be subject to moderation before being displayed.