Wed, 13 Jan 2016

Automating Deployments: Distributing Debian Packages with Aptly

Once a Debian package is built, it must be distributed to the servers it is
to be installed on.

Debian, as well as all other operating systems I know of, use a pull model
for that. That is, the package and its meta data are stored on a server that
the client can contact, and request the meta data and the package.

The sum of meta data and packages is called a repository. In order
to distribution packages to the servers that need them, we must set up and
maintain such a repository.

Signatures

In Debian land, packages are also signed cryptographically, to ensure
packages aren't tampered with on the server or during transmission.

So the first step is to create a key pair that is used to sign this
particular repository. (If you already have a PGP key for signing packages,
you can skip this step).

The following assumes that you are working with a pristine system user that
does not have a gnupg keyring yet, and which will be used to maintain the
debian repository. It also assumes you have the gnupg package
installed.

$ gpg --gen-key

This asks a bunch of questions, like your name and email address, key type
and bit width, and finally a pass phrase. I left the pass phrase empty to make
it easier to automate updating the repository, but that's not a
requirement.

$ gpg --gen-key
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: directory `/home/aptly/.gnupg' created
gpg: new configuration file `/home/aptly/.gnupg/gpg.conf' created
gpg: WARNING: options in `/home/aptly/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/home/aptly/.gnupg/secring.gpg' created
gpg: keyring `/home/aptly/.gnupg/pubring.gpg' created
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048)
Requested keysize is 2048 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
"Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>"
Real name: Aptly Signing Key
Email address: automatingdeployments@gmail.com
You selected this USER-ID:
"Moritz Lenz <automatingdeployments@gmail.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
You don't want a passphrase - this is probably a *bad* idea!
I will do it anyway. You can change your passphrase at any time,
using this program with the option "--edit-key".
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, utilize the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.
..........+++++
.......+++++
Not enough random bytes available. Please do some other work to give
the OS a chance to collect more entropy! (Need 99 more bytes)
..+++++
gpg: /home/aptly/.gnupg/trustdb.gpg: trustdb created
gpg: key 071B4856 marked as ultimately trusted
public and secret key created and signed.
gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
pub 2048R/071B4856 2016-01-10
Key fingerprint = E80A D275 BAE1 DEDE C191 196D 078E 8ED8 071B 4856
uid Moritz Lenz <automatingdeployments@gmail.com>
sub 2048R/FFF787F6 2016-01-10

Near the bottom the line starting with pub contains the key
ID:

pub 2048R/071B4856 2016-01-10

We'll need the public key later, so it's best to export it:

$ gpg --export --armor 071B4856 > pubkey.asc

Preparing the Repository

There are several options
for managing Debian repositories. My experience with debarchiver is mixed:
Once set up, it works, but it does not give immediate feedback on upload;
rather it communicates the success or failure by email, which isn't very
well-suited for automation.

Instead I use aptly, which works fine
from the command line, and additionally supports several versions of the
package in one repository.

To initialize a repo, we first have to come up with a name. Here I call it
internal.

This concludes the whirlwind tour through debian repository management and
thus package distribution. Next up will be the actual package
installation.

I'm writing a book on automating
deployments. If this topic interests you, please sign up for the Automating Deployments
newsletter. It will keep you informed about automating and continuous
deployments. It also helps me to gauge interest in this project, and your
feedback can shape the course it takes.