FAQ for package maintainers

Why should I use autopkgtest when my build runs the test suite?

autopkgtest is intended to test packages in their installed state,
and that is not the same as testing the code that is in the source tree.
During the build, the tests run against the code in the source tree.
Depending on the package, that could be enough, but also running
those same tests against the installed package can catch a host of problems
that can't be caught when running against the source tree, e.g.:

ABI breakages and other types of problems that are “magically” fixed with a
rebuild.

Missing dependencies (as opposed to build dependencies)

Some files that are essential for the software to function correctly were
not included in the binary package.

Another practical reason to have autopkgtest tests in your package is that
debci runs autopkgtest more frequently than packages are built. For
example, when any of your dependencies is updated, your package is tested
again against that new version.

Now, besides running tests that normally run during the package build under
autopkgtest as well, it is also a good idea to add tests that are specific
to the package as installed and are more “black box”, i.e. tests that
don't assume or have access to implementation details of the package
(those are usually called integration tests). For example:

if a package provides a service, add tests that exercises that service as a
client, and check the expected results of that interaction.

if a package provides a program, add tests that check that the invocations
of that program with different parameters and inputs produce the
corresponding expected results.

This type of test can also, and most often that not should, be contributed
upstream.

If autopkgtest can rebuild the package, why won't debci do that?

The primary reason is because we want to test the packages that are
actually in the archive, because that is what our users get from us.

Weaker, but practical reasons, are:

Rebuilding every package would add significant load to the system.

The buildd network already builds packages and we don't want to
duplicate that function.

How can I reproduce the test run locally?

NOTE: if you intend to run tests frequently, you should
consider installing apt-cacher-ng before anything else.
debci will notice the running proxy and will setup the testbed
to use it, so you won't have to wait for the download of each package
more than once.

Using no virtualization backend

This is the fastest way to run tests, but does not reproduce the continuous
integration environment faithfully and your local environment may make the
results unreliable. It's useful nevertheless when initially writing the
tests, or for quick runs against your main host system.

To run the test suite from the root of a source package
against the currently installed packages, run:

$ autopkgtest --output-dir /tmp/output-dir -B . -- null

For more details, see the documentation for the autopkgtest
package.

Using the lxc backend (default as of debci >= 1.0)

First step is to configure networking for the lxc containers. The easiest
way to to that is by using the libvirt networking support. First, install
the necessary packages and activate the default libvirt network:

To speed up test suite execution, you can also add the following line at
the end:

union-overlay-directory=/dev/shm

This will mount the chroot overlay on tmpfs which will make
installing test dependencies a lot faster. If your hard disk is already a
SSD, you probably don't need that. If you don't have a good amount
of RAM, you may have problems using this.

The following examples assume:

suite = unstable (the default)

architecture = amd64

To run the test suite from a source package in the
archive, you pass the package name to autopkgtest: