In this handbook we will continue with our Jenkins handbook series and
we’ll see how we can use Jenkins for package building under FreeBSD.

The package building process will be initiated and managed by Jenkins
and for all the heavy lifting of the package building itself we will
use
ports-mgmt/poudriere.

We all know that we can use
ports-mgmt/tinderbox
for this task already, but the purpose of this handbook is to show you
one more thing you can use Jenkins for - continuous building and
testing of packages for your systems.

Tinderbox also comes with a web
interface and it is easy to schedule a build from just using the web
interface.

On the other side
ports-mgmt/poudriere
is a command-line package test and build system and does not provide
any web interface currently, but using Jenkins we will change that and
we will use the nice web interface of Jenkins integrated with
poudriere.

Another benefit of using poudriere with Jenkins rathen than using
Tinderbox is that poudriere is quite easy to configure and manage
and you won’t fall into the configuration hell a lot of users have
been complaining about while configuring Tinderbox.

One more cool feature of poudriere, and I really like this -
poudriere is able to perform incremental package builds.

Please note that the build server does not require to be converted
to pkgng.

Configuration of poudriere

Now lets configure poudriere. To do that simply copy
/usr/local/etc/poudriere.conf.sample to
/usr/local/etc/poudriere.conf and edit the file. The options in the
file are quite well documented so everything should be pretty simple
to configure.

Below you can see the configuration I’ve used for poudriere in this
setup:

Creating a ports tree for poudriere

Now that we have poudriere configured, lets create a FreeBSD Ports
Tree to be used by poudriere.

To do that execute the command below on the build server:

$ sudo poudriere ports -c

Creating poudriere jails

For the purposes of this handbook we will create only one FreeBSD
9.0-RELEASE amd64 poudriere jail for package building.

So lets create the poudriere jail:

$ sudo poudriere jails -c-j 90amd64 -v 9.0-RELEASE -a amd64

poudriere is able to create legacy packages as well as
pkgng
packages. pkgng users who want to
have pkgng packages created will need to execute the commands below
in order to make poudriere pkgng-aware.

Creating a Jenkins user on the build server

Our Jenkins CI instance will be connecting to the poudriere build
server over SSH and then running the package build, so we will need a
Jenkins user with proper privileges in order to authenticate, run the
build on the poudriere build server and afterwards sync the packages
to the Jenkins CI instance.

In order to avoid confusion we will use the user jenkins-builder on
the poudriere build server and jenkins user on the Jenkins CI
instance.

In order to create the Jenkins user, login to the poudriere build
server and execute the commands below:

And now create the SSH keys for the jenkins-builder user on the
build server.

$ sudo-u jenkins-builder ssh-keygen

One last thing we need to configure is to allow our jenkins-builder
user to run poudriere builds, so edit your /usr/local/etc/sudoers
file and allow jenkins-builder to run poudriere. Adding the below
content to your /usr/local/etc/sudoers file would do the job:

jenkins-builder ALL=(root) NOPASSWD: /usr/local/bin/poudriere

Our build server with poudriere is now fully configured. In the next
chapter of this handbook we will integrate Jenkins with poudriere.

Now it is time to create a Jenkins job that will do continuous
building and testing of our packages.

Installing and configuring required Jenkins plugins

Jenkins PostBuildScript
Plugin
will be used to execute a shell script in the Post-build Actions
section of our project for syncing the already built packages and the
Jenkins Publish Over SSH
Plugin
will be used for connecting to our build server using SSH keys,
performing the build process and afterwards syncing back to Jenkins
the already built packages.

To do that navigate to Manage Jenkins -> Configure System and
scroll down to the SSH section as shown in the screenshot below.

In the SSH section we need to specify the location to the private
SSH key used by the Jenkins instance, which in the above screenshot is
/usr/local/jenkins/.ssh/id_rsa. The private SSH key for Jenkins you
can create by logging into the Jenkins instance and executing the
below command:

$ sudo-u jenkins ssh-keygen

Now copy the public SSH key for user jenkins (which should be in
/usr/local/jenkins/.ssh/id_rsa) from the Jenkins CI instance to the
poudriere build server so that we can connect to our build server
and run package builds. To do that from the Jenkins instance execute
the command below:

$ sudo-u jenkins ssh-copy-id jenkins-builder@poudriere.example.org

Remember that we are using jenkins-builder user on the poudriere
build server and jenkins user on the Jenkins instance.

If you do not have ssh-copy-id(1) package installed then just
copy-paste the public ssh key of jenkins user to
/home/jenkins-builder/.ssh/authorized_keys on the build server. Test
that you can succesfully connect to the build server from Jenkins as
well when ready.

Now that we have configured SSH key connection between the build
server and Jenkins instance, lets configure Jenkins itself so that it
can use that connection.

To do that lets add our poudriere build server to the SSH Servers
section in Jenkins as shown on the screenshot below. When ready click
on the Test Configuration button to test the connection between
Jenkins and the poudriere builder server.

If the test was successful as shown on the screenshot above click on
the Save button and we are ready with the configuration of the
Jenkins Publish Over SSH
Plugin.

Creating the Jenkins poudriere job

Now, lets create our Jenkins project for running poudriere package
builds. Navigate to New Job and give the new project a meaningful
name. In this example for the project name I will use the
freebsd-9.0-amd64-builder name as shown on the screenshot below.

Once the new project is created we need to configure it - project
description, SCM, number of builds to keep, etc.

We do not use an SCM system, so we leave the Source Code Management
section of the project empty.

The next section we configure is the Build Triggers one. This is
where we configure when a project is being scheduled for a build. I
choose to leave this section of the project empty and trigger a build
manually, but you may wish to enable periodic builds if needed. Feel
free to play with this section of the project.

Now lets configure the build itself. This is where we define how we
trigger the poudriere builds. To do that we configure the Build
section of our project as shown in the screenshot below.

In order to add the build step click on the Add build step button
from the Build section of the project and select the Send files or
execute commands over SSH option.

Then select the poudriere-builder server and in the Exec command
field we specify the commands used for building the packages as shown
in the screenshot below.

The Build section of the project is a simple shell script that takes
care of building our packages with poudriere.

Below is listed the shell script used for building the packages with
poudriere:

There are only two things you need to change in the above script - the
poudriere jail you are building packages on as defined by
POUDRIERE_JAIL variable and the packages you want to build as
defined by the PACKAGES_TO_BUILD variable.

In the above example script we are going to build the packages below
on the 90amd64poudriere jail:

security/apg

ports-mgmt/pkg

sysutils/zfsnap

ports-mgmt/poudriere

shells/zsh

Now lets configure the Advanced settings of our project, to do that
click on the Advanced* button and check the Fail the build if an
error occurs as shown in the screenshot below:

One last thing we need to configure in the Build section of our
project is to check the Exec in pty option and set the Exec
timeout as shown in the screenshot below. We set the Exec timeout
to 0 as we don’t know how long a build is going to last.

The last thing we need to configure about our project is to sync the
already built packages back to Jenkins so that we can make them
available for installation. To do that we configure the Post-build
Actions section of our project.

In the Post-build Actions section of your project click on the
Post-build action button and select [PostBuildScript] - Execute a
set of scripts then from the Build steps select Execute shell as
shown in the screenshot below:

Our shell script in the Post-build Actions section only serves for
syncing the packages from the build server to the Jenkins instance and
is configured as shown on the screenshot below.

The sync script used in the Post-build Actions section is listed
below:

Clone an already existing Jenkins builder and edit only the
variables in the scripts like POUDRIERE_JAIL, BUILD_SERVER,
etc..

Run a build on your new Jenkins poudriere builder

And that’s it, cloning an existing builder in Jenkins is easy, to do
that simply navigate to New Job -> Copy existing Job, provide the
new job a name and select an already existing Jenkins builder.

Further reading

As we’ve seen in this handbook we can use Jenkins CI for continuous
package building. We have seen how to configure a Jenkins project that
does continuous building and testing of packages on our FreeBSD
systems.

In this handbook we’ve configured only a single Jenkins job for
package building, but one can imagine that we can simply extend this
setup to include multiple servers in a build farm managed by Jenkins.