Flavio Poletti

I’m probably not the only one in the world to work in an Enterprise-like
world. Which basically boils down to little Perl knowledge (at least
in the admittedly few Enteprises I got in contact with) and usage of
streamline Linux distribution with horribly old Perl versions.

What I’m probably, again, not the only one in the world to do is to
just set up my own perl installation for usage by my applications. This
makes me comfortable about what I’m going to use, without messing with
the system’s perl. After that, of course, there comes the problem of
many possible applications living together… so I have to cope with
this as well.

One additional problem that might kick-in is when I have to use a different
perl version for an application. As an example, I recently discovered
Regexp::Grammars is compatible with perl starting from version 5.10, but
only if different from 5.18… so what if my environment is 5.18? There might
be the need to handle different perl installations too, then.

There are currently a lot of tools that simplify the task of having
one’s own perl, keeping a private library for an application, possibly
shipping also the modules because there’s no connectivity to the
Internet… let’s see one possible workflow.

The Plan

The plan is very simple:

install plenv to address the possibility of multiple Perl versions. It
is quite lean and it seems to do its work without getting too much in
the way

build a private perl through perl-build, which can be installed as
a plugin of plenv, so that we can use plenv only

install cpanminus (a.k.a. cpanm) to easily handle module installation
later. Again, this can be handled as a plugin of plenv, we’ll see how

declare the dependencies of our application in the root directory of the
application itself, through a cpanfile declaration.

use carton to handle the installation of an application-private library
of code from the dependencies, so that we will be able to decouple from
the shared libraries installed in whatever perl we will use. This allows
e.g. to have two different application use two different versions of a
library, but leverating on the exact same perl installation. Carton
will also help us run our application with the right environment, more on
this later.

Should you have your development and deployment environment equal to each
other? Yes and no. For example, I already have [perlbrew] installed on my
dev machine, and the shift to plenv is something to do with a bit of
calm… so as long as I’m sure that I use the same perl version in the
two places I should be fine. This means that steps 1..3 above will in
general be a pre-requisite on the target deployment machine, while
steps 4 and 5 are more on the development machine.

So let’s start!

Deployment Environment Set-Up

Depending on the target machine you will be using, it might be easy or
impossible to actually install your own perl and its modules (especially
when compilation is needed). If this is the case, try to see if you can
create a compatible environment somewhere else, e.g. in a virtual machine
in your computer or online, so that you can be pretty sure that when you
copy things over you will be fine.

In the following, then, we will assume
that the deployment machine is equipped with all the tools needed to
compile and install perl and modules, which e.g. for a Debian release
would mean ensuring that package build-essentials is in place, with
the addition of a system perl, curl and git. For other
enterprise-like distributions like RHEL and SLES there are the
applicable package managers, otherwise you will have to roll your own!

Before starting, we have to note that there is absolutely no need to
be root here. As a matter of fact, it’s probably better not to be
root at all.

The installation of plenv is (or can be) as simple as cloning its
repository on GitHub. We’ll also take care to install the other tools
as suggested by the documentation, that I will blatantly copy here (or so):

cd ~
git clone git://github.com/tokuhirom/plenv.git .plenv

Now it’s time to add ~/.plenv/bin to your PATH environment variable.
Where it is stored/set is actually a matter of the system you are in,
so your mileage may vary with the suggestion provided in the plenv
documentation. Good places to look are:

~/.bashrc or ~/.bash_profile or ~/.profile if you’re using bash

~/.zshrc if you’re using zsh

whatever provided by the other shells

You can either edit the place where PATH is set, or add a line like this
at the end (file and syntax depend from the shell and considerations above):

echo 'export PATH="$HOME/.plenv/bin:$PATH"' >> ~/.bashrc

At this point, you can execute the following and have plenvvisible
in your shell:

exec "$SHELL" -l

Time for initializing plenv now. A sub-command tells you what to add
to the file you modified for the PATH variable, just run:

plenv init -

and look carefully at the output. For example, I have (more or less) this:

If it’s OK with your policies, then you can proceed in installing those
lines like you did for PATH modification above. Actually, what is
suggested by the author of plenv is to add this:

echo 'eval "$(plenv init -)"' >> ~/.bash_profile

which is less save than installing the lines above. Unless you are going
to update plenv in the future (e.g. with git pull at some time) you’re
fine, but if you do update you might end up with less secure things
happening during that init process that would be executed at every
login… the choice is yours to take. My only consideration is that you
probably already rely upon work done by complete strangers, and this seems
more or less the same situation.

Perl-build is a separate tool from the same author, and
is targeted at assisting in the installation of a new perl. Being from
the same author, anyway, makes it easy to integrate with plenv, which
is what we will do here:

Your first perl

It’s time for some serious compilation now. Make sure you know which
perl version you need, and ask plenv to install it (it will use
perl-build behind the scenes for the heavy-lifting):

plenv install 5.20.1 # use *your* perl version of course!

Wait a few minutes for the installation to complete, then let plenv
regenerate the shims:

plenv rehash

Now, this is written in the documentation and I respect it, but I
wonder whether this step is really necessary and, if it is, why at
all. Can’t this be done as the final part of plenv install ...?
Maybe it is (looking at the contents of ~/.plenv/shims it appears
to be so) and I can’t read the documentation properly, go figure.

Another capability offered by plenv is to install cpanminus. I think
most know it today, so I’ll not add anything on it.

I don’t really like do to this using plenv though, because especially
in deployment machines I prefer to use the fatpacked self-containing
version. We already added a couple of directories to the PATH, so
we will install it in one of them:

# you can do this with wget as well, of course
curl -L 'https://github.com/miyagawa/cpanminus/raw/devel/cpanm' \
> ~/.plenv/bin/cpanm
chmod a+x ~/.plenv/bin/cpanm

This should Just Work.

Time to pack…

The deployment server is ready at this point. Ok, sort of at least for my
taste.

First of all there’s a lot of cruft left by plenv after installing
perl, so I usually get rid of it:

cd ~/.plenv
rm -rf build cache

For perl 5.20.1 this saved me some 170+ MB, which are not bad when you
eventually have to pack it all for distributing into multiple deployment
machines. What I ended up was about 66 MB that shrink down to about 17 MB
after bzip2 compression, so it’s perfectly acceptable to have human
transfer time (unless your datacenter has shiny gigabit or multi-gigabit
networking equipment).

Another thing that I like to have around is a script for making the
changes to the shell initialization when I will carry the whole
package around (because I usually have to). So I save something like this
inside `~/.plenv/bin/colonize.sh and provide execution permissions to it:

Development Environment Set-Up

You might have a brand new development machine and follow the steps above
to set it up exactly like a deployment one: congratulations! Otherwise, you
should at least ensure that when testing in your dev machine you are
using the same perl version as you installed in the deployment machines,
with similar compilation options (e.g. with or without threads).

In the following, we will take the hard way and assume that you don’t have
plenv in the development machine. This is what I have today, so whatever
I write here wouldn’t be tested if I assumed that plenv is used in the
dev machine.

It’s now time to concentrate on the application. Assuming it lives in its
own directory, with proper version control set up (e.g. git), we have to
ensure that non-core modules are properly tracked, and carton will help
us out on this.

As you can see, you can either specify the module version - e.g. because
you know that the specific version has a particular feature - or not. At
this point, carton helps you install these modules in a local directory
called local:

carton install

The first time you run this it will create the local directory and put
all installed modules there. Another important file that is created
is cpanfile.snapshot, that records the cpanfile and the results
of the installation in a manner that will allow the exact re-creation
of the environment created here. At this point git will be complaining
about these new files:

you definitely want cpanfile and cpanfile.snapshot to be tracked by
git. The first helps you keep track of what you need, the second will
allow you to recreate the environment multiple times, and will help
anyone that wants to collaborate too. We’ll see how in a moment.

you definitely do not want to track local. At the point where you
have the instructions to recreate it with cpanfile.snapshot, it does
not make sense to move it around. And, as you probably already guessed,
it will probably be difficult to move compiled modules around, unless
your development environment matches the deployment one perfectly (or
so).

Running applications

Carton has its own way of helping you start applications with the
right environment:

carton exec -- program option option option...

It also plays well with plenv, so if you set up a local version of
perl with it it will use that. My impression is that it fiddles with
PERL5LIB to point towards the local directory that was created.

Another approach that does not require you to wrap your application
calls via carton - which might get annoying - is to set up usage
of the local library directly from your application. You probably
already do something like this if you store most of your code inside
the lib directory:

use FindBin '$Bin';
use lib "$Bin/../lib", "$Bin/../local/lib/perl5";

Workflow

These are my suggestions for the update of the distributions. In my
case, the deployment machines do not have git installed (it was on
the first one for installing plenv but it is not a requirement) so
I do my deployments with good ol’ tarballs (well, I use deployable
but it’s another story).

There is a higher level container directory that will hold the different
releases and also all local data, i.e. data that are specific to the
installation in the deployment server. In this example, they will be the
etc, local and vendor directories, but there might be more of course.

New packages are deposited and expanded inside the distro, where you can
find the different releases in case quick rollback is needed. Installing
a new release is as simple as:

expanding the new tarball inside distro

create the symbolic links to link back to etc, local and vendor
(these might be part of the tarball itself)

move the application symbolic link to point towards the new release
from a previous one.

What’s the vendor directory for then? Most of the times the deployment
machines are not connected to the Internet, and so you have to carry the
dependencies with you somehow. Carton allows you to create a
bundle of these modules like this:

carton bundle

and this will create a vendor sub-directory with all the needed stuff.
If you go this route, the suggestion is to put vendor/ too inside
.gitignore.

Deployment Strategy

Before the very first deployment, you will have to create the
directory layout described above:

For your application you create a distribution tarball, e.g.
application-2.0.tar.bz2, and transfer it into the deployment
machine inside application-container/distro. You can begin the
installation then:

Most the times you will not need to install new dependencies, but
sometimes (e.g. the very first time) you will. If this is
the case, you can generate the bundle of the dependency files
inside the development machine:

carton bundle
tar cvjf dependencies.tar.bz2 vendor

and then transfer this bundle in the deployment machine and install
it (we will assume that application-container/distro is going to
keep all our packages):