Python on Linux is downright awkward. If you want to take
advantage of the latest tools and tricks that the scientific Python
community has to offer, that means installing Python packages that
aren't always available in your distribution's package manager. People
recommend using pip, but if you just apt-get install pip and sudo
pip install this that, now you've gummed up apt-get because it
doesn't know about the packages that pip installed.

What if you could keep your own independent version of Python in
your home folder? This way, you wouldn't have to worry about
breaking your system with experimental packages -- you could just
remove the folder to go back. This also lets you separate out each
of your projects' dependencies.

You don't have to be a rocket scientist or a UNIX-bearded guru to set
this up.

Overview

Here are the steps we'll take. Don't skip any because each one
depends on the last. You can skip installing OpenCV if you don't need
it though.

Once you're through, you'll have a nice way of keeping packages in
your home folder, a custom version of python, a fresh-compiled OpenCV,
and everything you need to run ipython notebook.

Set up your shell for local packages in your home folder so you
won't have to worry about your distribution's package manager.

Compile Python from source, placing it in your home folder

Set up virtualenv, which also gives you pip

Install OpenCV

Install your Scientific Python stack

Install the IPython notebook

Do not use sudo for any of the instructions below. You don't need
it and it will mess your home folder's permissions up.

Local packages

If you only read one section from this article, make it this one.

This step frees you from ever having to worry about apt-get again.
Once you have it set up, you don't have to ask root to install your
favorite packages anymore; you can just compile them yourself and keep
them tucked away in your home folder.

This is wonderful for shared systems, old systems, or when you want to
try inspecting/modifying a package's source code. This is the secret
sauce that makes local Python work. It's completely
distribution-agnostic, it doesn't make invasive changes, and it's easy
to go back if you decide you don't want it anymore.

You might already know about $PATH, the environment variable that
lists all the places where your programs live. When you run a command
like ls, the kernel will search your current PATH to find that
command. My $PATH currently has:

One great pattern is to keep all of your local programs in a
~/local folder. This way, you can have your own compiled Ruby in
~/local/ruby, a NodeJS in ~/local/node/, and so on. When you type
irb, your shell really runs ~/local/ruby/bin/irb.

Then, your $PATH would have /home/gcr/local/ruby/bin,
/home/gcr/local/python/bin, /home/gcr/local/node/bin/,
/usr/local/bin, /usr/bin, /bin, /sbin, /usr/sbin, and
/usr/local/sbin. Each of your packages would have its own entry in
the PATH.

It's really elegant. This pattern helps with debugging and version
isolation and you can just rm -r ~/local/ruby to cleanly and
completely remove the local ruby with no system-breaking side effects.
Want to switch to another version of node? Just move your node
folder out of the way and compile a different version into its place.

Let's build Python this way. First, create the ~/local folder
that will keep all the packages you install.

You now have to apply your changes by running source ~/.bashrc
(You should source your bashrc every time you install a package this
way.)

If something breaks or you want to undo all of your work today, just
delete ~/local, remove that part of your ~/.bashrc, and log out.
When you log back in, everything will be the way it was before you
meddled with this article.

Installing python

Now, let's compile our own Python from source.
Head up to http://python.org/download/
and download the latest source code release into /tmp, but make
sure it's Python 2.x and not Python 3.

Make sure you have all the dependencies: build-essential,
autoconf, automake. Don't forget zlib1g-dev (the thing that provides
zlib.h) or else Python's zlib package won't work. You also need
libssl-dev to download HTTPS packages.

Note that the ~/local package trick works for shared libraries
too, which means that by keeping OpenCV there, you can just compile
any C/C++ file against any arbitrary version of OpenCV that you
like, no matter how messed up the rest of your system's packages are.

This is what we call super convenient.

Head on over to http://opencv.org/downloads.html and grab yourself a
copy of OpenCV's source, again placing it into /tmp:

OpenCV uses the CMake build system, so let's just compile it.
Unfortunately, OpenCV requires literally tons of command line options
to make it work inside your virtual environment, so read the code
below before copying it into your terminal. In particular, I'm
assuming that you have Python 2.7 and that your virtualenv is in the
$VENV_FOLDER variable:

Make sure that all of these paths are inside your home folder! If
any of them begin with /usr, OpenCV won't work.

(Protip: At this point, if you want to build OpenCV with, say,
ffmpeg support, now's the time to compile your own ffmpeg.
Remember: If you compile FFMPEG with ./configure
--prefix=$HOME/local/ffmpeg, it will go straight in your home folder)

Anyways, it's time to compile OpenCV:

gcr@brannigan:/tmp/opencv-2.4.6/build $ make install
...-- Installing: /home/gcr/local/x86_64/opencv/bin/opencv_performance-- Set runtime path of "/home/gcr/local/x86_64/opencv/bin/opencv_performance" to "/home/gcr/local/x86_64/opencv/lib"-- Installing: /home/gcr/local/x86_64/opencv/bin/opencv_traincascade-- Set runtime path of "/home/gcr/local/x86_64/opencv/bin/opencv_traincascade" to "/home/gcr/local/x86_64/opencv/lib"

Horray! That took a while. Again, re-source your bashrc so you now
see your lovely library. Let's test OpenCV:

Whether you keep libraries in your home folder or not, pkg-config
knows how to compile them. Use it in your makefiles, use it in your
shell scripts, use it everywhere. You can see a list of packages that
pkg-config knows about by running pkg-config --list-all.

Note that you might need the libblas-dev and liblapack-dev
packages from your distribution's package manager, or you can compile
them yourself just like how we compiled OpenCV. They're like twenty
years old though, so you probably don't have to worry about your
distribution carrying out-of-date versions. (At the time of writing,
BLAS' first release was 34 years ago.)

After this, you can now just run ipython notebook and your browser
will pop up, ready for editing!

This blog is proudly powered by Pelican, which takes great advantage of Python.