Search

February 14, 2008

Shipwright, our new code packaging tool

Like any opensource software shop, we distribute the source codefor our software. How's that for the obvious statement of the decade?

Actually, I can beat it. It's a pain in the neck for end users tocollect and install all of the dependencies for our software.

And now I'm going to one-up myself again. Customers often buildour software against untested versions of libraries, making debugging'frustrating.'

RT, our flagship product, depends on 124 separate packages, 114 ofthem CPAN libraries. While CPAN has pretty good support for recursivelyinstalling dependencies, it's not perfect and can be time consumingand confusing for end users. And when it doesn't work right, as canhappen when a module author makes an incompatible change, debuggingrequires a wizard.

We've built a new source (and binary) packaging system calledShipwright. Shipwright allows you to track all of your package'sdependencies in a version control repository like SVN or SVK aswell as build order and build instructions.

It comes with tools for importing Perl modules, C libraries andother dependencies from CPAN, upstream version control repositoriesand tarballs. When it can discover dependency information (as itcan for Perl modules), Shipwright will automatically import thecurrent versions of all listed dependencies if the repository doesn'talready contain sufficient versions.

Shipwright can automatically set up build instructions for projectsusing autoconf as well as projects using Perl's MakeMaker,Module::Install and Module::Build mechanisms. If necessary, youcan customize the build instructions and dependency ordering afteryou import a package.

When it's time to ship your project to your end users, all you needto do is take a snapshot of your Shipwright repository and send itout. To build your project, an end user just needs to run"./bin/shipwright-build". If they want to, your users can chooseto skip certain dependencies (if they want to use system versions)or specify an installation path. By default, Shipwright buildsfully relocatable binary distributions into a temporary directoryand users can move them into place or copy them to any number ofhosts with the same base system libraries -- Shipwright automaticallywraps all your binaries and scripts so that they can find theShipwright versions of their dependencies, no matter where you movethe installed distribution. Shipwright also comes with sh and tcshscripts you can 'source' to add an installed distribution's librariesto your current environment.

At Best Practical, we've configured most of our Shipwright distributionsto bundle everything above libc. Perl, Subversion and GD are justsome of the packages we distribute as part of these relocatablebuilds. We now have a single-command tool to build, link and installSubversion, SVK and all their dependencies (including APR, Neon,Perl and a bunch of others). With a single command, we downloaded,extracted, checked in and tested Tatsuhiko Miyagawa's Plagger FeedAggregator and all 134 perl modules it depends on. After that, asingle command built a full binary distribution of Plagger, readyfor deployment on any Mac OS X system.

I'm quite proud to release Shipwright 1.0 today. I designed Shipwrightwith sunnavy, one of the hackers here at Best Practical. He'sresponsible for almost all of the project's implementation to date,though we're eager to have additional developers join us going forward.

If you're interested in Shipwright, download 1.0 fromhttp://search.cpan.org/dist/Shipwright and subscribe to the Shipwrightmailing list by emailing shipwright-subscribe@lists.bestpractical.com

You can always get the latest version of the Shipwright source codewith this command:

svn co svn://svn.bestpractical.com/Shipwright/trunk

You can get a glimpse of Shipwright builds of RT, SVK and Jifty at http://code.bestpractical.com/shipwright/