1- Introduction

Why would you want to use RPM on a non-Linux system ? It may seem strange as all
these environments already provide a native package format.

Let me just explain how
we got such an exotic idea.

I am a Unix system engineer. The platform I am
working on contains some HP-UX, Solaris, AIX, and Linux servers. Historically, on
this platform, the developers delivered their software in the form of a
gzipped tar file. But this format is very poor as it just allows to gather a
set of files into a single file, and the need of a smarter package format quickly
arose.

The question was then to choose the package format(s) we would use.
Each OS provides its native package format, which are incompatible. So,
using the native package format for each OS would have forced the development and system
teams to learn how to build and use all these formats, which is quite impractical.
Having eliminated this option, we searched for a format to be used everywhere. Only two of
the native package formats exist outside of their native OS : SD-UX (HP-UX) and
RPM (RPM is not really available in precompiled form on every non-Linux systems but, as it is open-source,
it can be compiled). SD-UX was discarded for several
reasons, the main ones being that it is closed-source, expensive, and its future
is not clear.

RPM remained alone with
several benefits :

It is easy to teach to the development and system administration teams,

It is widely used and known. Most people we hire already know it,

Combined with yum, it makes installation procedures very easy.

We didn't consider other Linux package formats, like apt, because the
only Linux distribution we use is RedHat's RHEL.

Once the decision is made, we 'just' had to build RPM/YUM packages for all
the environments. Although RPM is quite portable since version 5 (version 4 was
very Linux-specific), this is not the case for YUM, whose port took a lot of
time. This is due to several factors like poor documentation, complex
dependencies, the need for a dedicated python interpreter... Actually, I
wouldn't be surprised if it was the first time somebody builds it on a non-Linux
system.

Well, the work is over now. Just follow the rest of this document and
you too will be able to provide unified package installation/management
procedures on all your Linux and non-Linux environments.

It will bring you the admiration of your {boss|colleagues|neighbours|wife|children|dog},
and, who knows, an appreciable boost in your
career .

1.1- Intended audience

This document can be used in two different ways :

In order to apply the procedure and build the software exactly as it is
described here, you don't need to be an expert : it just requires some basic
Unix and shell knowledge, such as being able to download the source
packages, uncompress/untar them, and create files with a text editor.
At this level, the procedure given below can be splitted in two main steps :

you first set a limited number of environment variables (essentially
the installation directory)

and the rest is just a suite of copy/paste operations from the
document to your shell environment.

Now, if you want to understand everything that is explained below, or if
you want to adapt it to your specific needs, it requires of course more
knowledge on different domains, mostly :

and even, when it is really needed, the ability to understand and
adapt some C source code.

So, to resume, most people will just apply the procedure described below, which is
relatively easy and just requires a basic working knowledge of the Unix shell
environment.

Don't hesitate to post comments on issues you may have with this
document. Each time you do it, you help me and the community in general to make
the procedure more robust and reliable. I promise to reply to every question
asked in comments. Please prefer comments to direct mail as they may benefit
others.

1.2- Getting the source packages

Some readers sometimes ask why I don't provide copies of the source
packages I list below. Sure, it would be easier for you, but I won't do it as you
MUST get your source packages from a repository you can
trust ! You don't know me, you CANNOT trust source code downloaded from my
site (except
for my own projects).

So, once again, always get your
software from a trusted source and, when a package signature is provided, check it to make sure you get the official
code.

I insist on this because compiling some modified source code or installing
some corrupted precompiled binaries would give hackers everything they need to take
control on your site (remember that most software we compile here runs as root).

In every chapter below, you will find a link to the best place to get the
source package from.

So, remain a good, slightly paranoid, system administrator, and everything
will go well.

1.3-
Software dependencies

The following diagram displays the libraries RPM depends upon and their
relationships.

yum has a very complex dependency diagram, that you'll find in the table below.
If someone builds a readable graphical representation from this, I'll be glad to
include it :

C compilers can have such different behaviors that I strongly recommend
using gcc. If you are using another compiler, please don't ask for support.

Now, we set the 'make' commands we'll use in the rest of the document. The
difference between $MK
and $SMK is that, in $MK, you can add options for a 'parallel make' ('-j', '-l',
...). $SMK stands for 'serial make' and is used where parallel builds are known
to fail.

Mileage varies concerning parallel make. Depending on your host
characteristics and the options you set, the build can be faster or slower.
Personnally, after trying several options, I did not observe a noticeable gain
in performance and reverted to the default 'serial' make.

Please note that parallel make can
cause compilation failures. So, if you get unexpected results
while parallel make options are active, the first thing to do is to retry the same
without the parallel options. For more info about parallel make, see the
GNU make manual.

GNU make is mandatory as several packages absolutely require it. On
Linux, GNU make is native but, on non-Linux systems, you are strongly advised
not to use the native make.

If the -brtl option is not set, the linker (ld) ignores the
files with a '.so' suffix (and searches for '.a' files only).

export LDFLAGS="-L$BLIB/common/lib -Wl,-brtl"

On AIX, using /bin/sh to
execute the configure scripts can be very slow, due to the huge number of
useless temporary
files that get created during the execution. To work around this problem, we
will use bash :

export
CONFIG_SHELL=`which bash`

2.2- Creating the target structure

Every libraries and auxiliary software are installed under
'$BASE/libs'. There, each software/library is assigned a subdirectory, and we populate a 'common'
subdirectory with symbolic links to the actual file locations. Using a common directory
populated with symbolic links brings several benefits :

it allows to set a single path in the dynamic lib search
path (LDFLAGS).

it also allows to set a single path in the include file search path
(CPPFLAGS)

Putting a common 'bin' subdirectory as first element of the PATH ensures
that anything we put in this directory will be found first when searching
for an executable command.

Last, it allows to set a single path in the PKG_CONFIG_PATH variable.

Thanks to this directory structure, we don't have to modify the environment
variables each time we add a library to the package. It becomes also easier to
ensure that the compilation will reference our embedded library/include files,
and not the ones that can be installed elsewhere in the system.

First, we create the base common directory.

mkdir -p $BLIB/common

Then, we create the script to
populate the common directories. Every
time we build and install a library we need to reference in a subsequent build, we
call this
script to register the files in the common structure. This is done through the
$LINKDIR environment variable.

Note that the
link creation script is not in the common PATH. This way, it cannot conflict
with another file.

#-- Declare the variable we use to
call the script

export LINKDIR=$BLIB/common/linkdir

#-- Create the script

echo 'cd $BLIB/$1

#-- If second argument is unset, default to the last element of target dir (usual
case)

cdir=$2
[ -z "$2" ] && cdir=`basename "$1"`

#-- Directory to contain links. Create common subdirectory when needed

2.3- Embedding a private libgcc_s library

Some files we generate have a runtime dependency on the libgcc_s shared library.
It means that this library must be present on the target host. In accordance
with our 'black box' approach, we embed a libgcc_s library in the package and we
force everyone to reference this copy.

2.4- Compiling the stdc++ library (4.1.1)

Here, you
need to download gcc-core and gcc-g++. Uncompress/untar them in the
same directory. Note that you will need
GNU tar to read
the gcc-core source package.

If you get errors in this part, and if
you use a pre-built gcc compiler, you should first try to rebuild your gcc
compiler from source code. It is not guaranteed to solve your problems but it
generally did for me.

On AIX, we must compile in two steps, with different values for the
OBJECT_MODE variable (the first step ends with an error).

We also run configure through bash, because the default /bin/sh is
excessively slow
here.

2.7- Compiling the OpenSSL library (0.9.8j)

We must add a '-fPIC' option to the compile options, so that the compiler
produces some 'position independant code', which is mandatory when the code will
be included in a shared library. We modify the Configure script :

We force the Makefiles to link through gcc, instead of calling
ld directly, because we want our LDFLAGS to be used and it may
contain options which are not supported by ld (as the options starting with
-Wl).

2.13- Compiling gettext (0.17)

The '-with-included-gettext' configure option forces to build and use the
included libintl library, even if one is present in the system libraries. This
removes a dependency to the system libraries and ensures that we use the same
libintl code across all platforms.

Note that, even if we try to disable NLS localization everywhere it is
possible, some software don't allow it to be disabled. That's why we still need
to embed a libintl library.

2.14- Compiling the pcre library (7.8)

AIX system include files don't define the 'BOOL' type, which is used by
the pcre code. So, we need to define it in an header file included by every C
source file. We also enclose it in a '#ifdef/#endif' block because some files
include it several times (and a typedef cannot be redefined) :

As I work with several operating systems and several versions, I prefer
the rpm files I build to be named with
an explicit OS
and version. So, I modify the
_build_name_fmt definition.

By default, rpm is configured to display the architecture field
when replying to an 'rpm -q' query. I prefer the same behavior as on Linux, where only
the names, versions, and releases are displayed.

I don't want requires/provides entries to be auto-generated by
rpmbuild. The requires and provides elements I want to be
associated with my packages are the ones I explicitely mention in the spec
files and that's enough.

Of course, these basic tests are not exhaustive, but they at least ensure that
the basic mechanisms are OK : the RPM database is accessible, the sysdeps
and rpm virtual packages are correctly registered, the provides/requires engine is
functional, and yum correctly communicates with the RPM layer. It is quite basic
but, speaking for my own case, everytime I had problems with an RPM build, even
these basic tests did not work. So, if you get there, you should be quite
confident in your RPM/YUM software.

4- Packaging

4.1- Tar/gzip

A tar package just contains every files under $BASE.

4.1.1- Creating the package

The commands to build a tar/gz package are :

cd $BASE
tar cf - . | gzip --best >/tmp/rpm-2.1-1.ppc.tgz

This creates a '.tgz' package file in /tmp.

4.1.2- Installing the package

To install the package :

Download the package file on the target host

Create the $BASE directory

Ensure the file system containing the $BASE directory has enough free
space