F.A.Q.

The following is an assembled list of frequently asked questions and answers.
Before you ask something on the mailing lists please first check here.
Perhaps your question was already answered many times.

Click on an individual question to display its corresponding answer
or for convinience reasons display
ALL or
NONE
of the answers at once.

About OpenPKG

When you install and maintain many Unix machines or when you are doing that job in a team of engineers then OpenPKG is useful for creating consistent configurations. If you just maintain one or two servers, OpenPKG is usually not worth the effort and you are advised to use vendor packages only.

Yes, "OpenPKG" (pronounced "open-pee-kay-chee") is short for "Open Source Software Packages". And the logo shows a package containing soft pieces of cotton wool. This represents a package of soft ware and so symbolically stands for the term "software package". The idea and its symbol were invented by Ralf S. Engelschall in April 2001. He also painted the OpenPKG logo and created the corresponding website design.

But to be honest, these tools do not satisfy the requirements of an OpenPKG like system.

Writing a new packaging tool from scratch was not an option, because it would have required too much time and it was not clear whether we would have been really more successful than others. Instead we picked the solution which provided for all(!) of our essential wishes a good or at least reasonable solution. The RedHat Package Manager (RPM) version 4 is not a perfect solution, but even with its drawbacks and pitfalls it fulfills the fundamental needs of OpenPKG. One of the most important need was that the tool has to support the whole(!) life cycle of a package.

Yes, you can. It might even be necessary, as our RPM tool provides a different set of functionality as the traditional one. Unfortunately, both use the same program name rpm. Pay attention to the program paths when choosing to operate on the OpenPKG repository or a traditional RPM populated one. In any case, do not panic. You cannot accidentally apply the wrong RPM tool, because OpenPKG RPM specifications do not work with other vendors' implementations. Such other implementations of RPM lack the virtual "OpenPKG" pre-requisite definition (which is provided by the OpenPKG bootstrap package only). Using other vendor RPM specifications with the OpenPKG RPM tool usually works, but stores installation data in the wrong RPM database. This is really the only problem to watch out for. It is good practice to put your vendor rpm tool first in the $PATH, and call the rpm of OpenPKG through an absolute path.

There are various reasons for this. With our own implementation of RPM, the OpenPKG filesystem hierarchy can achieve a high degree of independence and furthermore be self-contained. A second reason involves flexibility, and only with a custom made RPM can we adjust RPM to truly meet the needs of a OpenPKG system. Finally, with our own RPM we can take a consistent approach to installations over all supported platforms. Using the vendor RPM on Linux, and another on Solaris and FreeBSD was not acceptable for us.

This is not really an issue for OpenPKG, because OpenPKG is intended for sys-admins who have to manage a large set of diverse Unix platforms. So, they are already forced to use multiple tools. OpenPKG's RPM might be one more such tool, but OTOH it now allows him to redirect most of his daily tasks to only this tool. Additionally, for the daily standard tasks of a sys-admin, RPM is very easy to use.

No, because our source RPM packages depend on RPM extensions which are not available with the plain RPM v4. For example, a plain RPM v4 does not supply the rpmtool and shtool programs. Additionally it will fail during any reference to our %{l_xxx} variables.

This is a sensible question and not easy to answer. In contrast to lots of RPM-based (Linux) distributions, OpenPKG did not take over existing source RPMs from other vendors (like RedHat or SuSE, etc) at all. OpenPKG is a from-scratch clean-room implementation of a packaging system. And except for the fact that OpenPKG uses the RPM tool, it does not share anything with other RPM-based systems.

This approach certainly caused us lots of trouble in the beginning and required lots of extra efforts, but it was important to follow in order to achieve the requirements we had on OpenPKG. Mainly this was that the packaging has to be clean (style), minimal (redundancy), portable (dependencies) and flexible (hard-coded things). So we decided to start entirely from scratch (except for the RPM tool) and not to be confused or influenced by existing RPM specifications. And experience showed that this was a good decision, although it is not shared by lots of people (especially those who dislike re-invention of wheels and want to quickly have a solution).

So, we really advice you to not take over entire RPM specifications. Nevertheless (as long as the license on this specification permits it), you can try to help yourself by taking over ideas and the roadmap for packaging a particular product.

But if you still insist on reusing a source RPM of another vendor with OpenPKG, here are the steps:

OpenPKG packages have to build-able as non-root for security and OS-trashing reasons. So you have to force the underlying vendor procedures to build and install in this situation. This mainly means that the install procedure is not allowed to chown or chgrp any files. This has to be first skipped and later emulated with %attr in %files.

OpenPKG packages have to be independent of the filesystem root directory of the OpenPKG instance. This means that all packages have to be configured, build and rolled for %{l_prefix}. This mainly means that you are not allowed to use any hard-coded root paths.

OpenPKG packages have to comply to the OpenPKG filesystem layout standard ("name" is the package name, given are directories only): %{l_prefix}/bin/ for user binaries; %{l_prefix}/sbin/ for system administrator binaries; %{l_prefix}/etc/name/ for the configuration files; %{l_prefix}/share/name/ for the constant (read-only) data files; %{l_prefix}/var/name/ for the variable (read/write) data files; %{l_prefix}/libexec/name/ for the internal binaries; %{l_prefix}/man/manN/ for the manual pages; %{l_prefix}/lib/ for the C libraries; %{l_prefix}/include/ for the C headers.

OpenPKG packages have to be build inside $RPM_BUILD_ROOT for correct file determination and OS-trashing reasons. So you have to force the underlying vendor install procedure to install into $RPM_BUILD_ROOT%{l_prefix} instead of %{l_prefix}, although it still was configured and built for %{l_prefix}. This mainly means you have to use the DESTDIR and/or prefix variables on make install. If all things fail, you have to patch the Makefile between make and make install or even have to install manually.

OpenPKG packages have to roll the binary package via %files -f files, i.e., through a $RPM_BUILD_ROOT-generated files list. This is accomplished by running rpmtool files at the end of %install. But it especially requires the previous point.

Writing conforming packages from scratch usually takes much less effort and time than the alternative approach of adjusting a not conforming one.

As, you can see, every RPM package implicitly provides a physical target which directly corresponds to its particular name, version and release. Additionally, a package can provide zero or more so-called virtual targets. There are two strictly distinct instances of a virtual target:

The virtual target "FOO" (without any version and release and with the name in uppercase letters only) is used by alternative packages. Those packages usually do not also have the name "foo".

This virtual target is for automatically handled package alternatives. All packages providing this target have to conflict by default, because they are true alternatives.

The intention is that those packages are fully equal and compatible from the semantical point of view of the virtual target and any can be chosen and used. All are supported and exists for regular usage.

Having multiple those packages providing this virtual target is a permanent solution and will remain in the long-term. All those packages are usually based on different vendor products.

The virtual target "foo = VER-REL" (with version and release and with the name in lowercase letters only) is used by replacement packages. Those packages are all required to have the name "fooX" where "X" is the compressed major version string not longer than 2 or 3 digits.

This virtual target is for manually enforced package replacement. All packages providing this target do not have to conflict by default, because they are package variants which sometimes can coexist. But the "fooX" packages often can be enforced (by convention through the build option "with_foo yes") to fake the "foo" package in order to replace it.

The intention is that those packages are fully equal and compatible from the semantical point of view of the virtual target, but although any can be chosen, only one should be used (foo). Only one is supported ("foo") and exists for regular usage, while the others ("fooX") exists for temporary backward compatibility, upgrade preparation or bleeding edge testing reasons only.

Having multiple those packages providing this virtual target is a temporary solution and will certainly change in near short-term. All those packages are usually based on the same vendor product.

OpenPKG by design focuses on source RPMs and the building and installing directly from them. Binary RPMs are just an intermediate and temporary result in this approach. From our perspective, they exist just temporarily on the target machine or on our FTP server because of bootstrapping and for emergency situations only. There are multiple reasons for this. The most important are:

Stability: binary RPMs are inherently weak when it comes to run-time stability. The reason is that there are always differences between the build and install host — sometimes more, sometimes less. But just the smallest difference (versions of vendor shared libraries, different kernel patch-levels, system configuration differences, etc) can lead to a broken application on the install host due to inherited assumptions from the build host.

Example: build host has higher configured maximum allowed size for shared memory segments (usually because runs also a RDBMS), building OSSP mm on it determines this high limit and has to hard-code this into the package, package is deployed to install host and breaks horrible because install host has default maximum allowed size for shared memory. Bang! And OSSP mm has no chance, some parameters of a system cannot be easily determined under run-time. Instead they have to be determined in a complex way under configuration/build-time.

Flexibility: OpenPKG source RPMs often have a bunch of build "options" (can be queried with rpm -qpi) for allowing one to build a package in multiple different variants. For instance, our Apache package has 55 boolean options. This allows you (theoretically) to build 2^55 = 36,028.797,018.963,968 different binary RPMs out of a single source RPM. There is always one combination which fits your situation well. A binary package has no more possibilities, it just was built for a fixed combination of options. And we doubt that neither a simple "no options enabled" nor an "all options enabled" Apache binary module would be sufficient for our community.

Security: even with package signing by the OpenPKG project, from a security paranoia point of few, one never can really just trust a package — neither a source nor binary one. For real security, every piece of software has to be audited, or at least be auditable. Source packages make this possible, binary packages make this completely impossible except you are trying to perfect the art of disassembling object code and reviewing complex algorithms on assembly layer.

As the previous answer explained, OpenPKG by design focuses on source RPMs and the building and installing directly from them. Binary RPMs are just an intermediate and temporary result in this approach. Binary RPMs make too much trouble and so we try to reduce their issues and not to add extra organizational complexity by the introduction of architecture independent packages. Hence we do not distinguish between architecture independent and dependent binary packages.

OpenPKG technically is already a rather large Open Source project and with still only very limited manpower resources. For providing some services like security updates for older releases, reducing or increasing release cycles, making release engineering decisions, etc., it is absolutely essential to first know who are our users, how much users we have and what OpenPKG releases they are actually using. Without first knowing this information we can neither afford continuing all of the current services nor even providing additional services.

We're paranoid, too. The whole website authentication is handled by OSSP ase (Affiliation Service Environment), running under HTTPS only. This way your clear text password is transferred encrypted to the registry.openpkg.org website. Then it is immediately hashed with the one-way functions MD5, SHA1, crypt/MD5 and crypt/DES and stored into the underlying database without any further logging or manipulation. You can review this Open Source implementation under http://cvs.ossp.org/ossp-pkg/ase/. Nevertheless, for security reasons please do not reuse any existing password.

We do not really restrict the download of our software. Everyone still can download all of our resources and fully free of charge. He just has to identify. What we restricted is the anonymous download of some more expensive parts of our software as it really hurts the project. FOSS is mainly about freedom in the meaning of non-discrimination of users and the possibility to review the source code. For OpenPKG and its extremely liberal BSD-style distribution license it is even a lot more, too. But FOSS is not about anonymity.

Yes, but the login intentionally is for identification only, not authentication. If someone uses a foreign identity he just hurts himself, because he is not known and this way not taken into account by the OpenPKG project.

Yes, if somebody prefers full anonymity he can download from an (unfairly but fully legally) established mirror of the restricted download files. We fully believe in the seriousness, fairness and support of our user community and hence not actively using any existing public mirrors when downloading restricted files. Instead every serious OpenPKG user is called to actively support the OpenPKG project by identifying and downloading from the official distribution source location or using a private mirror only.

At the cost of increasing the complexity we made the instance registration really ultra-transparent if wished. See especially the $prefix/etc/openpkg/registry.prep file for details what is posted online. It contains no really sensitive personal information.

MISC

To begin with, OpenPKG works on a number of other Unix variants. It can further be patched with small changes to work on an even larger set of Unix variants. However, we promise full support only to Solaris, Linux and FreeBSD users because they are the most popular server platforms. OpenPKG will run just as well on many other operating systems, however. These additional platforms are not fully supported only because the resources needed to ensure correct package builds at OpenPKG central are not sufficient. To find out more about how well OpenPKG will likely run on your platform, please refer to OpenPKG handbook

There are many reasons why even packages which are shipped with NLS support are not packaged with NLS support. First, NLS is great if the whole system, i.e., all packages, provide NLS support. But this is not the case for OpenPKG, because unfortunately still not all vendors support NLS in their programs. So OpenPKG cannot provide NLS for them as a whole, and a half solution was not acceptable to us.

Second, we dislike the bloat gettext-based NLS support causes on the filesystem for each package, because in OpenPKG we try to strip down a package to its bare minimum. Third, although the founders of OpenPKG are non-native English speakers, they like the idea that English was and should be kept as the only language used by Unix systems. This further ensures against strange translated messages which often serve to confuse rather than aid an engineer.

In a perfect world all vendor packages would ship with the same amount of NLS support (number of supported languages). All translations would be done correctly and consistently. Filesystems would not bloat from hundreds of extra files for each program just because it is localized to hundreds of languages. We as OpenPKG enthusiasts are patiently waiting for this dream world to appear, and will then provide OpenPKG with fully NLS support, of course. But until this happens, we think it is better to avoid NLS entirely. We extend our sympathy to those who prefer a non-English language even on computers.

Maybe you are thinking of the OpenPKG run-command (RC) system or the fact that tools and configuration files sometimes are located at unusual filesystem paths.

The implemented run-command facility is partly modeled after the classical UNIX™ System V run-command facility plus ideas taken from the FreeBSD and NetBSD run-command facilities. It mainly merges the classical startup/shutdown procedures with periodical procedures into a single approach. For details read "openpkg man rc". The common task of log file rotation is explained in "openpkg man shtool-rotate" with the man page being part of the shtool package. In general OpenPKG follows the philosophy "keep it simple, stupid" (KISS) and "principle of least astonishment" (POLA). If something can be done more orthogonally then OpenPKG prefers the consistent and clean approach in favor following administrative habits. Some de-facto standards had to be broken as result of creating the unique multi-instance feature of OpenPKG.

You need to understand some basics about RPM usage, the OpenPKG filesystem layout, the shell environment and the run-command facility. The User Tutorial and the Quick Reference are good starting points to gain that knowledge.

A clarification: in OpenPKG executables are dynamically linked against operating system (external) libraries, of course. What you are talking about is the linking against the OpenPKG provided (internal) libraries. These are currently build as static libraries (.a) instead of shared libraries (.so) for various reasons.

We use static linking for them mainly to avoid cross-platform trouble. Because with shared libraries you have to fiddle around with LD_LIBRARY_PATH (and/or ldconfig if existing) and especially can run into trouble for libraries which the OS vendor also provides (examples are libdb, libz, etc). In using only static linking inside OpenPKG we are a little bit less flexible and our object code grows in size, but OTOH we already avoid lots of trouble in advance.

Nevertheless we certainly will try to change this after some settlement of OpenPKG. At least it is on our wishlist for forthcoming OpenPKG releases. So it certainly can be changed, but we have to evaluate first and make sure we do not open a can of worms related to the cross-platform aspect.

The OpenPKG team uses a fully automated process for tracking vendor source versions which reports to the team twice per day the list of packages which need to be updated. Additionally, there is always the job of a Package Master On Duty (PMOD) assigned to a person which trys to immediately perform this update on a daily basis. This way OpenPKG-CURRENT packages are kept always up-to-date for our community as fast as possible and this way we are able to get feedback as early as possible.

The primary development platform, where the Package Master On Duty (PMOD) performs his daily tasks, is FreeBSD. Because of time constraints the PMOD after an OpenPKG-CURRENT package upgrade only tests it also on Linux and Solaris if time permits. If time does not permit it, the brokenness is discovered later only. But it is at last discovered before an official OpenPKG release because there are all involved packages tested in depth, of course.

If you compare just the number of packages, this is correct. But you are comparing apples with pears here, because:

FreeBSD and Debian usually package everything, although 90% of the packaged software are just neat toys and far away from killer applications. OpenPKG is the other way round: 90% are formed by essential packages only and just the remainder are toys.

FreeBSD and Debian provide packaged software for all types of deployment, ranging from stripped-down embedded devices, over networking servers, up to colorful desktops with all bells and whistles. OpenPKG mainly focuses on deployment on network servers and up to now has just a few desktop-related packages. If you really want to deploy software in non-server situations you should not focus on OpenPKG, please. Then please stick with the packages of other vendors which focus on your situation.

FreeBSD and Debian usually package all variants, versions and alternatives of a piece of software. For instance, they provide dozens of possible shells while OpenPKG mainly just provides the most popular ones (bash, ksh, tcsh, zsh). Additionally, they package very often multiple vendor versions (stable, development, snapshot, etc) while OpenPKG most of the time provides only a single version.

All those points together result in the dramatically different numbers of packages. But it is wrong if you think the lower number of packages would mean OpenPKG is incomplete. OpenPKG actually provides far more packages for software than you usually need to deploy on a server platform.

The bootstrap procedure has only one purpose, and that is to install a new OpenPKG instance. Remember that the procedure accomplishes this in two stages. The first stage can be run as any user and mostly builds the tools that OpenPKG needs (tar, shtool, bunzip…). The second part needs to be run as root however, and will alter the underlying system. Admins running intrusion detection should take note. The five entry points are:

Users and groups are added to /etc/passwd and /etc/group, or whichever default mechanism the corresponding platform supports.

Crontab entries are made (typically to /etc/crontab) to allow subsequent OpenPKG packages to operate periodically.

An init script (typically at /etc/init.d/<prefix>) is added to start all active daemons and other OpenPKG packages at system startup. Remember that packages can always be 'deactivated' and started or stopped manually as well.

OpenPKG 2.0 and later log their prefix into file /etc/openpkg. This allows inventory mechanisms to find all instances installed on a machine.

Naturally, OpenPKG will take space in the file system corresponding to the prefix given during the first bootstrap stage.

Curious admins can learn more about these entry points by comparing the system before and after bootstrapping a new OpenPKG instance. In each case, the bootstrap procedure uses the information given (--prefix, --user, --group, [--rusr]…), so finding any system alterations should be easy.

To deinstall OpenPKG, simply remove the package called 'openpkg' in the same way that any other package is removed (/prefix/bin/rpm -e openpkg). If any dependent package exists, OpenPKG will require that it first be removed.

Once the deinstallation finishes, the system will return to its initial state. Any exceptions to this rule are due to files manually installed or others inhibiting the removal of the complete file hierarchy. This is desirable due to the preservation of log files, for example.

To verify this, an admin can compare the system (by copying files associated to the entry points above) before and after bootstrapping OpenPKG to confirm this (de)installation consistency.

You have to get the bootstrap package "openpkg" building and running on the new platform. This basically means that that the contained software packages (Bash, cURL, Tar, RPM, etc) have to build and that the linking into the system (users/groups and cron/init scripts) is known by the bootstrap. For getting the software building, you perhaps have to add one or more patchfiles to the "openpkg" package or at least use some "shtool" substitutions in "openpkg.spec". For the linking into the system, add the corresponding commands to the "%pre" and "%preun" sections of "openpkg.spec".

You have to port your wished OpenPKG packages to the new platform. Because all OpenPKG packages are inherently portable (because do not contain any platform specific things), porting them usually always means to get the underlying vendor software package building. OpenPKG most of the time (only a few exceptions exists) package only already portable vendor software. So as long as your new platform provides a fair amount of the POSIX APIs, mostly all OpenPKG packages will work out of the box. For the remaining packages, you have to add a patch file to the package which fixes your particular platform problems.

The default values are derived from the options --user=<user> and --group=<group> on the command line of openpkg-*.src.sh. For instance, the "Example" values above are achieved with --user=opkg --group=opkg. In case of a non-privileged OpenPKG instance, the {mrn}{usr,grp} are usually identical.

For security reasons it is important to treat at least the "managing user/group" equal to the "super user/group", similar to what has to be done with the usual Unix "root" and "bin" user/group ids. The reason mainly is that the "super user/group" executes files intentionally owned by the "managing user/group".

Similarly the "restricted user/group" and "nobody user/group" have to be treated like the usual Unix user/group id "nobody" with the addition that the OpenPKG "restricted user/group" has little bit more privileges than the "nobody user/group" because (mostly generated) files are also owned by him.

You can override the RPM default %l_cc permanently in your ~/.rpmmacros. This is especially useful when bootstrapping platforms where OpenPKG does not initially find a C compiler in the path. The most prominent example is Solaris. Example:

%l_cc /usr/local/bin/gcc

Alternatively, you can override %l_cc for a single rebuild by defining "use_cc".

The build-time of a module with XS parts is the run-time of Perl. Those modules require exactly the C compiler at their build-time which was previously used at the built-time of "perl". To control C compiler usage in a deterministic way both at "perl" build-time and module build-time aka Perl run-time the "perl" package has "gcc" listed as both a BuildPreReq and a PreReq.

Find updated packages for releases in the UPD directory below the release on the FTP Server. Find a brief description for each update in the 00README file beneath the package. Example: release/1.3/UPD/00README file. Find in-depth developer level information by browsing the CVS timeline of the package. Example: OpenSSH.

While we cannot guarantee this won't it is very unlikely that an update will break an existing setup. In fact, one reason why updates are created is to fix existing problems in the least intrusive and most compatible way. Updates will always include the same vendor version. Sometimes we even preserve bugs intentionally. Most updates are driven by security incidents which come with an advisory describing the problem, scope and required actions to prevent the problem. Please examine our security pages. In any case we work hard to make upgrades a no-brainer. Packages are designed to be drop-in replacements for it predecessor(s).