Hierarchical Packages

1.0 Introduction

The Common Lisp package system, designed and standardized some years ago, is not
hierarchical. Since Common Lisp was standardized, other languages, including Java and
Perl, have evolved namespaces which are hierarchical. This document describes a
hierarchical package naming scheme for Common Lisp. We hope that CL implementations
other than just Allegro CL will include this facility. All the source code used in Allegro
CL to implement the facility is included below.

The goals of hierarchical packages in Common Lisp are:

Reduce collisions with user-defined packages: it is a well-known problem that package
names used by the Lisp implementation and those defined by users can easily
conflict. The intent of hierarchical packages is to reduce such conflicts to a
minimum.

Improve modularity: the current organization of packages in various implementations has
grown over the years and appears somewhat random. Organizing future packages into a
hierarchy will help make the intention of the implementation more clear.

Foster growth in Common Lisp programs, or modules, available to the CL community: the
Perl and Java communities are able to contribute code to repositories, with minimal fear
of collision, because of the hierarchical nature of the name spaces used by the
contributed code. We want the Lisp community to benefit from shared modules in the same
way.

In a nutshell, a dot (.) is used to separate levels in package names, and a leading dot
signifies a relative package name. Absolute package names require no modifications to
the underlying CL implementation. Relative package names require small and simple
modifications, for which the source code is given below.

The choice of dot follows Java. Perl, another language with hierarchical packages,
uses a colon (:) as a delimiter, but the colon is already reserved in Common Lisp.

Franz intends to extend the hierarchy below currently existing packages. For
example, we have a test harness available as a patch to ACL 5.0.1 which is in the util.test
package.

To get the Hierarchical Packages module, get the new version of code/packages.fasl
(perhaps by running sys:update-allegro )
and rebuild your images. Note that there is no way to suppress hierarchical package
features once you have the new packages.fasl except by deleting code/packages.fasl
and rebuilding your images again. (That may sound bad, but it in fact causes Allegro CL to
revert to the packages.fasl in the bundle file.)

2.0 Relative package names

Relative package names are needed for the same reason as relative pathnames, for
brevity and to reduce the brittleness of absolute names.

A relative package name is one that begins with one or more dots. A single dot
means the current package, two dots mean the parent of the current package, and so on.

Here are some examples, assuming that packages named mypack, mypack.foo,
mypack.foo.bar, mypack.foo.baz,
mypack.bar, mypack.bar.baz,
foo, and foo.bar, have all
been created:

relative name

current package

absolute name of referenced package

foo

any

foo

foo.bar

any

foo.bar

.foo

mypack

mypack.foo

.foo.bar

mypack

mypack.foo.bar

..foo

mypack.bar

mypack.foo

..foo.baz

mypack.bar

mypack.foo.baz

...foo

mypack.bar.baz

mypack.foo

.

mypack.bar.baz

mypack.bar.baz

..

mypack.bar.baz

mypack.bar

...

mypack.bar.baz

mypack

Note 1: To repeat what we said just before the table, all packages in
the hierarchy must exist.

Note 2: WARNING ABOUT NICKNAMES! Unless you provide nicknames for your
hierarchical packages (and we recommend against doing so because the number gets quite
large), you can only use the names supplied. You cannot mix in nicknames or alternate
names. cl-user (and user)
are nicknames of the common-lisp-user package. Consider
the following:

(defpackage :cl-user.foo)

When the current package (the value of *package*) is common-lisp-user, you might expect .foo
to refer to cl-user.foo, but it does not. It refers to the
non-existent package common-lisp-user.foo. Note that the
purpose of nicknames is to provide shorter names in place of the longer names designed to
be fully descriptive. The hope is that hierarchical packages makes longer names
unnecessary and thus makes nicknames unnecessary.

Note 3: multiple dots can only appear at the beginning. foo.bar..baz does not mean foo.baz
-- it is invalid. (Of course, it is perfectly legal to name a package foo.bar..baz but cl:find-package will not
process such a name to find foo.baz in the package
hierarchy.)

3.0 Compatibility with ANSI Common Lisp

The implementation of hierarchical packages modifies cl:find-package
and provides certain auxiliary functions, package-parent, package-children,
and relative-package-name-to-package, as described in this section. (defpackage
itself requires no modification.)

While the changes to cl:find-package are small and described below, it
is an important consideration for authors who would like their programs to run on a
variety of implementations that using hierarchical packages will work in an implementation
without the modifications discussed in this document. We show why not after describing the
changes to cl:find-package.

Changes to cl:find-package:

In ANSI CL, cl:find-package, if passed a package object, returns it;
if passed a string, cl:find-package looks for a package with that string
as its name or nickname, and returns the package if it finds one and returns nilif it does not; if passed a symbol, the symbol name (a
string) is extracted and cl:find-package proceeds as it does with a
string.

For implementing hierarchical packages, the behavior when the argument is a package
object (return it) does not change. But when the argument is a string starting with one or
more dots not directly naming a package, cl:find-package will, instead of
returning nil, check whether the string can be resolved as
naming a relative package, and if so, return the associated absolute package object. (If
the argument is a symbol, the symbol name is extracted and cl:find-package
proceeds as it does with a string argument.)

In Allegro CL, cl:find-package passes a string to the function excl::package-name-to-package,
which returns a package object or nil. Without
hierarchical packages, cl:find-package returns nil
if excl::package-name-to-package returns nil.
With hierarchical packages, if excl::package-name-to-package returns nil, cl:find-package calls relative-package-name-to-package,
which again returns a package object or nil. If nil is returned at this point, cl:find-package
returns nil.

The complete source code for the modifications made to Allegro CL (except for the
modifications to cl:find-package) is given below and is in the public
domain. It can be used by any CL vendor to augment their implementation to behave as
this document suggests. cl:find-package should be modified according to
the description just above -- trying to resolve a name as relative if the name does not
itself name a package.

Note that you should not use leading dots in package names when using
hierarchical packages.

Using hierarchical packages without modifying cl:find-package:

Even without the modifications to cl:find-package, authors need not
avoid using relative package names, but the ability to reuse relative package names is
restricted. The following example illustrates this:

Consider a module foo which is composed of the my.foo.bar and my.foo.baz
packages. In the code for each of the these packages there are relative package
references, ..bar and ..baz.

Implementations that have the new cl:find-package would have on their *features* list the symbol :relative-package-names.
Then, in the foo module, there would be definitions of the my.foo.bar and my.foo.baz
packages like so:

Then, in a #-relative-package-names implementation, the
symbol my.foo.bar:blam would be visible from my.foo.baz
as ..bar:blam, just as it would from a #+relative-package-names implementation.

So, even without the implementation of the augmented cl:find-package,
one can still write CL code that will work in both types of implementations, but ..bar
and ..baz are now used, so you cannot also have otherpack.foo.bar and otherpack.foo.baz
and use ..bar and ..baz as relative names. (The point of hierarchical
packages, of course, is to allow reusing relative package names.)

4.0 Package prefixes reserved by Allegro CL

From ACL 6.0 and beyond, Franz Inc. intends to put newly created packages under the
following top-level names:

net -- network related modules

uri -- URI/URL/URN parser

iserve -- web server

xml -- used for generated namespaces

java

cg -- common graphics

ide -- cg-based graphical development environment

corba

ffi -- foreign function related modules

util -- utility modules

The use of these top-level names as packages in applications might run into problems
with ACL 6.0 and future versions. Note that all current package names (such as excl,
system etc.) will also be used.

5.0 New Common Lisp functions

To facilitate using hierarchical packages, we introduce several new functions, other
than the changed cl:find-package: relative-package-name-to-package,
package-parent and package-children. The source
code for these functions is also given below.

These functions are documented below with their implementation.

6.0 The implementation

;; The following source code is in the public domain.
;; Provided "as is" with no warranty of any kind. Use at your own risk.

7.0 Tests

The following test code can be used to check a hierarchical package implementation. You
must use the Allegro CL test harness, documented in the Test Harness document. The test harness is
loaded by the (require :tester) form.

;; The following source code is in the public domain.
;; Provided "as is" with no warranty of any kind. Use at your own risk.