Using package autoloading in Tcl

Peter Briggs, November 2002

This is a quick write-up of my experiences with setting up packages
to use the Tcl "package require" mechanism to automatically load
extensions.

A package in Tcl can be a Tcl script (*.tcl), a loadable binary
(*.so), or a combination of the two. Examples of the first are xml.tcl
and sgml.tcl (distributed with CCP4 in the $CCP4/ccp4i/src directory);
examples of the second are BLT and ccp4map.

Tcl provides a command called pkg_mkIndex, which can build an index
file to enable the automatic loading of packages via the "package require"
command. The man pages for pkg_mkIndex give the full story but a quick
guide to the steps is:

Create the package: in the case of a script-based package, it
must contain an invocation of the "package provide" command, e.g.

package provide junk 1.0

In the case of a loadable binary it must have a call to "Tcl_PkgProvide",
in the Tcl C library.

Create the index: by invoking the pkg_mkIndex command, e.g.

pkg_mkIndex /usr/users/pjx/junk *.tcl

This will scan the files matching the expression (in this case all files with
extension "tcl") in the named directory, and create a file called
pkgIndex.tcl which should contain the commands to load all the
packages that it finds there.

Install the package: the package will consist of a directory with
the file(s) making up that package, plus the pkgIndex.tcl file. There are a
couple of options which will enable autoloading to work:

Make it a subdirectory of one of the directories given in the tcl_pkgPath
variable, or

If you install it elsewhere on your system, then make sure you add the
directory name to the auto_path global variable. (This can be
either explicitly e.g. lappend auto_path $dir within the
application itself, or via the TCLLIBPATH environment variable.)

Use the package: by adding a line like e.g.

package require junk

in your application. Providing everything has been set up as described, the
Tcl interpreter will be able to locate and load the package automatically.

Problems

I encountered two particular cases when I tried this on the examples
here. (It is useful to use the "-verbose" option of pkg_mkIndex if you have
difficulties, since there is very little indication otherwise as to what
might have gone wrong.)

Package name in loadable binary contains numeric characters:
The "ccp4map" library
is provided in a loadable library as libccp4map.so. However, running
pkg_mkIndex on the appropriate directory gave an error when trying to
process this file; the message was `can't find procedure "Ccp_Init"'. The
problem stems from the fact that the procedures ignore numbers in the package
name (which means that e.g. libBLT24.so can still be recognised as the BLT
package).

Script-based package requires a variable from another package:
Although it is completely legitimate for one package to depend on another,
this can cause a problem for pkg_mkIndex if the first package explicity
names a variable from the second. This is because pkg_mkIndex doesn't
load the required package when processing the first. (Apparently this is a
known feature).

It may be that it is possible to overcome these problems by using the various
options available to pkg_mkIndex, but I haven't investigated this yet.

Making pkgIndex.tcl manually

The problems above forced me to make my own pkgIndex.tcl files. For a
script-based package the pkgIndex.tcl file should contain a line of the
form:

e.g. "package ifneeded ccp4map 2.1 [list load [file join $dir libccp4map[info sharedlibextension]] ccp4map]". I imagine that more complex commands are also possible - for
example if a package was made up of both script and binary components.

Where to get more information

I used the man pages for the Tcl package and pkg_mkIndex
commands as my primary reference for the things I tried above, and would recommend
these as good starting points for finding out more (particularly regarding the
mechanics of how ``package require'' searches for packages, and for how
pkg_mkIndex processes files.