Note that there are many classes defined as part of the defsystem
facility and named by exported symbols in the defsystem package. The
facility is in the defsys module (and associated
modules). Defsystem code is normally included automatically in
development images, but if you need to load it, evaluate
(require :defsys) (it is not an error to evaluate
that form when the module is already loaded).

Defsystem is a tool designed to help developers deal with large
programs contained in multiple files. The defsystem facility improves
the development environment by providing quick and easy ways to
perform complicated tasks on a set of files.

For example, let's say a developer has a program with 20 different
files and has just edited 3 of the files. Those 3 files will need to
be recompiled. There may also be other files that will need to be
recompiled because the code in them depends in some way on the code in
the files that were just edited. Without defsystem, the developer
would have to separately issue commands to compile and load each of
the files in the correct order. With defsystem, the developer can set
up a system by indicating which files should be included in the system
and in what order they need to be compiled. Then after editing some of
the files, the developer can just issue one function call and the
defsystem will figure out which files need to be recompiled and will
do the compilation in the correct order.

The defsystem facility is strongly based on CLOS, which means that the
defsystem can be easily extended and customized to meet individual
needs. The second part of this chapter describes the ways in which the
defsystem facility can be extended and customized.

Many symbols associated with the defsystem facility are in the
defsystem package. It has nicknames
defsys and ds. However, most
operators (functions, macros, etc.) are named by symbols in the excl
package.

source file: a file which can be compiled to produce a product file. In Allegro CL,
a source file name usually ends in .cl (i.e. foo.cl).

product file: the result of compiling a source file. In Allegro CL, a product file
name usually ends in .fasl (i.e. foo.fasl).

module: a module represents one source and product file pair. Modules contain data
concerning dependencies between it and other modules and attributes of the file. The
filename without the extension names the module. Module names are strings. For example, if
foo.cl is the source file and foo.fasl is the product file, "foo"
is the module name.

module-specification: a module-specification describes one or more modules and
dependencies between modules. There are two types of module-specifications: short form and
long form.

short form module-specification: A short form module-specification provides a simple
syntax to describe one or more modules.

long form module-specification: A long form module-specification provides a more
complex syntax to describe one or more modules.

module-group: a module-group contains one or more module-groups and/or one or more
modules. A module group is created from a defsystem module specification.

system: a system contains one or more module-groups.

system name: A system name is a symbol that refers to a system. We recommend that
one use a keyword so the system-name is not in any specific package.

system-object: A system-object is a data construct that contains information about a
system.

system-specifier: A system-specifier is either a system-name or a system-object.

Systems are defined with the defsystem macro. There are two parts to a
system definition: the system options and the
module-specification. System options specify attributes of the system
or define default values for attributes of modules in the
system. Module-specifications describe the module-groups of the system
and dependencies between modules and module-groups.

An item in a module-specification can be specified with a file name,
the name of a named module group defined earlier in the same system,
or the name of another system. Supplying the name of another system
in a module-specification causes the named system to be a component
system of the system being defined. This means that system operations
will be performed recursively on the component system (if the
:include-components keyword to the system operation
method has the value t).

The defsystem macro is
defined on its own page. It takes a module name (a symbol) and a
possible empty list of system-options as required arguments followed
by a list of module-specifications.

The allowable system-option keywords are shown in the table
below. After the table, we describe module specifications.

This pathname will be merged with the filenames of the
modules of the system in order to find files.

:default-file-type

string

depends on value of :default-module-class
- see notes at right.

Specifies file type (extension) for source files for modules
in system. If :default-module-class is
ds:lisp-module, default is value of
*source-file-types*; if ds:text-module, default is nil; if ds:c-module, default is
"c".

More than one module-specification may be specified when defining a
system. Module-groups can be specified using a short form
module-specification or a long form module-specification. We describe
short and long form module-specifications below. Note that we use the
terms short form specification and short form module-specification
interchangeably. They mean the same thing. We do the same thing with
long form specification and long form module-specification.

(:module ...) and
(:module-group...) create module groups, as do all
other defsystem module-specifications. Module-specifications which
begin with the keywords :module and
:module-group provide ways of associating a name
with a module-group.

Note that a module-specification which begins with
:module really creates what we call a module-group.

The (:module ...) syntax for long-form
module-specifications is used to maintain syntax compatibility with
the defsystem facility available on Symbolics machines. Note that our
term module-group is similar to what is called a
module in the Symbolics' documentation of their
defsystem.

Short form module-specifications provide a simple syntax for
describing systems that:

have only serial and parallel dependencies

have only one package (given by the :default-package option to defsystem)

have only modules of the default type

More complex needs can be met by using long form module-specifications.

A short form module-specification is a list beginning with one of
:serial, :parallel,
:definitions, or
:module-group. The table below gives a description
of each of the short forms. In the table, module-spec-element
may:

name a module

specify another module-group using a short form specification

name a module-group defined earlier in the system definition

name another system to be treated as a component system.

Short form name

Arguments

Discussion

:parallel

{module-spec-element}+

This module-specification keyword specifies that the elements
listed have no dependencies between them. A :parallel
module-specification list causes each element to be processed as an individual. This
module-specification list does not cause each element to be loaded before the next element
is compiled.

:serial

{module-spec-element}+

This module-specification keyword specifies that each element
depends on the element listed before it in the list. Therefore, during compilation, each
module-spec-element specified is compiled and loaded before the next.

:definitions

primary element

[Both arguments are module-spec-elements]

This module-specification keyword option specifies that element
has a serial dependency on primary and also has a compile-dependency on primary
such that if primary is compiled, element must be compiled. In other words element
uses definitions in primary. If primary is touched, then element
should be recompiled or reloaded.

This keyword is often used to describe dependencies
between files containing macros and files that use macros.

:module-group

name short-form

{long-form-option}*

This module-specification keyword is used to name a
short-form. :module-group gives the name name
to the short-form system described by short-form. name can then be used to
refer to short-form in other specifications. name is a symbol (it is not
evaluated). short-form is a list that describes a module-group. long-form-option
can be any option which can be used for long form specifications as described below. Zero
or more long-form specifications can be specified.

Table 2: Short Form module
specifications

Most simple needs can be met by use of the :parallel
and :definitions short forms. The following are
examples of defining short form systems.

In this example, file2 is dependent on file1. Also, filea and the
system my-other-system are dependent on file2.

The :definitions module-specification keyword
is different from the :serial
module-specifications keyword in that :serial
does not force recompilation of modules farther down the list. As an example, suppose we
have:

Long form module-specifications should be used when more control over
some module-group or module-groups is desired. With the long form it
is possible to override the system default values for various
parameters and to specify more complex dependencies.

There are two types of long form module-specifications, one allows a
group of modules to be given a name which can be referenced by other
module-specifications in the system and the other type provides a
simple way to specify non-default options for just one module.

A long form specification is a list of one of the following forms:

(:module name system-or-files {long-form-option}*)

or

("filename" {long-form-option}*)

Here name is a symbol that names the module-group. name can be used by
other module-groups in the system to refer to the module-group it names. system-or-files
can be a string or (non-empty) list of strings representing the one or more source files
of the module-group, or a symbol representing the name of another system.

Note that :module-group is used to name short
form module-specifications, while :module is used
to name long form module-specifications. In both cases, the name is associated with the
resulting module-group.

("filename" {long-form-option}+)

is functionally equivalent to

(:module foo "filename" {long-form-option}+)

except

("filename" {long-form-option}+)

does not have the added name foo associated with it.

Zero or more keyword options can be specified in the
long-form-option position. The allowable keyword options are given in
the following table. Note that some options have more than one or a
variable number of arguments. All have at least one argument. The `+'
after an argument indicates at least one value must appear but as many
as you like can appear.

Keyword option name

Arguments

Discussion

:package

package-name

Must be a package object or a symbol or string identifying a
package. If specified, this will override the default package for the system

:module-class

module-class-name

Argument must be a symbol naming a module class. If
specified, this overrides the default module class of the system. Pre-defined valid values
are ds:lisp-module, ds:text-module, and ds:c-module.
Users may define additional classes. See the information under
the heading Section 6.2 Defining new classes
below.

:in-order-to

operation requirement

operation can be :compile
or :load or the list (:compile :load). requirement must be a list whose first element is either
:compile or :load
and whose other elements (there must be at least one) are symbols each denoting a
module-group. This option says that before operation can be performed on the module,
requirement must first be satisfied.

:uses-definitions-from

definition-module-group+

Argument(s) must be symbols naming module-groups. This option
says that (1) the module-group must be recompiled if any definition module is recompiled;
(2) if the module-group is to be compiled, then all definition-modules must be recompiled
first; (3) if the module-group is to be loaded, all definition-modules must be loaded
first.

:recompile-on

needed-module-group+

Argument(s) must be symbols naming module-groups. Same as :uses-definitions-from except needed-module-groups
do not have to be loaded if module-group is loaded.

:compile-satisfies-load

boolean

If true, tells system that compiling module-group makes the
module up-to-date in the lisp image as if it had been both compiled and loaded.

:load-before-compile

module-groups-or-systems

Argument may be a symbol naming a module-group or system or a
list of module-group and/or system names. The module-groups and systems specified must be
loaded before module is compiled.

:force-dependent-recompile

boolean

Causes all modules examined, after a module with this option,
to be recompiled even if they are up-to-date. The main use of this is to cause
recompilation of the rest of a system because critical definitions exist in a particular
module that is out-of-date.

modules which represent the same source files in the old and new systems maintain
whatever state they had before the redefinition, which means, for example, that a module
which has been compiled and loaded will not be recompiled or reloaded after the
redefinition.

The full description of each function described in this table is given in the page
description of the function. :simulate means to
print the actions that would be taken but do not actually perform them.

Compiles (if necessary) each module in system. Returns
t if any action was taken to bring system up-to-date and returns nil if no action
was necessary. See the description for descriptions of the keyword arguments.

Because defsystem is CLOS-based, it can be easily extended in many
ways in order to add functionality or meet special needs. It is
possible to extend existing system operations or to define totally new
operations. The module actions that implement the operations on the
modules of a system can also be customized. New classes of system,
module group and module can be created to have specialized behavior
and new slots. It is also possible to extend the syntax of the defsystem macro that defines
systems. Each of these extension methods is described below.

The following is a description of the defsystem class hierarchy. Users
can build new classes based on these classes. The following can be
used as a guide for choosing which classes to build on.

defsys::defsystem-base-class

All defsystem classes inherit from this class.

defsys:module-container

This is a class of objects that contain modules. Objects of this class have a list of
modules and a default-module-class which indicates the default class to be used when
modules are created for this module-container.

defsys:default-module-group and defsys:default-system both inherit from defsys:module-container.

By defining new subclasses of existing defsystem classes, it is
possible to add new slots to defsystem objects and to write
specialized methods on defsystem generic functions which will affect
only objects of the new class.

If new classes are defined, defsystem needs to be told when to use
them for creating defsystem objects. The following variables determine
which classes the defsystem macro will use when creating
defsystem objects.

The syntax of the defsystem macro can be extended through the
use of shared-initialize
methods. In order to understand how to extend the defsystem syntax,
first it is necessary to explain how a defsystem form is processed to
create the systems, module-groups and modules.

When defsystem is called,
the system name is looked up in a table of systems to determine if
this is a new system definition or the system already exists. In the
former case, a system object of class *default-system-class* is created for the
system and the options list is passed to make-instance as a list of initialization
arguments. In the case of reinstalling an existing system, the
existing system object is reinitialized with the new options list. In
either case, shared-initialize
is called to perform initialization of the object. The options are
handled either by the shared-initialize method for the system class
which handles all slots that have initargs or by keyword arguments to
an after-method of shared-initialize defined for the system
class. So new system options can be added in either of two ways:

by defining a new system class with a new slot where the initarg of the slot will be a
valid option.

by defining an after-method of shared-initialize for the new system class which has a
keyword argument for the new option.

The list of module-specifications in the defsystem form is processed in a similar
manner by shared-initialize methods and after-methods handling the
options for module-groups and modules.

As an example of defining and using a new system class, let's say
we want a form to be evaluated after a system is loaded. We will need
to create a subclass of defsys:default-system which
will contain a new slot, eval-after-load. Then we
will need a way to initialize this slot at system creation time, and a
way to evaluate the form at system load time. For initialization we
will want to add a new option to the defsystem macro, which can be done through the
use of a shared-initialize method for our new system class. Evaluation
of the form can be implemented with an around-method on the load-system generic function.

First we will need to create the new system class. The following
code defines a subclass of defsys:default-system with the new
slot.

Now to enable initialization of the eval-after-load slot, we will
need to extend the defsystem
macro syntax for this class by creating a shared-initialize
after-method that takes an :eval-after-load
keyword.

Next, we need to define an around-method on load-system so that the contents of the
eval-after-load slot can be evaluated at system load time. In this
method, we will first call the other methods to load the system, then
check the return value to see if the system was loaded or not. If it
was, then we evaluate the eval-after-load method.

The following is a description of how system operations and module
actions work together.

System operations recursively invoke methods on each of the
components of the system. The methods that are invoked are named
foo-module-action, where
foo-system is the name of the system
operation. The foo-module-action method for
a component system invokes foo-module-action
on its components if ds::.include-components. is
true. The
foo-module-action method for a module-group
invokes foo-module-action on each of its
components. The foo-module-action for a
module decides if the foo-module method
needs to be called and calls it if necessary. The
foo-module-action methods also print
messages about the actions that are being performed. In simulation
mode, the foo-module method is not called by
the foo-module-action method.

For example, load-module-action checks if the
module has been loaded and if not, calls the
load-module method to perform the actual
load. compile-module-action and
load-module-action are the only actions that do
checking on the modules. The other module-actions unconditionally
invoke the method to perform the action unless ds::.simulate-mode. is
non-nil.

The following example demonstrates how to define a new system
operation and the module-action methods that go along
with the operation. This example implements a
grep-system operation which searches each source file
in the system for a given string. Note that this could be implemented
more easily using map-system, but this is just an example of
defining new system operations and module actions.

This example involves creating new system, module-group and module
classes and specializing defsystem methods for the
new classes.

If a developer wants to edit a few files of a large file tree, but
doesn't want to make a complete copy of the tree, he/she can define a
system that looks for files first in the developer's copy of the tree,
then in the master tree. If some modification is done that requires
the recompilation of a source file not yet copied to the developer's
tree, then defsystem will compile using the source in
the master directory and put the fasl (product) file in the developer
directory.

The default system and module classes will be subclassed with the
new slots master-directory and
developer-directory, that will contain the
pathnames of the master and developer directories.

The source-pathname
method will first look for the source file in the developer directory
and then in the master directory.