Context Navigation

Updating from Subversion and rebuilding Clozure CL from source

Getting Clozure CL via Subversion (as described on the main page) creates a "working copy" of
the CCL source code and binaries on your local system. It is also possible to get an initial working copy (including the svn metainformation in .svn directories) from a release tarball. These are periodically made available from ftp://ftp.clozure.com/pub/release/.

With a working copy in hand, you can now use Subversion to get updates.

In a shell, enter the "ccl" directory that contains the working copy of the sources.

shell> cd ccl

Tell svn to update the sources, creating new directories and removing empty ones as needed.

shell> svn update

This will try to reconcile changes made in the svn repository with the local, working copy. If you've made changes to files in
the working copy and changes have been made in the repository, svn will attempt to merge those changes into an updated copy of
the local file. It's often able to do so reasonably well (for text files); sometimes, the repository changes and local changes can't be resolved automatically and the affected file is said to be "in conflict." svn has no good way to even try to merge changes to binary files (like the CCL kernel and heap image).

As "svn update" reconciles changes to the repository with any changes made to the working copy, it prints a line consisting of
a one-letter code, some whitespace, and the pathname of the affected item. These codes include:

A

path

The file or directory at path was added to the repository and has been added to the working copy

D

path

The file or directory at path was deleted from the repository and has been deleted from the working copy

U

file

The file at file was changed in the repository and the local copy has been updated

G

file

The file at file was changed in both the repository and working copy, and the local copy contains the merged content

C

file

The file at file was changed in both the repository and working copy, and svn was unable to resolve conflicts

Binary files that have been locally modified (a rebuilt lisp kernel or heap image counts as a modification) generally can't be merged with newer versions from the repository. The lisp kernel and heap image generally don't change often in a nominally stable "release", but when they do the changes are often significant. Therefore, if you notice that the lisp kernel or heap image have been updated, and the line is marked with a "C", the correct response is to revert the file:

shell> svn revert some-file

will effectively discard any local changes to some-file and replace it with the repository version.

Note that version 1.5 and later of svn will offer a menu of options that can be used to resolve conflicts during "svn update", at least when it believes that it's running interactively. The option "tf" ("theirs full") will overwrite a local file with the repository version, which is generally appropriate when binary files change.

Once svn has finished creating local copies of files that have changed on the svn server since the last time you did this (or since the tarballs were created), use the currently installed Clozure CL to compile the updated sources and build a new kernel and heap image:

shell> ccl # or "ccl64", or m-x slime, or whatever you do to start Clozure CL
Welcome to whatever version this is!
? (ccl:rebuild-ccl :full t)

(Windows users: use (ccl:rebuild-ccl :clean t). See WindowsNotes for more information.)

Important Note: to rebuild Clozure CL on Mac OS X 10.6 Snow Leopard, you must install the optional 10.4 support when installing Xcode.

REBUILD-CCL with the :FULL option will try to build the lisp kernel (which is written in C and assembler) on platforms where it's possible to do so. For this to work, the C compiler toolchain needs to be installed. See http://ccl.clozure.com/manual/chapter3.4.html#Building-the-kernel for information about building the kernel and the required tools.

That'll recompile the lisp kernel and all lisp sources used to build a heap image and build that image.
Once that completes (usually takes 1 minute +/- 30 seconds) there should be a new heap image and new lisp kernel.
There may be compiler warnings during compilation, usually related to calls to functions that will be defined
when the process finishes but which are not yet defined during compilation (or which take a different number of arguments at compile time than they will when the image is built, etc.) Once in a while, some constants get redefined and a continuable error is signaled; continue from the CERROR to use the new value.

After a while, this (simple and simple-minded) bootstrapping process will cease to work: the newer source code will diverge from the code used to build the lisp image enough that it'll have to be bootstrapped in a more complicated (and usually totally ad-hoc) way. When this happens, new heap images and/or lisp kernels are checked in to svn (as soon as possible after the affected sources are checked in.) As noted above, this shouldn't happen too often in the "release" tree (where most changes are simple, relatively isolated bug fixes.) If it ever does happen, it's worth being aware of the fact that this may result in those files being marked as being in conflict and that "svn revert" can be used to resolve that conflict.

An all-in-one alternative

As an alternative to the above multi-step process, you can start up the lisp and evaluate (rebuild-ccl :update t :full t). This will run svn update and try to take care of conflicts involving the lisp kernel and heap image binaries automatically.