H.1.1 CVS and generated files

Background: distributed generated Files

Packages made with Autoconf and Automake ship with some generated
files like configure or Makefile.in. These files were
generated on the developer’s machine and are distributed so that
end-users do not have to install the maintainer tools required to
rebuild them. Other generated files like Lex scanners, Yacc parsers,
or Info documentation, are usually distributed on similar grounds.

Automake output rules in Makefiles to rebuild these files. For
instance, make will run autoconf to rebuild
configure whenever configure.ac is changed. This makes
development safer by ensuring a configure is never out-of-date
with respect to configure.ac.

As generated files shipped in packages are up-to-date, and because
tar preserves times-tamps, these rebuild rules are not
triggered when a user unpacks and builds a package.

Background: CVS and Timestamps

Unless you use CVS keywords (in which case files must be updated at
commit time), CVS preserves timestamp during ‘cvs commit’ and
‘cvs import -d’ operations.

When you check out a file using ‘cvs checkout’ its timestamp is
set to that of the revision that is being checked out.

However, during cvs update, files will have the date of the
update, not the original timestamp of this revision. This is meant to
make sure that make notices sources files have been updated.

This timestamp shift is troublesome when both sources and generated
files are kept under CVS. Because CVS processes files in lexical
order, configure.ac will appear newer than configure
after a cvs update that updates both files, even if
configure was newer than configure.ac when it was
checked in. Calling make will then trigger a spurious rebuild
of configure.

Living with CVS in Autoconfiscated Projects

There are basically two clans amongst maintainers: those who keep all
distributed files under CVS, including generated files, and those who
keep generated files out of CVS.

All Files in CVS

The CVS repository contains all distributed files so you know exactly
what is distributed, and you can checkout any prior version entirely.

Maintainers can see how generated files evolve (for instance, you can
see what happens to your Makefile.ins when you upgrade Automake
and make sure they look OK).

Users do not need the autotools to build a checkout of the project, it
works just like a released tarball.

If users use cvs update to update their copy, instead of
cvs checkout to fetch a fresh one, timestamps will be
inaccurate. Some rebuild rules will be triggered and attempt to
run developer tools such as autoconf or automake.

Calls to such tools are all wrapped into a call to the missing
script discussed later (see maintainer-mode), so that the user will
see more descriptive warnings about missing or out-of-date tools, and
possible suggestions about how to obtain them, rather than just some
“command not found” error, or (worse) some obscure message from some
older version of the required tool they happen to have installed.

Maintainers interested in keeping their package buildable from a CVS
checkout even for those users that lack maintainer-specific tools might
want to provide an helper script (or to enhance their existing bootstrap
script) to fix the timestamps after a
cvs update or a git checkout, to prevent spurious
rebuilds. In case of a project committing the Autotools-generated
files, as well as the generated .info files, such script might
look something like this:

In distributed development, developers are likely to have different
version of the maintainer tools installed. In this case rebuilds
triggered by timestamp lossage will lead to spurious changes
to generated files. There are several solutions to this:

All developers should use the same versions, so that the rebuilt files
are identical to files in CVS. (This starts to be difficult when each
project you work on uses different versions.)

Or people use a script to fix the timestamp after a checkout (the GCC
folks have such a script).

Or configure.ac uses AM_MAINTAINER_MODE, which will
disable all of these rebuild rules by default. This is further discussed
in maintainer-mode.

Although we focused on spurious rebuilds, the converse can also
happen. CVS’s timestamp handling can also let you think an
out-of-date file is up-to-date.

For instance, suppose a developer has modified Makefile.am and
has rebuilt Makefile.in, and then decides to do a last-minute
change to Makefile.am right before checking in both files
(without rebuilding Makefile.in to account for the change).

This last change to Makefile.am makes the copy of
Makefile.in out-of-date. Since CVS processes files
alphabetically, when another developer ‘cvs update’s his or her
tree, Makefile.in will happen to be newer than
Makefile.am. This other developer will not see that
Makefile.in is out-of-date.

Generated Files out of CVS

One way to get CVS and make working peacefully is to never
store generated files in CVS, i.e., do not CVS-control files that
are Makefile targets (also called derived files).

This way developers are not annoyed by changes to generated files. It
does not matter if they all have different versions (assuming they are
compatible, of course). And finally, timestamps are not lost, changes
to sources files can’t be missed as in the
Makefile.am/Makefile.in example discussed earlier.

The drawback is that the CVS repository is not an exact copy of what
is distributed and that users now need to install various development
tools (maybe even specific versions) before they can build a checkout.
But, after all, CVS’s job is versioning, not distribution.

Allowing developers to use different versions of their tools can also
hide bugs during distributed development. Indeed, developers will be
using (hence testing) their own generated files, instead of the
generated files that will be released actually. The developer who
prepares the tarball might be using a version of the tool that
produces bogus output (for instance a non-portable C file), something
other developers could have noticed if they weren’t using their own
versions of this tool.

Third-party Files

Another class of files not discussed here (because they do not cause
timestamp issues) are files that are shipped with a package, but
maintained elsewhere. For instance, tools like gettextize
and autopoint (from Gettext) or libtoolize (from
Libtool), will install or update files in your package.

These files, whether they are kept under CVS or not, raise similar
concerns about version mismatch between developers’ tools. The
Gettext manual has a section about this, see Integrating with CVS in GNU gettext tools.