Advanced makefiles

In this article, we analyze a fairly complicated makefile. This makefile was captured from actual use in the wild, and sections specific to that project have been removed for this article.

This is a companion article to Introduction to make, and it builds on the topics covered there. See Introduction for a simple makefile and a guide on how to use and construct simple makefiles.

Both the introduction and this advanced article focus on make as a tool for developers. Sysadmins, make can be useful to you, too! As we step through the makefile, think about how the techniques here can be applied to rebuilding a configuration script, or an installation script, or as an auto-updating tool.

Thanks to Mitch Davis for allowing me to use his file. The modified version won't run as presented because I've removed lines, but it will allow me to display various techniques. The original is in frequent use.

The paths necessary for compilation are set up as variables. This allows them to be easily changed if the installation is moved--and you can use conditionals and shell functions to change things automatically for different machines. Conditionals and shell functions are discussed in more detail later in this article.

Recursively expanded variables

There are two forms of variable. Recursively expanded variables use an "=" between the name and the value and are evaluated at the time the variable is needed. They cannot be defined recursively, as this would create an infinite loop. (make checks for this and forbids this usage.) Functions will be calculated each time make needs the value of the variable.

OK:class_example_d = $(class_d)/example
(refers to a different variable within the definition)

Not OK:class_d = $(class_d)/example
(recursive definition--refers to itself within its definition)

Simply expanded variables

Simply expanded variables use a ":=" between the name and the value and are evaluated immediately when make encounters the variable. Recursive definition is fine, and the right-hand-side value will be the former value of the variable. Simply expanded variables are a feature of GNU make that may not exist in other versions of make.

OK:class_example_d := $(class_d)/example

OK:class := $(class)/example
(OK if class was defined before this point)

Environment variables

# These exports are for xmlc, which is
# a java program which calls javac.
export JAVAC:= $(java_h)/bin/javac
export JAVA:= $(java_h)/bin/java

Sometimes it's necessary to set environment variables within the makefile, usually when you have a project that must use a specific version of a compiler. Choose your variables carefully--changing the EDITOR variable will annoy your development team!

The classpath variable is a standard, simply expanded variable, with its definition split over several lines with the new lines escaped. Mitch then uses a make function to replace the spaces separating paths with colons. He considers this to be a much more readable variable definition than a single line, colon-separated classpath. Don't use this if you have spaces inside the classpath.

make has many useful built-in functions. The syntax of a function call is $(function arguments). Functions are evaluated when the surrounding construct is evaluated: functions in simply expanded variables are evaluated when make first encounters the variable; functions in recursively expanded variables are evaluated when make needs the value of the variable; functions in rules are evaluated when make runs that part of the rule.

Conditionals

Make has conditionals, often used when setting variables. Available conditionals are ifeq, ifneq, ifdef, and ifndef. The else clause is optional, and the arguments may be surrounded by brackets, apostrophes, or quote marks.

In the example, we could have used an ifdef rather than the ifneq to determine whether the classpath environment variable is defined.

The syntax used in this makefile is:

ifeq (arg1, arg2)
statement/s
else
statement/s
endif

Shell functions and more make functions

Make is not an ideal tool for Java compilation. The Java compiler likes to resolve dependencies itself, which can cause make some confusion. When resolving dependencies, Java works from the source files and make works from the target. Java's structure invites the casual creation of new source files, and make is designed for new source files to be added manually.

In an ideal world, each new file would be added to the make script as part of a variable definition. This project is done in the real world, so make searches the source directories and constructs the list of targets automatically. Because of Java's idiosyncrasies and some aspects of our project, this requires four variables and a mix of shell and make functions.

In the previous version of this makefile, I generated the targets with an external shell script, exporting the list to a file as make variables, then importing the make variables into the makefile. Mitch improved my makefile so targets are generated in a much neater way.

This makefile includes shell functions as well as standard make functions. The syntax for a shell function is $(shell command). This returns the output of the shell function (stripping new lines).

The patsubst function has the syntax $(patsubst pattern,replacement,text). It uses the percent symbol (%) the same way pattern rules do--as a string which matches in both the pattern and the replacement text. It searches the text for whitespace-separated words that match the pattern and substitutes the replacement for them.

Makefile variables as commands, and automatic variables

These are recursively expanded variables, calculated when the variable is actually needed. The variables need to be recursively expanded because they include automatic variables that rely on prerequisite lists and will be used as command strings in rules. (Automatic variables are explained in Introduction to make.)

$< and $? are the automatic variables. $< expands to the name of the first prerequisite. $? expands to a space-separated list of prerequisites that are newer than the target.

The $(filter %.java,$?) is necessary because when javacompile is used later in a rule, one prerequisite of the rule is the directory into which the Java files are compiled. The filter removes the directory, leaving only the .java files.

Note the use of make functions in the variable definition.

Pattern rules

The first line of these rules has three parts. These are a special type of GNU make rule called static pattern rules. There is a simple pattern rule described in Introduction to make. The syntax of a static pattern rule is:

targets: target-pattern: dependency-patterns commands

A .class file in the $(class_example_view_d) directory can be created from a corresponding .html file in the example/views/%.html directory, but only if the .class file is part of the $(VIEWFILES) list. Use static pattern rules if you have multiple ways of converting file type .A to .B and the way you choose depends on the files being converted.

The critical part of a pattern rule is the percent symbol. It refers to the same stem for both the target and the prerequisite file.