The Trouble with Hidden Targets

Make programs are very good at keeping track of targets, files that need to be built, and the dependencies between targets. But the Make program is only as good as its inputs. If you don't tell Make about a relationship between two files, it wont discover it on its own and it'll make mistakes because it assumes it has perfect knowledge about the files and their relationships.

On the face of it this looks pretty simple. If you run this through GNU Make it'll build foo (which creates the files foo and foo.c), then it'll use the pattern at the bottom to make foo.o from foo.c. It ends up running the following commands:

touch foo foo.c
touch foo.o

But there's a fatal flaw. Nowhere does this Makefile mention that the rule to make foo actually also makes foo.c. So foo.c is a hidden target, and hidden targets cause no end of problems.

Make programs are very good at keeping track of targets, files that need to be built, and the dependencies between targets. But the Make program is only as good as its inputs. If you don't tell Make about a relationship between two files, it wont discover it on its own and it'll make mistakes because it assumes it has perfect knowledge about the files and their relationships.

In this example Make only works because it builds the prerequisites of all from left to right. First it encounters foo, which it builds creating foo.c as a side effect and then it builds foo.o using the pattern. If you change the order of the prerequisites of all so that it doesn't build foo first then the build will fail.

There are (at least!) five nasty side-effects of hidden targets.

Unexpected error message if the hidden target is missing

Suppose that foo exists, but foo.c and foo.o are missing. You'd expect the Makefile to update foo.o, but since it doesn't know how to make foo.c (since it's not mentioned as the target of any rule), invoking Make results in the error:

No rule to make target `foo.c', needed by `foo.o'.

The helpful -n debugging option fails

The -n option to Make tells it to print out the commands that it would run to perform the build without actually running them. Above we saw that Make would actually perform two touch commands (touch foo foo.c followed by touch foo.o), but doing a make -n (with no foo* files present) results in an error. Make doesn't know that the rule for foo makes foo.c, and because it hasn't actually run the touch command foo.c is missing. This means that the -n doesn't represent the actual commands that Make would run, making it useless for debugging.

touch foo foo.c
No rule to make target `foo.c', needed by `foo.o'.

You can't parallelize the Make

Make provides a handy feature that allows it to run multiple jobs at once. If you've got many compiles in a build specifying the -j option (followed by a number indicating the number of jobs to run at the same time) can maximize CPU utilization and shorten the build.

Unfortunately a hidden target spoils that plan. Here's the output from make -j3 running three jobs at once. Make tried to build foo, foo.o and foo.c all at the same time and discovered that it didn't know how to build foo.c because it had no way of knowing that it should wait for foo to be made.

touch foo foo.c
No rule to make target `foo.c', needed by `foo.o'. Waiting for unfinished jobs....

Make does the wrong work if the hidden target is updated

Suppose the file foo.c already exists when Make is run. Because Make doesn't know that the rule for foo is going to mess with foo.c it'll get updated even though it's up to date. In the example it's a benign touch operation, but it could simply destroy the contents of the file.

touch foo foo.c
touch foo.o

Can't direct Make to build foo.o

You'd hope that typing make foo.o would result in Make building foo.o from foo.c and if necessary building foo.c. But Make doesn't know how to build foo.c, it just happens by accident when building foo. So, if foo.c is missing make foo.o results in an error:

No rule to make target `foo.c', needed by `foo.o'.

Hopefully you're convinced now that hidden targets are a bad idea and can lead to all sorts of odd build problems.

About the author

John Graham-Cumming is Co-Founder at Electric Cloud, Inc . Prior to joining Electric Cloud, John was a Venture Consultant with Accel Partners, VP of Internet Technology at Interwoven, Inc. (IWOV), VP of Engineering at Scriptics Corporation (acquired by Interwoven), and Chief Architect at Optimal Networks, Inc. John holds BA and MA degrees in Mathematics and Computation and a Doctorate in Computer Security from Oxford University. John is the creator of the highly acclaimed open source POPFile project. He also holds two patents in network analysis and has others pending.