Make to Act Only on Modified Files From Prerequisites

Make is very clear on how it tracks changes. There are output files and input files. If input is newer than the output, things need building. It may not be fast and it does not lack layers upon layers (e.g., Archive Members), but its essence is rather clear. With:

%.b: %.a
	cat $< >$@

hello.b:

We can read that whenever hello.a is modified a new hello.b will be generated from its content.

Today let's talk about situation in which we have multiple prerequisite files. A sample rule that permits to concatenate multiple .a files can be written in a makefile as follows:

%.b: %.a
	cat $^ >$@

hello.b: hi.a hey.a

We can already make two significant observations. The first point of interest is common and usually, I hope, consciously ignored. Usually described as "an implied dependency on hello.a". To be precise, what is implied here is not the dependency itself, but the entire recipe - name of the output and one of the inputs. We simply add two more dependencies on top of something that is not really visible but matches. If we were to change the name of the output to greetings.b, nothing would generate.

Second thing to note here is that the output does not allow updating. On each run it must regenerate the entire hello.b.

To tweak the first behaviour we can remove prerequisites from the rule:

%.b:
	cat $^ >$@

greetings.b: hello.a hi.a hey.a

This changes rule deduction behaviour a bit since there is no longer a visible input file extension that we can use for deduction. The rule is chosen solely with the output pattern. Of course, we also need to specify the previously implied input file that matched output name based on the previous rule.

In order to tweak the rule to allow partial regeneration (or simply: updating), we must change context and use different automatic variable: $?. For now, let's try with just the new variable. The new makefile will look like this:

%.b:
	cat $? >$@

greetings.b: hello.a hi.a hey.a

Context problem is visible only after we perform some subsequent builds:

$ make
cat hello.a hi.a hey.a >hello.b
$ cat hello.b
hello
hi
hey
$ touch hi.a
$ make
cat hi.a >hello.b
$ cat hello.b
hi

First run is just fine, like predicted the problem shows after it and thanks to make printing commands it is also nicely visible. Using redirection that appends to the end of a file would simply write another hi at the end. The problem is that we are operating on plain text files and simply append to them. The same situation would occur with other similar formats (e.g., tarballs).

The problem is the object does not support updating that's equivalent to the first creation. That's not always the case:

%.b:
	ar r $@ $?

greetings.b: hello.a hi.a hey.a

ar supports such operation. Consequent runs will replace content of the greetings.b archive as files are modified. If a new file is added to the dependencies, it will only append this new files assuming nothing else was modified.

What if we do not have an output file? Recently I was playing around with a new idea and I used NodeMCU as a platform to implement it on. It's small. Its filesystem and set of services is very simple, so usual things like rsync were not an option. To reduce the time spent on uploading files I wanted to send only ones that were updated, but they were sent to another device, so there was no target file to compare the dates with.

Usual approach to similar problems is usually just ignoring it and making a phony target instead. Sometimes even with a file present we choose to do so, for example installation targets often completely omit output specification.

Well, if we don't have a file, we simply need to create one:

upload: .marker

.marker: a.lua b.lua c.lua
	nodemcu-uploader -p $(PORT) upload $?
	@touch $@

.PHONY: upload

Here, a special file .marker is silently created or updated on each upload. Action is performed silently thanks to @ at the beginning of the command. The idea is to avoid clutter in logs. The additional upload target is there to provide a more human recognizable name. Marker filename is not put into a variable to reduce the complexity of the example.

$ make
nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua
$ touch a.lua
nodecmu-uploader -p /dev/ttyUSB0 upload a.lua
$ make clean upload
rm -f .marker
nodemcu-uploader -p /dev/ttyUSB0 upload a.lua b.lua c.lua

Of course, state of the files in the device is not tracked and different devices are not recognized. Still, this method has a rather good balance between the overall complexity and how often each branch of its logic is used.

This sounds like a good opportunity to use an archive member target and pretend we put files into the marker file, but it will not work as expected, because it will attempt to rebuild (and thus upload) each individual target every single time. Shame.