Saturday, April 04, 2015
Some thoughts on make
From time to time,
I find make limiting,
think there has to be a better way and I start playing around with the idea of building a new make.
It's not that uncommon for programmers to think this (SCons,
CMake,
ant,
rake and a bunch of other programs)
but oddly enough,
as I try them,
I keep going back to make
(specifically,
GNU make;
note that when I mention make,
I am talking about GNU make).
I think the main reason I go back is that make is mainly a syntax-light declarative langauge with bits of shell scripting thrown in.
The other build systems are generally based around another language I do not know,
they tend towards being very imperative,
and rarely can you parallelize building the software
(and when you get the makefile right, a parallel run of make just flies).
So when I had that “let's remake make” itch this time,
I thought that perhaps I would see if I could stay within the confines of the existing syntax of make as much as possible.
I wasn't trying to actually program my own make,
but rather,
just play around with how I would like make to look and work.
The main problem with make is declaring the dependencies.
For instance,
I'm embedding a UUID library into a larger project,
and because recursive make is considered harmful,
I include in my makefile:
lib/libspcuuid.a : build/third_party/uuid/luauuid.o \ build/third_party/uuid/uuid_ns_dns.o \ build/third_party/uuid/uuid_ns_null.o \ build/third_party/uuid/uuid_ns_oid.o \ build/third_party/uuid/uuid_ns_url.o \ build/third_party/uuid/uuid_ns_x500.o \ build/third_party/uuid/uuidlib_cmp.o \ build/third_party/uuid/uuidlib_parse.o \ build/third_party/uuid/uuidlib_toa.o \ build/third_party/uuid/uuidlib_v1.o \ build/third_party/uuid/uuidlib_v2.o \ build/third_party/uuid/uuidlib_v3.o \ build/third_party/uuid/uuidlib_v4.o \ build/third_party/uuid/uuidlib_v5.o build/third_party/uuid/luauuid.o : third_party/uuid/src/luauuid.c build/third_party/uuid/uuid_ns_dns.o : third_party/uuid/src/uuid_ns_dns.c build/third_party/uuid/uuid_ns_null.o : third_party/uuid/src/uuid_ns_null.c build/third_party/uuid/uuid_ns_oid.o : third_party/uuid/src/uuid_ns_oid.c build/third_party/uuid/uuid_ns_url.o : third_party/uuid/src/uuid_ns_url.c build/third_party/uuid/uuid_ns_x500.o : third_party/uuid/src/uuid_ns_x500.c build/third_party/uuid/uuidlib_cmp.o : third_party/uuid/src/uuidlib_cmp.c build/third_party/uuid/uuidlib_parse.o : third_party/uuid/src/uuidlib_parse.c build/third_party/uuid/uuidlib_toa.o : third_party/uuid/src/uuidlib_toa.c build/third_party/uuid/uuidlib_v1.o : third_party/uuid/src/uuidlib_v1.c build/third_party/uuid/uuidlib_v2.o : third_party/uuid/src/uuidlib_v2.c build/third_party/uuid/uuidlib_v3.o : third_party/uuid/src/uuidlib_v3.c build/third_party/uuid/uuidlib_v4.o : third_party/uuid/src/uuidlib_v4.c build/third_party/uuid/uuidlib_v5.o : third_party/uuid/src/uuidlib_v5.c
Or, if I wanted to save myself some typing:
define OBJECT_template = $(1) : $(2) endef UUIDSRC = $(wildcard third_party/uuid/src/*.c) lib/libspcuuid.a : $(subst third_party/uuid/src,build/third_party/uuid,$(UUICSRC)) $(foreach target,$(UUIDSRC),$(eval $(call OBJECT_template(build/third_party/uuid/$(notdir $(target)),$(target)))))
and have no idea what is going on three months from now (if indeed, I got that right). Instead, I would like to type:
lib/libspcuuid.a : build/third_party/uuid/*.o build/third_party/uuid/*.o : third_party/uuid/src/*.c
and be done.
This should be possible.
“lib/libspcuuid.a” doesn't exist,
but it depends upon all the “.o” files in “build/third_party/uuid”,
which don't exist,
but we have a rule that says “for ‘.o” files in ‘build/third_party/uuid’,
there should be a corresponding ‘.c” file in ‘third_party/uuid/src/’ that is is generated from.”
Which also leads to another problem with make:
if “build/third_party/uuid” doesn't exist,
make should make it!
Automatically!
I mean,
make has no problems with “uuidlib_v5.o” not existing—that's the reason we're using make in the first place,
to make files!
If a directory a file lives in doesn't exist,
make should make that too.
Right?
By the same token, if I wanted to include some Lua modules but not all of them, I would like to do:
SPCMODC = env errno fsys hash math net pollset process strcore sys syslog
SPCMODL = debug getopt string table unix
lib/libspcmod.a : build/third_party/lua-conmanorg/*.o
build/third_party/lua-conmanorg/*.o : third_party/lua-conmanorg/src/{ $(SPCMODC) }.c \
third_party/lua-conmanorg/lua/{ $(SPCMODL) }.lua
(Ah! So that's where yesterday's code snippet came from!)
I'll spare you the expansion.
Another small bit of annoyance is writing implicit rules to build the files.
I learned the hard way that in make,
this:
%.o : %.c $(CC) $(CFLAGS) -c -o $@ $<
only works if the “.c” and “.o” files are in the same directory! So the above rules works fine for:
long/path/to/src/foo.o : long/path/to/src/foo.c src/bar.o : src/bar.c snafu.o : snafu.c
But not for:
build/foo.o : long/path/to/src/foo.c build/bar.o : src/bar.c build/snafnu.o : snafu.c
Nope, I also have to write:
build/%.o : long/path/to/src/*.c $(CC) $(CFLAGS) -c -o $@ $< build/%.o : src/%.c $(CC) $(CFLAGS) -c -o $@ $< build/%.o : %.c $(CC) $(CFLAGS) -c -o $@ $<
if I want to segregate the “.o” files from the “.c” files.
How hard is it to realize that if I have a “.c” file, then the command to make a “.o” file is the same, regardless of where the “.c” file comes from, or where the “.o” file is being generated. Hello?
And while we're on that subject, one implicit rule I use is the following:
%.o : %.lua $(LUAC) -o $(@D)/$(*F).out $< $(BIN2C) -9 -o $(@D)/$(*F).c -t $(*F) $(@D)/$(*F).out $(CC) $(CFLAGS) -c -o $@ $(@D)/$(*F).c $(RM) $(@D)/$(*F).out $(@D)/$(*F).c
That bit of jibberish first compiles the Lua code using luac,
then that output is converted to a C file which is then compiled into a object file so I can link it into the final executable,
them removes the temporary files it needed to do all that.
The noise you see is the cruft necessary to specify temporary files that won't get overwritten when doing a parallel build.
Much nicer would be something like:
%.o : %.lua $(LUAC) -o $$.1 $< $(BIN2C) -9 -o $$.2 -t $(*F) $$.1 $(CC) $(CFLAGS) -c -o $@ $$.2 $(RM) $$.1 $$.2
(even better—make automatically deletes the intermediate files unless I tell it not to)
One complaint about make
(or rather,
makefiles themselves)
is that a change to the makefile does not cause a recompile,
mainly because the makefile itself is never listed as a dependency anywhere.
This is never what you want!
If the makefile was listed as a depenency
(either explicitly or implicitly),
then adding a new file to be compiled in would cause everything to be recompiled,
when it should be the new file,
any files that call code in the new file,
and a final relinking of the code is all that is needed.
Also,
if you change a variable,
like $(LDFLAGS),
then all that should happen is any rule that uses $(LDFLAGS) should be run,
not a complete build.
In other words,
you really have a dependency on a portion of the makefile,
not the makefile as a whole.
But solving this would require a possible rethink of how make works
(perhaps a cached version of the makefile that mode checks against,
and intelligently applies the changes as dependencies? You know,
so if you change $(CC),
any rule using $(CC) is automatically run).
I realize these some of my ideas are not that easy to implement
(heck,
I wasted a few hours yesterday writing code to properly build a list of targets and dependencies based on ideas presented here),
but I think they may go a long way to making make less horrendous to use.
![[The future's so bright, I gotta wear shades]](https://www.conman.org/people/spc/about/2023/1023.t.jpg "The future's so bright, I gotta wear shades")