How to Write Install Targets for Lua Modules in Makefiles

You can skip the full context and go right to the explanation of the proposed makefile.

"Installation" can mean two things: installing packages using package manager (e.g., pacman(8), apt(8)) or copying files from one spot to another. Of course, package managers may copy files during the installation process, but in the latter case I mean only copying and maybe creating destination directories.

simplified pigeon installation process

The usual encounter goes like: download some sources, build them, run checks, then run as is or install to /usr/local to make it available system-wide and then run.

With make(1) it's like: 

$ make
$ make test  # or: check
# make install

We might see many different installation targets, but if there's any, install should be one of them. These targets conventionally use several variables to change their behaviour. Among those we can see: 

DESTDIR=
PREFIX=/usr/local
BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
INCLUDEDIR=$(PREFIX)/include
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1

When we run make PREFIX=/usr install, the PREFIX will change to /usr and so will all downstream directories. Note that nothing uses DESTDIR yet. DESTDIR is used for staging the files with their intended structure before they make it to the real target location. This is commonly used by package maintainers along with fakeroot(1) or similar method.

For more details about these paths consult Filesystem Hierarchy Standard, hier(7), file‑hierarchy(7) and system specific documentation. For more information about names of the variables see Standard Targets for Users and Variables for Installation Directories from GNU Make documentation.

Now consider a simple target definition: 

install: all
	install -m644 -Dt $(DESTDIR)$(INCLUDEDIR) include/*.h
	install -m755 -Dt $(DESTDIR)$(LIBDIR) libuseful.so.1.0.0
	ln -s $(DESTDIR)$(LIBDIR)/libuseful.so.1.0.0 $(DESTDIR)$(LIBDIR)/libuseful.so.1
	ln -s $(DESTDIR)$(LIBDIR)/libuseful.so.1 $(DESTDIR)$(LIBDIR)/libuseful.so

Note the $(DESTDIR) use.

First we copy all header files then the shared library. Here I use install(1), but a combination of mkdir(1), cp(1) and chmod(1) would create an equivalent result. I also make some specific-version symbolic links for the library. Variables to hold filenames and the version could be used. They are omitted to limit scope of the example.

What about Lua?

Even with standards, conventions, recommendations makefiles in the wild have a lot of variation. For Lua there are no conventions around, so we will see even more variation among the projects. I want to respect this variation when proposing an install target for a Lua project. Let's take a look.

I selected a set of projects available in arch package repository, took a look upstream, took a look at PKGBUILDs, searched some more through Github. From that I identified four groups of interest:

Some projects had both makefiles and rockspecs (using builtin build type). I did not try to dig out why, but I guess it was either: migration to rockspec, developer preference, or similar. Another scenario are projects that did not have their own rockspec got one written by a package maintainer for LuaRocks (since it's required) or distro-specific package repository (due to preference). These kind of rockspecs could get merged to the upstream project.

From projects with makefiles I selected:

How are these makefiles used when their parent project is packaged for different OS distributions? Well, each distribution has its own packaging processes. Let's take a quick look at two.

luarocks illusionary logo

Debian/Ubuntu in case of Lua packages will not use them. They have dh(1) and dh_lua(1) which together take care of the packaging process. We could try to take a look at the helper scripts, but at the time of writing they didn't appear helpful for this article.

Arch Linux and its PKGBUILDs for lua packages prefer to use LuaRocks to support package making. In some cases if a makefile from an upstream project was close enough to a conventional makefile, it could and was used. However, at least one package maintainer later on judged that it's simply easier to use LuaRocks for everything and even recommended writing rockspecs if not present upstream. Despite that there's a number of packages that simply copy things around since the upstream projects did not have makefiles, install targets or rockspecs.

I think this is enough information to make decisions. I also looked at some CMakeFiles and pkg-configs.

Let's set goals. Our makefiles and install targets must be:

Proposition

There's a shorter version better suited for closer inspection and copying below. 

LUA_VERSION=5.4

The first or second spot seems fitting for the version. Before build definitions, too.

I couldn't decide. Full name seems the best. I was thinking about LUAVERSION quite a lot since I was using it for my projects. In the end underscore is consistent with Lua's LUA_PATH variables. Shorter variants like LUAV seem to create an impression of a library or utility name (e.g., luvit/luv, guidanoli/luav).

Change your default value from 5.4 as necessary. 

PREFIX=/usr/local

The first spot rival. It lost because it wouldn't need to appear before build definitions.

It defines the base path for everything else. Do not mistake or mix it with DESTDIR. The default value rarely changes, to understand it better consult sources listed somewhere above. 

BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1

Not-Lua-specific install directories. Like before I skipped some since there's only a low chance of using them.

We want these for consistency and to allow users do more granular path injection, e.g., LIBDIR with a platform triplet. 

LUA_CMOD=$(LIBDIR)/lua/$(LUA_VERSION)
LUA_LMOD=$(DATADIR)/lua/$(LUA_VERSION)

These are the target locations respectively for shared library and script Lua modules.

These names slowly grow in me every time I review the article. They read nicely: "C language, so library, MOD-ules for LUA" and similarly "pure Lua" or "Lua, so pure". These are used mainly by pkg-config files (e.g., Arch Linux, homebrew, chromebrew) and sometimes in projects.

Another candidate for pure modules was LUADIR which is the most popular and recommended by LuaRocks but it is conflicting with the same name pointing at Lua include files (e.g., LPeg uses it like this). A candidate for library modules LUA_LIBDIR has the exact same problem (this one is even worse, because it reads as "LIBDIR for LUA).

One concern I still think about is the broken parallel between LUA_LMOD/LUA_CMOD and LUA_PATH/LUA_CPATH. 

LUA_CFLAGS=\
  `pkg-config \
  --cflags \
  lua$(LUA_VERSION)`
LUA_LDLIBS=\
  `pkg-config \
  --libs \
  lua$(LUA_VERSION)`

If you need to compile some modules you will need these. I'm lazy and prefer to use pkg-config(1). It nicely cooperates with the version variable allowing to meet the simple version switching goal.

Now, since pkg-config is used rather than just writing things by hand, a question appears. More than one actually, since a simple CFLAGS creates multiple questions about customization (some are answered by e.g., +=), defaults, optimization. However, I don't think this is the time to discuss them.

Here, I only wanted to point out that LUA_VERSION can and most likely should be reused when getting Lua versioned include directories.

Lines are broken for the sake of the column width, readability of this page and to save me from tweaking the style more than I need, since I can get lost in an endless loop of moving elements by one pixel in random directions. 

export LUA_PATH=\
	$(PWD)/src/?.lua;\
	$(PWD)/src/?/init.lua;;
export LUA_CPATH=;;

Occasionally, we may need to execute some Lua code as part of our makefile and use some of our code.

export is not yet part of POSIX, so workaround it with shell or env(1).

LUA_CPATH is an almost no-op for the sake of appearance. 

LUA=lua

Traditionally executed commands have their own variables (btw. they are also called macros). This adds just another opportunity to modify them since make already runs in some environment that can be tweaked. Still, it can be useful for e.g., injecting LUA_PATH mentioned above.

I had to break the two column view for the last part, because it would look comically. Installation target itself: 

install: all
	install -m755 -Dt $(DESTDIR)$(BINDIR) wrap
	install -m644 -Dt $(DESTDIR)$(MAN1DIR) wrap.1
	install -m755 -Dt $(DESTDIR)$(LUA_CMOD) useful.so
	install -m644 -Dt $(DESTDIR)$(LUA_LMOD)/wrap wrap/*.lua

This assumes that:

I adjusted two of my unfinished projects to use this convention, so that you can see a makefile without this kind of assumptions. See activity and srcinfo.

As an alternative to direct use of install(1) and to increase control over commands executed with make see Variables for Specifying Commands.

Finally, the short version of the makefile: 

LUA_VERSION=5.4

PREFIX=/usr/local
BINDIR=$(PREFIX)/bin
LIBDIR=$(PREFIX)/lib
DATADIR=$(PREFIX)/share
MANDIR=$(DATADIR)/man
MAN1DIR=$(MANDIR)/man1
LUA_CMOD=$(LIBDIR)/lua/$(LUA_VERSION)
LUA_LMOD=$(DATADIR)/lua/$(LUA_VERSION)

LUA_CFLAGS=`pkg-config --cflags lua$(LUA_VERSION)`
LUA_LDLIBS=`pkg-config --libs lua$(LUA_VERSION)`
CFLAGS=$(LUA_CFLAGS)
LDLIBS=$(LUA_LDLIBS)


all:
	touch wrap useful.so


install: all
	install -m755 -Dt $(DESTDIR)$(BINDIR) wrap
	install -m644 -Dt $(DESTDIR)$(MAN1DIR) wrap.1
	install -m755 -Dt $(DESTDIR)$(LUA_CMOD) useful.so
	install -m644 -Dt $(DESTDIR)$(LUA_LMOD)/wrap wrap/*.lua

I still think I would prefer them without the underscores, but then LUALMOD would be one letter away from LUAMOD which would be annoying to deal with since it sounds like it would hold a name of a module.

What do you think? What do you use yourself (other than LuaRocks)? Do you like the two column code view thing? Until I create some proper feedback channel, just feel free to send me an email!