Organizing Your Lua Project
Published on 2021-01-07 15:45:00+01:00
From time to time I hear complaints about how Lua handles modules. Here and there I see and even answer myself
require and adjusting the paths in
package to allow some desired
behaviour, with the most prominent issue of relative imports that always work.
Before we hop into the explanation of how to organize files in your Lua projects, let's talk about default importing
mechanism in Lua:
require handles paths
require are surprisingly interesting tools. At first glance they are
simple. When you look into them, they are still understandable while gaining some complexity that doesn't reach
unnecessary extremes. They are elegant.
They use a mechanism to find desired files called path resolution or usually simply path. The main
component of is a sequence of patterns that may become a pathname, e.g.:
What does it tell us? First off,
? is going to be replaced by the argument that was provided to the
require. All dots will be replaced by an appropriate path separator so that:
a/b/c in *nix systems. So, for call like
require "a.b.c", out path will look
Now, each of these paths are tried and the first one that actually exists in the system will be used. If none of them
match an existing file, the import fails. Simple as that.
The path that is used in resolution is set in
package.path. You can modify it in Lua, but it is
intrusive and may depend on a single entry point. Generally, if you plan to release your project as a module for people
to use, I encourage you to avoid modifying anything global. And that's global. Anyway,
appear out of nowhere - it is populated by one of:
- Environmental variable
x_x is version such as
- Environmental variable
- Default as defined in luaconf.h
Interestingly, if two separators
;; show up in the environmental variable path, they will be replaced by
the default path. Meaning
/path/to/project/?.lua;; works as prepending your custom path to the default one.
Of course, there is way more to it than just this i.a.: requiring modules written in C, searchers or preloads.
However, in our case this knowledge will suffice.
If you are curious how exactly path is loaded be sure to check out
To prepare for development, we need to know where we are heading. First step is to consider the execution
environment. Of course, this and packaging are journeys on their own, so let's just look at two common examples: an
application that uses some framework that uses Lua (e.g. a game made in LÖVE) and a
standalone module for others to use.
In the first case, it's the duty of the framework to configure the path properly and inform you through the
documentation about it. Paths in LÖVE use their own file hierarchy that is managed by
by default contains both the game's source (directory or the mounted
.love archive) and the save directory.
This means that the structure in your source files is directly reflected in the calls to
require, so that
require "module.submodule" will always try
game/module/submodule.lua, no matter how you run
the game. This case usually doesn't involve any additional environment configuration for the development stage.
In the second case, your project will end up in an already configured environment and will need to fit in. The
installation of the package usually involves copying your files to the directory that is already included in the path,
so that no further configuration is needed for the execution (at least regarding the path). You can assume, that the
successful installation will make your modules available in the way you want them.
This doesn't happen in the development stage, when you rarely install your package, and most certainly you don't
install it each time you want to test it. This means, that you need to adjust the path so that your modules appear in it
as if they were installed in the system. The principle of minimizing the intrusiveness remains, so the best option is to
use the environmental variables to prepare for development. If you run your application or any tool in such environment,
then Lua will have access to your modules no matter where it is run. Additionally, it will be consistent with the
target environment and won't need any additional hacks.
All this talk comes down to: set
LUA_PATH in your development environment so that it includes your
project files even if they are not installed in system. A simple approach is to source following in each session:
Note the double semicolon that will get replaced by default path, so that other modules that are already installed
are also available.
Let's try it out:
$ source env.sh
$ find .
$ cd modulea
> require "moduleb"
> require "modulea"
> require "modulea.submodule"
As you can see, despite being in the subdirectory, you can still use modules with their fully qualified names that
will remain the same once you install the package. Note, that you could
require "init" or
"submodule" in this case, but I strongly recommend against it. Remain specific, follow the rules and pretend that
you use an installed package from an unknown working directory. Don't depend on current working
directory as it is not always the same. Using full names that consider the path setup guarantees
Organizing your files
Finally, this is what we're waiting for. Assume you have a directory that is a parent of all of your project files.
We'll call it a project root. Usually, this is also root directory for your version control system, be
it git or anything else, and for other tools such as building systems or even entire IDEs.
Because it is such a central place to the project, I usually just go ahead and prepend it to
the very same way as in the section above:
Just like previously, any Lua file that will be descendent of the root will be accessible to us through
require. But what is that init.lua?
It's there to create a way to improve hierarchical structure of your project - to allow splitting bigger modules into
smaller parts (or even submodules that could be included on their own), so that the module doesn't grow into a single
millions-lines-long file. In simpler words: you can create a directory named after module and put
init.lua file there and it will act just like a sole module.lua in root.
You could also create a directory named after module and module.lua file in root at the same time, but this
way you will have two entries per module in the root instead of just one.
Additionally, you can then put any module-related files into that directory. You can also use init.lua as
a simple wrapper that calls
require for each of its submodules and returns a table with them.
Consider a verbose example:
$ find .
$ cat wave/init.lua
-- This is a wrapper example.
sawtooth = require "wave.sawtooth",
sine = require "wave.sine",
square = require "wave.square",
$ cat persistence/init.lua
-- This is a normal module example.
$ cat persistance/tests.lua
-- This is a script that tests an example module.
local p = require "persistence"
assert(type(p) == "table")
$ cat main.lua
-- This is an example main of love application.
local persistence = require "persistence"
local wave = require "wave"
$ cat version.lua
-- This is an example module that acts as version string of the application.
Now, this is a mash-up of everything we've discussed. Despite it pretending to be LÖVE application it has
env.sh. Why? The reason is simple: the persistence and wave modules are not meant to be distributed
alone, and they won't ever appear in path of any other environment than LÖVE's. But LÖVE is not the only execution
environment in here: persistence/tests.lua is also meant to be executed. Possibly alone through Lua interpreter.
To allow it env.sh is present and used.
Let's have another example of a simple module meant for installation:
$ find .
$ cat Makefile
@echo Nothing to be done
cp -r hello $(PREFIX)
rm -fd $(PREFIX)/hello/* $(PREFIX)/hello
As you can see Makefile in this example has targets for installation and removal of the package. The structure
again is simple. Root works as part of the resolution path and so our module is placed in it's own directory named after
The last example is a project of a single file module:
$ find .
Yes, it's that simple.
Now, having env.sh in every single project might get bothersome, so I usually use a shell function for
managing them, similarly to what Python's venv does or LuaRocks' env. Speaking of,
LuaRocks is yet another interesting story to be told.
- Consider the execution environment when preparing the development environment.
- Contain your project. Avoid changing globals if the modules may be used by someone else in an unknown
- Never depend on the current working directory being something specific.
- While developing include your project's root in
LUA_PATH to influence
- Split modules into parts and use init.lua which is available by default to enclose the module into a
This is just one of the ways to handle structuring your Lua project. It's based on simple rules but has broad usage.
One tempting alternative is this little snippet:
local parents = (...):match "(.-)[^%.]+$"
require(parents .. "sibling")
Another already mentioned alternatives is adjusting
package.path directly in Lua. However, I decided to
skip it due to it's intrusiveness.
All in all, Lua is extremely customizable and adjustable. I would be surprised if these three would be the only ways
to organize projects in Lua.