Makefile Recipes

Collection of Makefile recipes, in especially for linting, testing, and documentation of projects with a Python or C/C++ pipeline.

While almost every recent ecosystem comes with its own build tools, their basic functionality often resembles the one of make – which is widely available, language-agnostic, and easy to use/manage without additional complexity. Also, pipeline definitions of the most common code management platforms are largely incompatible. A make check test as common ground allows to locally reproduce QA pipeline runs in advance and keeps the different pipeline configurations minimal.

This collection of assorted Makefile recipes shows examples that can be useful for bootstrapping new projects, in especially for linting and testing support in a Python or C/C++ pipeline.

For more information on C/C++ performance benchmarking and optimization see also the GCC Profiling and Coverage project.
Note that tab indentation should be ensured after copy/paste, as required by the Makefile syntax.

Python Virtual Environment

A virtual environment in Python allows to install dependencies locally – i.e., decoupled from global or per-user packages. For example during development, when multiple Python versions are intalled, for fixed versions that could conflict, or for non-runtime testing/linting pipeline requirements, this can help to keep the state clean and local.

Given Python sources to run below my_module/, this will install all dependencies from the requirements file requirements.txt into a local virtual environment:

PYTHON ?= python3
VENV ?= venv
ACTIVATE = $(VENV)/bin/activate
NAME ?= my_module

.PHONY: run
run: $(ACTIVATE)
    . $(ACTIVATE) ; $(PYTHON) -m $(NAME)

$(ACTIVATE): requirements.txt
    $(PYTHON) -m venv $(VENV)
    . $(ACTIVATE) ; pip install -U pip setuptools wheel
    . $(ACTIVATE) ; pip install -U -r $(<)
    @touch $(@)

.PHONY: clean
clean:
    rm -rf $(VENV)

No re-installation is done as long as requirements.txt does not change or the venv directory is cleaned up.

Python Linting Pipeline

This basic Python QA pipeline consists of mypy for type checking, flake8 for linting, and bandit for security checking. The corresponding packages should be stated as (development-time only) dependencies in the requirements.txt or be otherwise installed in the virtual environment (see above).

.PHONY: check
check: $(ACTIVATE)
    . $(ACTIVATE) ; mypy -- $(NAME)
    . $(ACTIVATE) ; flake8 -- $(NAME)
    . $(ACTIVATE) ; bandit -r --quiet -- $(NAME)

.PHONY: clean
clean:
    rm -rf .mypy_cache

Python Unittest Coverage

Running tests/test_*.py files with the standard unittest framework is equally simple.

.PHONY: test
test: $(ACTIVATE)
    . $(ACTIVATE) ; $(PYTHON) -m unittest discover -v -c -t ./ -s ./tests/ -p 'test_*.py'

By additionally wrapping tests with the coverage package, statement (and optionally per-branch) reports can be printed or exported.

.PHONY: test
test: $(ACTIVATE)
    . $(ACTIVATE) ; $(PYTHON) -m coverage run -m unittest discover -v -c -t ./ -s ./tests/ -p 'test_*.py'
    . $(ACTIVATE) ; $(PYTHON) -m coverage report --skip-empty
    @rm -f .coverage

Clean pycache Directories

Using -depth when recursively deleting directories with find prevents the “readdir race” error.

.PHONY: clean
clean:
    find . -depth -type d -name '__pycache__' -exec rm -rf {} \;

GoogleTest

Screenshot: Example task list view via CLI

GoogleTest is a simple to use yet powerful testing framework for C++. The general approach is to link the object files of the actual program, the tests, and googletest together. (Note that in order to use the test main, the objects should not also include one.)

As by now there are packages available for several common Linux distribution images, using googletest should also be supported by most QA pipeline runners with little additional effort.

The (former) sample GoogleTest Makefile gives a first introduction on how to build. Given a project layout with src/ and tests/, a more complete Makefile for a basic setup could look like:

GTEST_DIR    ?= /usr/src/googletest/googletest
GTEST_INCLUDE = $(GTEST_DIR) $(GTEST_DIR)/include
GTEST_HEADERS = $(wildcard $(GTEST_DIR)/include/gtest/*) $(wildcard $(GTEST_DIR)/include/gtest/internal/*) $(wildcard $(GTEST_DIR)/src/*)

GTEST_OBJECTS = tests/gtest_main.o tests/gtest-all.o
TEST_OBJECTS  = $(patsubst %.cpp,%.o,$(wildcard tests/*.cpp))
OBJECTS       = $(patsubst %.cpp,%.o,$(wildcard src/*.cpp))

.PHONY: test
test: test.out
    @# valgrind --quiet --leak-check=full --show-reachable=yes --track-origins=yes \
    ./$(<) --gtest_shuffle

test.out: $(GTEST_OBJECTS) $(TEST_OBJECTS) $(filter-out %/main.o,$(OBJECTS))
    $(CC) -o $(@) $(^) $(LFLAGS)

src/%.o: src/%.cpp $(wildcard src/*.hpp)
    $(CC) $(CFLAGS) -o $(@) -c $(<)

tests/%.o: tests/%.cpp $(wildcard tests/*.hpp) $(wildcard src/*.hpp) $(GTEST_HEADERS)
    $(CC) $(CFLAGS) -Isrc/ $(addprefix -I,$(GTEST_INCLUDE)) -o $(@) -c $(<)

tests/gtest%.o: $(GTEST_DIR)/src/gtest%.cc $(GTEST_HEADERS)
    $(CC) $(CFLAGS) -DGTEST_HAS_PTHREAD=0 $(addprefix -I,$(GTEST_INCLUDE)) -o $(@) -c $(<)

.PHONY: clean
clean:
    rm -f test.out $(OBJECTS) $(TEST_OBJECTS) $(GTEST_OBJECTS)

Self-signed Certificate

For testing purposes, a self-signed wildcard certificate such as localhost.pem can be created with a single openssl commandline and simple make target.

%.pem %-key.pem:
    openssl req -x509 \
        -newkey rsa:4096 -sha256 -nodes -keyout $(*)-key.pem \
        -days 365 -subj "/CN=$(*)" -addext "subjectAltName=DNS:$(*),DNS:*.$(*),IP:127.0.0.1" -out $(*).pem

Clang-Tidy Cache

Using the clang-tidy coding convention linter can become tedious due to its poor performance. By writing fixes to a file, the results can be “cached”, though – only changes to the respective source-file (or any header) will trigger the rule again. In addition, make -j can be used to parallelize the whole operation. Checks can be configured in greater detail via the .clang-tidy YAML file.

SOURCES = $(wildcard src/*.cpp)
HEADERS = $(wildcard src/*.hpp)
CFLAGS ?=

.PHONY: check
check: $(addsuffix .tidy.yml~,$(SOURCES) $(HEADERS))
%.tidy.yml~: % $(HEADERS)
    @touch $(@) && truncate -s0 $(@)
    clang-tidy --quiet --warnings-as-errors='*' --export-fixes $(@) $(<) -- $(CFLAGS)

On the other hand, the clang-format style linter can usually be invoked directly. The coding style to apply is then configurable via the .clang-format YAML file.

.PHONY: lint
lint: $(SOURCES) $(HEADERS)
    clang-format --dry-run -Werror -style=file -- $(^)

Clang-Tidy and Clang-Format Example

For small projects, a good starting point can be the following clang-tidy and clang-format commands as make check target:

.PHONY: check
check: $(wildcard *.c) $(wildcard *.cpp) $(wildcard *.h) $(wildcard *.hpp)
    clang-tidy --quiet --warnings-as-errors='*' --checks='cert-*,bugprone-*,clang-analyzer-*,misc-*' $(^) -- $(CFLAGS) $(CPPFLAGS) $(CXXFLAGS)
    clang-format --dry-run -Werror -style='{BasedOnStyle: Google, IndentWidth: 4, TabWidth: 4, ColumnLimit: 120, MaxEmptyLinesToKeep: 2, SpacesBeforeTrailingComments: 1}' -- $(^)

No additional files or configuration is involved.

Pipeline apt Dependencies

Maintaining a list of software dependencies in a separate file is common practice for several reasons. While apt-get does not support reading directly from a trivial file such as requirements.apt.txt, this can easily be done using xargs.

.PHONY: deps
deps: requirements.apt.txt
    sudo -n apt-get update
    xargs -r -a $(<) -- sudo -n apt-get --yes --no-install-recommends --show-progress install

Sticking versions is then also supported as usual, by the standard install commandline syntax. The same approach can also be viable for example in pipeline definitions or Dockerfiles.

Doxygen Commandline Parameters

Screenshot: Example task list view via CLI

By default, Doxygen reads its configuration from a Doxyfile that needs to be maintained according to the current project’s preferences. While it does not support commandline parameters, all statements can alternatively be read from standard input in a whole – which can quickly become tedious.

Using the @INCLUDE tag allows to separate “dynamic” parameters from more verbose “static” configuration. As the include is done first, the explicit assignments possibly override the defaults read from file.

The below example uses Makefile variables and the Awesome Doxygen CSS Theme as local configuration to augment the global Doxyfile.

SRCDIR = src
NAME = my-project

doxygen-awesome.css:
    wget -nv -O $(@) https://raw.githubusercontent.com/jothepro/doxygen-awesome-css/main/doxygen-awesome.css

.PHONY: docs
docs: docs/index.html
docs/index.html: $(wildcard $(SRCDIR)/*) Doxyfile doxygen-awesome.css
    @mkdir -p $(dir $(@))
    echo "@INCLUDE = Doxyfile\nPROJECT_NAME = $(NAME)\nINPUT = $(SRCDIR)/\nOUTPUT_DIRECTORY = $(dir $(@))\nHTML_EXTRA_STYLESHEET = doxygen-awesome.css" | \
    doxygen -

.PHONY: clean
clean:
    @rm -rf -- doxygen-awesome.css docs