Fastmake Documentation

Since the fastmake is going to be GNU make compatible it is not wise to duplicate all the documentation here. Therefore this document covers only fastmake-specific features. Basic functionality documentation is at GNU make doc page

Command line

--error-undefined-variables argument

GNU make has --warn-undefined-variables flag that warns if an undefined function is used. --error-undefined-variables throws error and exit in this case. This flag added to satisfy complaints about blind skipping of undefined identifiers in Make.

--scripting argument

Fastmake is being built by Fastmake itself but the source code of Fastmake is a tarball with GNU Make Makefile. It is to be able to compile Fastmake in an environment where Fastmake is not yet available. Makefile is actualy trivial. It just calls bash shell script with commands that compile and link sources. But how to provide such a script? The answer is in --scripting command line argument. When it is specified fastmake logs into specified file all the external recipe commands it runs. Before packing a tarball fastmake is built by itself with --scripting argument that generates a batch with all commands needed to build fastmake. Of cause this batch will work only in proper environment (bash, gcc, cd commands should present).

--no-startup argument

Tells make not to look up for startup file.

--preserve-temporary argument

Tells fastmake not to remove any temporary files after execution. This is useful to analize canned shell recipes if they fail.

New functions

Fastmake has new useful functions. See function reference for more information.

Syntax enhancements

Initially fastmake came from dmake, not GNU make and inherited dmake syntax mainly. Now it is being changed towards GNU make to be GNU make compatible. In the future it will have the best features of both utilities and possibly other utilities (like makepp, cmake, omake, scons, ...). If you didn't find something in this doc you can check GNU make documentation. However fastmake has many own features like .COMBINE,.CONTENT modifiers, .DATECACHE directive, $(make ) function, .FOREACH statement to fasten compilation and improve makefiles. Some features are conflicting. In this case GNU make feature are preferable. For example taking the first matching pattern rule even if several pattern rules match is rather GNU make feature not dmake's.

C-style include

In GNU make the only possible inclusion is the inclusion from current directory while in C language files are included from the directory relative to file from which inclusion takes place. In Fastmake it can be accomplished by placing included file in corner brackets:

include <makedefs.mak> # search for makedefs.mak near this module
include makedefs.mak # search for makedefs.mak in current directory

Function 'make'

The key thing of the fastmake is the 'make' function:

$(make <command line>)

It takes one argument - command line. This function invokes new make procedure as if it is a separate process. Actually the process is the same. It allows to share cache data between calls to fasten execution.

Explicit rules

<target> [modifiers] : [prerequisite, ...] ; [@][command]
or
<target> [modifiers] : [prerequisite, ...]
    [@][command]
    ...
    [@][command]

The main fastmake difference is that it does not require TAB symbol before commands. This is convenient because in many editors the difference between TAB and spaces hardly distinguished. The second feature is the possibility to specify modifiers after target before ':' rather than separately.

The table describes rule modifiers

.PHONY	Tells that target is not a file or directory
.IGNORE	Ignore recipe failure
.CARELESS	Ignore prerequisite failure
.UNRELIABLE	Run recipe if some prerequisite failed (useful for generated dependence rules)
.NOSHELL	Don't use shell to invoke the commands if shell is used by default
.SHELL	Use shell to invoke the commands if shell is not used by default
.SILENT	Don't echo commands to the stdout
.COMBINE	Used for passing multiple files to compiler (batched compilation)
.PARALLEL	Gather such rules to run their recipes in parallel (e.g. parallel compilation)
.CONTENT	Tells to compare file content not dates

Pattern rules

Pattern rule is a rule that describes how to make a class of targets. For example:

%.obj : %.cpp ; @cl /c $< -Fo$@
%.obj : %.c ; @cl /c $< -Fo$@

says how to make all .obj files. When fastmake searches for target to make first it looks for explicit rule then for the first pattern rule that matches the target. Then it tries to make all the prerequisites. If it fails it takes another pattern rule and so on. In the example above fastmake will try to find .c file if .cpp does not exist.

.COMBINE modifier

Suppose we have the rules

%.obj : %.cpp ; @cl /c $< -Fo$@
some.lib : a.obj b.obj c.obj ; link -OUT:$@ $<

In normal situation 4 commands will be invoked to make some.lib:

cl /c a.cpp -Foa.obj
cl /c b.cpp -Fob.obj
cl /c c.cpp -Foc.obj
link -OUT:some.lib a.obj b.obj c.obj

But in case of MSVC it is more effective to pass to compiler all .cpp files simultaneously. Especially on multicore processors where compiler can use all the cores to compile passed source files:

cl /c a.cpp b.cpp c.cpp -Fo$(BLDSUBDIR)
link -OUT:some.lib a.obj b.obj c.obj

To achieve this .COMBINE modifier can be used:

%.obj .COMBINE : %.cpp ; @cl /c $< -Fo$@
some.lib : a.obj b.obj c.obj ; link -OUT:$@ $<

In this case prerequisites and targets of the combined pattern rule are gathered accordingly parent rule prerequisites and then recipe is invoked for all gathered prerequisites and targets.

.PARALLEL modifier

Though GNU make has the possibility to process independent targets in parallel it does not have the possibility to specify concrete targets that should be parallelized. Fastmake can be told to parallelize a concrete targets. Suppose we have the rules:

%.o : %.cpp ; @gcc -c $< -o$@
some.a : a.obj b.obj c.obj ; ar -o$@ $<

In normal situation commands will be invoked to make three .o files one after another:

gcc -c a.cpp -o a.o
gcc -c b.cpp -o b.o
gcc -c c.cpp -o c.o

To make them run in parallel rule modifier .PARALLEL could be used:

%.o .PARALLEL : %.cpp ; @gcc -c $< -o$@

By default there are two parallel threads. Number of threads can be specified in -j command line argument.

.CONTENT modifier

By default file modification date is used to determine that target should be updated. If modifier .CONTENT fastmake compares file content. In example:

new_version.cpp .CONTENT : version.cpp ; cp $< $@

version.cpp is copied to new_version.cpp if these files differ

.UNRELIABLE modifier

In a normal situation a recipe is run if a target is not up to date or is not found. If some of prerequisites failed then a target failed too. But there are situations where prerequisite failure means that prerequisite structure changed and target should be updated. This is achieved with .UNRELIABLE modifier. Prerequisite failure is ignored in this case. The example is the generated dependence rule. Suppose we have generated dependence file:

a.o : a.c a.h b.h c.h

If one of the a.c a.h b.h c.h changes then a.o recompiles - this is normal. But if the programmer decides to remove one of the files then the error will be thrown because the deleted file is still in the prerequisite list. Correct behavior is to treat a.o is not up to date and recompile it. This is achieved by specifying .UNRELIABLE modifier:

a.o .UNRELIABLE : a.c a.h b.h c.h

The dependence generator integrated in fastmake ($(makedep) function) generates UNRELIABLE dependence rules which can be included by include_dep operator.

Richer Assignments

GNU make has just = ?= := += assignment types. Fastmake has the following exhaustive set of assignments:

[:][?|+]=

That is it have: = += ?= := :+= :?= operators. As you see missing :+= and :?= added. Fastmake processes simple expantion (:) and deferred expantion different from GNU make. GNU make binds an expansion type to variable while Fastmake does not. In Fastmake expansion type means just how to expand right expression - now or later by demand.

Special targets

Special targets look like explicit dependences, but this is just syntax likeness. Special targets affect internal fastmake services. The syntax of a directive:

.<NAME> [modifiers] : <params>

The directives are:

.DATACACHE	adds data cache
.INCLUDE	includes another makefile
.IMPORT	imports environment variable

.IMPORT directive

By default environment variables are not turned into makefile variables. This is done for performance reasons. To import environment variable from the outside world .IMPORT directive should be used:

.IMPORT : variable-name

Conditions

Fastmake introduces new condition syntax. This is because GNU make conditions (ifeq, ifneq) are limited to equality checking. The syntax of condition:

.IF <logical expression>
    statements
[.ELIF <logical expression>]
    statements
[.ELSE]
    statements
.END

where logical expression is the result of logical operations:

==, !=, &&, ||

A macro can be used as logical expression. In this case it is TRUE if a variable has been initialized to non-empty value. In the following example statements are executed if PCH macro has been defined.

.IF $(PCH)
    statements
.END

To check if a variable has been initialized to eigher empty or non-empty value use ifdef directive.

Logical expressions are evaluated in-place. For example the output of this code will be 1, not 2

A = 1
.IF $(A) == 1
all :; @echo 1
.ELIF $(A) == 2
all :; @echo 2
.END
A = 2

To have a condition that evaluates on each usage $(if ) function can be used.

.FOREACH statement

Fastmake has the possibility to generate assignments or rules using cycle. .FOREACH statement can be used for this purpose. For example the code

.FOREACH x .IN c cpp
%.o : %.$(x) ; cc -c -o $@ $<
.END

is the equivalent of

%.o : %.c ; cc -c -o $@ $<
%.o : %.cpp ; cc -c -o $@ $<

Comments

Fastmake has block comments. All code enclosed in /* ... */ is skipped.

Infix operators

Infix operator is a small modifier function that can be applied to variable or function. GNU make has very limited infix functionality. It supports several modifications such as 'F' and 'D' that can only be applied to automatic variables such as $*, $^ and so on. Fastmake has extended set of modifications that can be applied to any variable or function not only automatic variable. Infix can be with a parameter or without a parameter. All infix operators starts with ':' symbol.

Infix operators without parameter are 'f','d','b','db','u','l'. For example, if:

VAR=dir/file.obj

then
$(VAR:f) is file.obj
$(VAR:d) is dir
$(VAR:b) is file
$(VAR:db) is dir/file
$(VAR:u) is DIR/FILE
$(VAR:l) is dir/file # to lower case

Infix that is applied to autovalues causes brackets. For example this rule creates the directory of the target:

%.obj : %.cpp ; @mkdir -p $(@:d)

The infixes with a parameter are '^','+','<','-'. For example if:

VAR = build/a.o build/b.o build/c.o
DIR = dir
SUF = .dep

then
$(VAR:^$(DIR)/) is dir/build/a.o dir/build/b.o dir/build/c.o
$(VAR:+$(SUF)) is build/a.o.dep build/b.o.dep build/c.o.dep
$(VAR:<build/) is a.o b.o c.o
$(VAR:-.o) is build/a build/b build/c

Applying infixes to functions look like this:

$(ls:b .,f) - returns all files in a current directory without extensions.

Infixes can be combined:

$(ls:b:^dir/ .,f) 0 returns all files in a current directory without extensions and with dir/ prefixes.

Determining C++ dependences

Classic make proposes to use gcc -M command to generate C++ dependencies. This is not enough because there are many other compilers beside gcc. For example msvc does not have such a possibility. It is also very slow and ineffective. First is that gcc is invoked on each source file. The second is that the results are not reused from one invokation to another. Fastmake has internal C++ dependence finder. All dependences are generated within one process and common header results are reused between source files. Function 'makedep' is served for this purpose:

$(makedep source_files, dep_files, out_files, incpaths, incfiles, enclosure)

where

• source_files - source files to process
• dep_files - corresponding dependence names to be written in outer dependence file
• out_files - corresponding output files
• incpaths - search 'include' paths
• incfiles - additional included header files
• enclosure - enclosure symbols for included files

This function is usualy called after invoking a compiler. The reason of passing arrays of parameters possibility is COMBINE compilation mode (MSVC) where many source files are passed to compiler together. Note that the size of source_files, dep_files, out_files arrays should be the same. Subsequent calls to makedep will reuse results of previous calls. For example if two source files include one header file then the dependences of that header file are determined once.

There is another useful thing in 'makedep'. It allows to specify enclosure symbols for included file to be processed. For example to process the only files that inclosed in quotes you need to pass \"\" as a enclosure parameter. To process both quoted and corner bracketed files specify <> \"\"

The function creates files of the format of makefile dependency, where every word is a full path to some file:

dep_file : dependence1 .. dependenceN

This file should then be included by include_dep directive in makefile. The difference of include_dep from regular include directive is that it is much faster (simple parsing) and adds .UNRELIABLE modifier to this dependency.

Keeping small dependency files in the single file

Dependency files are small and there are usualy a lot of them - one for each source file. To access them one by one is inefficient. Fastmake has integrated engine that implements file system within the single file. To initialize this engine Fastmake should be told a file location where to store filesystem. It is implemented by calling the directive:

.nFS : file_path /nFS/

where file_path is the desirable location of file system that usualy resides in binary directory. /nFS/ is a mount point. Since this directive is called all output files that are specified in $(makedep ) function will be in the single file where mount point points.

Running fastmake without Makefile

It is often annoying to create a Makefile in each directory. It can be avoided by placing startup.mak some where in a root directory for which fastmake looks up. Placing the following directive in startup.mak:

.NOMAKEFILE : yes

tells fastmake not to read regular Makefile and take all the rules from startup.mak only

'lookup' directive

This directive works like 'include' directive but it looks for a file in upper directories. For example

include ../../makedefs.mak

can be replaced by a statement

lookup makedefs.mak

because 'makedefs.mak' is located two directories higher. This directive is more convenient. It allows not to remember exact location of common included makefiles located in upper directories. In conjunction with lookup.mak it allows to avoid having some makefiles at all.

'lookhup' directive

It stands for look here and up. It looks for a file in a current directory and then in upper directories.

lookup.mak file

This file allows to avoid some equal makefiles. Simply if you run fastmake in a directory that does not contain makefile then fastmake searches for lookup.mak in upper directories. If you have more than one directory with makefiles with the same content you can replace them with one lookup.mak file in the upper directory. For example if you have the following directory structure:

dir1/dir2/makefile.mak
dir1/dir3/makefile.mak

and makefiles in 'dir2','dir3' are the same. It can be replaced by:

dir1/lookup.mak

Operator .CD

This operator can be used inside a recipe to create a block of commands to be run in the other working directory. The following is the example with the operator.

SHELL = bash
SHELLFLAGS = -c
all .PHONY:
    @echo working is `pwd`
    .CD other_dir
        @echo Now working dir is `pwd`
        @echo Now working dir is `pwd`
    .END
    @echo returned to dir `pwd`

First of all operator .CD is faster then 'cd' command. Second 'cd' command works just for one command line. Third operator .CD creates a scope of independent command lines. If some command fails then recipe finishes executing and previous working directory is restored.

Date Cache

There are files in the build process that are created and modified by tools that are invoked by fastmake. Such files are .obj,.o,.exe,.dll,.class and so on. Their difference from source files is that fastmake knows when modification time of such files is changed. For this reason fastmake can cache modification dates for that files to fasten build process and reduce file system calls number significantly. To create date cache for example for .obj files do the following:

.DATECACHE : $(SOMEWHERE_IN_YOU_BUILD_PATH)/fastmake.DateCache $(SOMEWHERE_IN_YOU_OBJS_PATH)%.obj

This statement tells fastmake to store date cache in fastmake.DateCache file and allows to cache all .obj files that are located in $(SOMEWHERE_IN_YOU_OBJS_PATH) directory.

Makefile optimization

Makefiles should be optimized properly. Only in this case fast passes are guaranteed. The following topics cover makefile optimization techniques.

.PHONY targets

It is preferable to use .PHONY target modifier for targets that are not files or directories. For example targets all, clean, dep are phony. Fastmake does not check if file or directory of the target name exists and therefore does not call for file system which significantly increases performance.

startup.mak file

Normally fastmake goes through directories and in each directory it reads configuration makefiles. These are makedefs.mak and makeruls.mak in the sample source tree. But there are some settings that are invariant for all directories. For example paths to sdk, compiler command lines and so on. There is no need to read and parse such settings in each directory. It is much more efficient to read them once. For this purpose startup.mak file is used. Fastmake searches for it up through directory tree. You don't need to specify its location in command line. Just place it in a root directory of your sources.

Pattern rule order

Pattern rules are matched in the order they stated in makefile. There are some rules that are alternative. For example to specify rules for .obj files two rules are used:

%.obj : %.cpp ; gcc ...
%.obj : %.c   ; g++ ...

To build a.obj, first fastmake checks if a.cpp exists then if a.c exists. So for .c files there is one useless check. Therefore the peace of code above is more suitable for C++ projects. For C projects %.obj : %.c rule should be the first. In other words pattern rules for most frequent files should be first.

Functions instead of shell commands

Use $(cp ), $(mv ), $(ln ), ... functions instead of corresponding shell commands. It greatly improves performance.

Date cache

Use .DATECACHE directive to tell fastmake to cache dates for compiler generated files (like .obj .o files).

Integrated dependency scaner stored in one file

Use integrated dependency scaner. Store files in /nFS file system that is one file. Include dependency by include_dep directive which is optimized to read dependency.

.NOSHELL rule modifier

if SHELL variable is initialized then each recipe invokes $(SHELL) to run a recipe. This means that 2 processes are spawned - shell and recipe. If you do not need shell that is your recipe is just text of command that does not have shell syntax then specify .NOSHELL modifier to run you recipe directly

.obj .NOSHELL : .cpp ; cl -c $< -Fo$@

Include subsets of variables/rules localy

Try to separate rare used rules to corresponding makefiles and include them localy where they are needed. For example if you have bison/flex parser generator folders then separate rules which use bison/flex to bison.mak file and include it only in that folder makefile where it is used. You can find this technique in Fastmake source package.