The simplebuild.cfg file

Each simplebuild bundle of packages is defined by a single simplebuild.cfg file, usually located at the root directory under which all the packages are located. It serves a few different purposes:

When invoked, the sb command will look first in the current directory, then recursively in the parent directories for such a file - which is then taken as the actual main cfg file associated with that invocation of sb. This is for instance analogous to the way most Git-users are used to working (the git commands searches the current and parent directories for a .git/ directory), and allows for instance a mode of operation where simplebuild users can simply clone a code repository and then step into it and invoke the sb command to start compiling the simplebuild packages in that repository (assuming for instance that the repository contains a simplebuild.cfg package at its root).
It specifies dependencies on other simplebuild package bundles, which will cause simplebuild to search for these bundles and include their packages.
It can be used to fine-tune the build process, e.g. to enable debug builds, specify the cache directory location, or to only build certain packages, which might be useful in larger projects.

Automatic creation

If you are creating a new simplebuild bundle for a new project, you can get easily started by simply invoking sb --init [optional flags] in the directory under which you wish to keep your package bundle. As explained in the dedicated documentation for sb --init, this will generate a simplebuild.cfg file for you. Below here is repeated the file which is generated by sb --init without any other arguments. As you can see, it contains a lot of comments with explanation and examples, which is convenient for when you will be editing the file later:

Click to show simplebuild.cfg generated by sb --init

# ; -*-conf-toml-*-

# This file was originally autogenerated by 'sb --init'. It was generated with a
# lot of comments and commented out examples. The intention is of course to make
# it easy to quickly set up your configuration by simply editing this file as
# needed. Feel free to either remove the comments or simply leave them for later
# reference. For reference you can find more information about simplebuild.cfg
# files on https://mctools.github.io/simplebuild/simplebuilddotcfg.html .
#
# A note about paths in this file: Any paths specified can either use absolute
# paths, or paths relative to the directory of this cfg-file. For instance, a
# path like './mypkgs' indicates a subdirectory called 'mypkgs' in the same
# directory as the cfg-file, and a path like '../otherrepo' indicates a
# directory called 'otherrepo' located next to the directory of this cfg-file.

[bundle]
  # If you wish to let other bundles use your bundle of packages, you must give
  # it a name (so they can specify it in their list "bundles" list in the
  # [depend] section below):
  #
  #   name = 'mybundlename'
  #
  # If the directory below which you keep your simplebuild packages (i.e. "the
  # package root") is not meant to be the same directory as the directory
  # containing this cfg-file, you can modify it here. If for instance, all the
  # packages are below a subdirectory called 'mypkgs', you could specify:
  #
  #   pkg_root = './mypkgs' #default is '.'
  #

[depend]
  # Here you can give a list of names of any other package bundles we depend
  # on. You will always depend implicitly on the 'core' bundle (which provides
  # just a single package: 'Core'). You could add also 'core_val' to get a few
  # more unit tests, to verify your system:
  #
  #   bundles = ['core_val']
  #
  # If for instance you have installed simple-build-dgcode, you should have two
  # more bundles available: 'dgcode' and 'dgcode_val', where the latter contains
  # unit tests for the former. So you could add for instance:
  #
  #   bundles = ['dgcode']
  #   bundles = ['dgcode_val']
  #   bundles = ['dgcode','dgcode_val']
  #   bundles = ['core_val','dgcode','dgcode_val']
  #
  # The last three of those are actually equivalent, since the dgcode_val bundle
  # itself specifies a dependency on the 'dgcode' and 'core_val' bundles.
  #
  # If working with bundles like core/core_val/dgcode/dgcode_val, they will most
  # likely have been installed as a pip or conda package, and simplebuild will
  # be able to locate them automatically. But sometimes you wish to depend on
  # other bundles of packages in local directories. Perhaps you have a custom
  # repo with a simplebuild bundle called 'myutils', cloned in
  # '../../myrepos/myutils' which you wish to use. In that case, you must add
  # 'myutils' to the bundles=[..] list above, and then setup your search path
  # by the following line:
  #
  #   search_path = [ '../../myrepos/myutils' ]

[build]
  # By default, the cache directory used for build output will be
  # './simplebuild_cache'. If for instance you would like everything to go into
  # /tmp/myname/sbcache you could use:
  #
  #   cachedir = '/scr/myname/mysbcache'
  #
  # If you want to add many more sanity checks to your code (to help debugging
  # code CORRECTNESS, at the cost of runtime performance), you can use:
  #
  #   mode = 'debug'
  #
  # If you wish to just enable debug-symbols in your build, but not otherwise
  # modify compilation options (to help debugging code SPEED), you can use:
  #
  #   mode = 'reldbg'
  #
  # If you have many packages, and would like to only consider a subset of them,
  # you can set one or more package filters. Note that this concerns packages
  # *explicitly* enabled. Any package needed by those, will be *implicitly*
  # enabled. Thus, to build a package 'MyPkg', you only need to add 'MyPkg' to
  # the list, even if 'MyPkg' itself depends on many other packages.
  #
  #   pkg_filter = ['MyPkg','MyOtherPkg']
  #
  # More about filters:
  #
  # A filter can be positive (default) or negative (those prefixed with
  # '!'). Thus, you can exclude 'MyPkg' (and any pkgs needing 'MyPkg') by:
  #
  #   pkg_filter = ['!MyPkg']
  #
  # If any positive filters are defined, a pkg must match at least one of them to pass.
  # If any negative filters are defined, a pkg must match none of them, to pass.
  #
  # You can use wildcards (fnmatch). The following matches both MyPkg and MyOtherPkg:
  #
  #   pkg_filter = ['My*']
  #
  # Filters with '/' in them are matched on the full path relative to their
  # pkg_root, otherwise matching is just based on the package name. So if MyPkg
  # is physically located at <pkgroot>/some/where/MyPkg, the string to match for
  # filters with '/' in them would be 'some/where/MyPkg'. I.e. MyPkg would be
  # enabled by:
  #
  #   pkg_filter = ['some/where/*']
  #
  # Filters can use regular expressions, simply prefix them with 'RE::'. Here is
  # an example with both a positive and a negative filter, selecting packages
  # whose names include the word 'Util', but does not include the word
  # 'test'. The '(?i)' makes the second filter case-insensitive, so
  # 'MyUtilsTests' and 'MyUtils_tests' would both be excluded by it. 'MyUtils'
  # would be accepted, but 'My_utils' would not:
  #
  #   pkg_filter = ['RE::.*Util.*','!RE::(?i).*test.*']
  #
  # Commas (,) can not be used in filters, but serve to separate filter
  # items. In fact, the two following pkg filters are identical:
  #
  #   pkg_filter = [ 'My*','!MyBrokenPkg' ]
  #   pkg_filter = [ 'My*,!MyBrokenPkg' ]

Format

Technically, simplebuild.cfg files are in TOML format, and contain entries (TOML keys) in three sections (TOML tables) discussed in the following. All entries are optional and even an empty simplebuild.cfg might be enough for simple projects. As such, many users will not have to worry about the format of this file. Note that file system paths specified can be either relative or absolute, and that relative paths are always interpreted relative to the directory in which the simplebuild.cfg file is located.

The `[bundle]` section

Information about the associated package bundle.

name (string): If you wish to let other bundles use your bundle of packages, you must give it a name (so they can specify it as a dependency (see below for the bundles key in the [depend] section). Valid names are lowercase and alphanumeric (underscores allowed, must start with a letter).
pkg_root (string): Path to the top-level directory of the package bundle. Defaults to ".", i.e. the same directory as the one containing the simplebuild.cfg.
env_paths (array of strings): An advanced setting, allowing bundles to modify environment paths to inject simplebuild installation folders as appropriate. As the core bundle already modifies PATH and PYTHONPATH, most users should NOT need to use this parameter. The following example (from the core bundle’s simplebuild.cfg file) illustrates the syntax:
```
[bundle]
  #..
  env_paths = [ "PATH:<install>/bin:<install>/scripts",
                "PYTHONPATH:<install>/python" ]
```

The `[depend]` section

Information about which other bundles the current package bundle depends on.

bundles (array of strings): Names of other bundles on which the bundle depends. The core bundle (with the Core package) is always an implicit dependency and does not need to be listed.
search_path (array of strings): Local paths to simplebuild.cfg files (or the directories in which they reside). This advanced variable will help simplebuild to actually locate the package bundles that are specified as dependencies in the bundles key. Any dependency not found via these search paths will be attempted to be located via the python-module-search mechanism (see below). The core and core_val bundles are always available via this latter mechanism (the same goes for the dgcode and dgcode_val bundles, if the simple-build-dgcode Python or Conda package has been installed).

The `[build]` section

Information related to the build process. Note that only the [build] section of the main simplebuild.cfg file is considered. Any [build] section in a bundle which is simply being used by another bundle, will be completely ignored.

cachedir (string): A relative path to the temporary directory in which simplebuild will place temporary build output. Defaults to "./simplebuild_cache".
pkg_filter (array of string): A list of filter expressions, which can be used to selectively enable only some packages in the enabled bundles for consideration (the default is to enable all packages). This is obviously primarily useful for bundles with a very large number of packages. Note that this concerns packages being explicitly enabled. Any package needed by those, will be implicitly enabled. Thus, to build a package ‘MyPkg’, you only need to enable explicitly enable ‘MyPkg’, even if ‘MyPkg’ itself depends on many other packages. The syntax for package filter specification is rather extended, and is discussed in a dedicated section below.
mode (string): A string which must be either "release" (the default), "reldbg", or "debug". The "release" mode enables all binaries to be build with debug symbols enabled, which is sometimes useful for expert-level debugging. The debug mode additionally reduces the level of compiler optimisations, enables assert(..) statements in C/C++ code, etc.

Package filters

The most simple package filter entries, are simply the names of packages:

pkg_filter = ['MyPkg','MyOtherPkg']

A filter can be positive (default) or negative (those prefixed with !). Thus, you can exclude MyPkg (and any packages needing MyPkg) by:

pkg_filter = ['!MyPkg']

If any positive filters are defined, a pkg must match at least one of them to pass. If any negative filters are defined, a pkg must match none of them, to pass.

You can use fnmatch-like wildcards. The following matches both MyPkg and MyOtherPkg:

pkg_filter = ['My*']

Filters with / in them are matched on the full path relative to their pkg_root, otherwise matching is just based on the package name. So if MyPkg is physically located at <pkgroot>/some/where/MyPkg, the string to match for filters with / in them would be some/where/MyPkg. I.e. MyPkg would be enabled by:

pkg_filter = ['some/where/*']

Filters can use regular expressions, simply prefix them with RE::. Here is an example with both a positive and a negative filter, selecting packages whose names include the word Util, but does not include the word test. The (?i) makes the latter case-insensitive, so MyUtilsTests and MyUtils_tests would both be excluded. MyUtils would be accepted, but My_utils would not:

pkg_filter = ['RE::.*Util.*','!RE::(?i).*test.*']

Commas (,) can not be used in filters, but serve to separate filter items. In fact, the two following pkg filters are identical:

pkg_filter = [ 'My*','!MyBrokenPkg' ]

pkg_filter = [ 'My*,!MyBrokenPkg' ]

Advanced topics

The SIMPLEBUILD_CFG variable

Advanced users maintaining several interdependent simplebuild package bundles, might wish to override the automatic search for a main simplebuild.cfg file based on the current working directory. For that purpose, one can use the SIMPLEBUILD_CFG environment variable to point to a particular simplebuild.cfg file. In this case, the indicated file is simply taken to be the main file, and the current working directory is ignored (in fact, in this case the file does not even have to be named simplebuild.cfg).

For instance, they might wish to be able to freely switch their current working directory between directories inside two different interdependent bundles, and be able to always build both bundles when invoking sb, irrespective of their current working directory. For such working modes, these advanced users can simply create a simplebuild_everything.cfg file (the name can be anything) with appropriate entries in the [depend] section and set:

export SIMPLEBUILD_CFG=/some/where/simplebuild_everything.cfg

Bundle search via Python modules

In addition to the depend.search_path key in the simplebuild.cfg file above, simplebuild is also able to discover local bundles through a Python plugin mechanism. This is in fact how bundles like core and core_val are made easily available for all users without requiring them to edit their depend.search_path (the same goes for the dgcode and dgcode_val bundles, if the simple-build-dgcode Python or Conda package has been installed).

Specifically, simplebuild will look for Python modules whose names follow the pattern [_]simplebuild_[anything].simplebuild_bundle_list. Inside that module there must be a function called simplebuild_bundle_list() which returns a list of pathlib.Path objects, each being an absolute path to a simplebuild.cfg file. As an example, installing the simple-build-dgcode Python or Conda package, results in a new Python module becoming available in the environment: simplebuild_dgcode.simplebuild_bundle_list, with a simplebuild_bundle_list() returning the full path to two simplebuild.cfg files: one for the dgcode bundle, and one for the dgcode_val bundle.

Search path redirection

Very few people will need this feature, but for completeness we mention here the possibility to add simplebuild_redirect.cfg files, which themselves essentially just contain a list of search paths. If a directory added to the depend.search_path contains such a redirection file, all of the search paths found in it will themselves be added to the search path.

Click to show an example of such a redirection file.

# ; -*-conf-toml-*-

# Special simplebuild cfg file which allows one to use the pkgs in the contained
# bundles directly from a git clone of the repository rather than after
# installing as a python package, and simply by adding the root directory of the
# git clone to the depend.search_path list in a given simplebuild.cfg file. This
# exists to facilitate development work, and is not as such intended for
# end-users.

[special]
  redirect_search_path = [
  "./src/_simple_build_system/data/pkgs-core",
  "./src/_simple_build_system/data/pkgs-core_val",
  ]