.. _sbdotcfg: ************************ The simplebuild.cfg file ************************ Each simplebuild bundle of packages is defined by a single ``simplebuild.cfg`` file, usually located at the root directory under which all the packages are located. It serves a few different purposes: * When invoked, the ``sb`` command will look first in the current directory, then recursively in the parent directories for such a file - which is then taken as the actual *main cfg* file associated with that invocation of ``sb``. This is for instance analogous to the way most Git-users are used to working (the ``git`` commands searches the current and parent directories for a ``.git/`` directory), and allows for instance a mode of operation where simplebuild users can simply clone a code repository and then step into it and invoke the ``sb`` command to start compiling the simplebuild packages in that repository (assuming for instance that the repository contains a ``simplebuild.cfg`` package at its root). * It specifies dependencies on other simplebuild package bundles, which will cause simplebuild to search for these bundles and include their packages. * It can be used to fine-tune the build process, e.g. to enable debug builds, specify the cache directory location, or to only build certain packages, which might be useful in larger projects. Automatic creation ================== If you are creating a new simplebuild bundle for a new project, you can get easily started by simply invoking ``sb --init [optional flags]`` in the directory under which you wish to keep your package bundle. As explained in `the dedicated documentation for sb -\-init <./cmdline.html#new-bundle-initialisation-options>`_, this will generate a ``simplebuild.cfg`` file for you. Below here is repeated the file which is generated by ``sb --init`` without any other arguments. As you can see, it contains a lot of comments with explanation and examples, which is convenient for when you will be editing the file later: .. collapse:: Click to show simplebuild.cfg generated by sb --init .. literalinclude:: ../build/autogen_freshsbinit/simplebuild.cfg :language: toml | .. Note that the | character just above is to insert a bit more vertical space here, since the collapse box seems to not add enough space after itself. Format ====== Technically, ``simplebuild.cfg`` files are in `TOML `_ format, and contain entries (TOML keys) in three sections (TOML tables) discussed in the following. All entries are optional and even an empty ``simplebuild.cfg`` might be enough for simple projects. As such, **many users will not have to worry about the format of this file**. Note that file system paths specified can be either relative or absolute, and that relative paths are always interpreted relative to the directory in which the ``simplebuild.cfg`` file is located. The ``[bundle]`` section ------------------------- Information about the associated package bundle. * ``name`` (string): If you wish to let other bundles use your bundle of packages, you must give it a name (so they can specify it as a dependency (see below for the ``bundles`` key in the ``[depend]`` section). Valid names are lowercase and alphanumeric (underscores allowed, must start with a letter). The name ``main`` is reserved, and is used to refer to the main bundle when it is unnamed (for instance in the output of ``sb -s``). * ``pkg_root`` (string): Path to the top-level directory of the package bundle. Defaults to ``"."``, i.e. the same directory as the one containing the ``simplebuild.cfg``. * ``env_paths`` (array of strings): An advanced setting, allowing bundles to modify environment paths to inject simplebuild installation folders as appropriate. As the :sbpkg:`bundleroot::core` bundle already modifies ``PATH`` and ``PYTHONPATH``, most users should NOT need to use this parameter. The following example (from the :sbpkg:`bundleroot::core` bundle's ``simplebuild.cfg`` file) illustrates the syntax: .. code-block:: toml [bundle] #.. env_paths = [ "PATH:/bin:/scripts", "PYTHONPATH:/python", "PATH:./project/cmds:/other/cmds:~/mycmds" * ``dynamic_generator`` (string): Path to a Python file which will be invoked by simplebuild whenever it is itself invoked, which can for instance be used to dynamically generate code in simplebuild packages. This feature is obviously for rather advanced users only. One use-case could be to include code from repositories whose layout does not match that required by a simplebuild package. The script is invoked (with the same Python interpreter as simplebuild itself, in isolated mode, and from the current working directory) by all of ``sb``, ``sbrun``, and ``sbenv`` before anything else is done, with the installation directory and the build mode as arguments. It should therefore be fast, and only modify files when something actually changed. If the script fails (exits with a non-zero exit code), so does the simplebuild command (also showing the output of the script on stderr in ``--quiet`` mode, where it is otherwise hidden; without ``--quiet`` all output of the script goes to stdout). Simultaneous invocations of the script are avoided (e.g. when ``sbenv`` is invoked in several terminals at the same time). * ``extdep_root`` (string): Path to a directory containing additional CMake files for defining external dependencies (expert-only). An external dependency defined like this can also provide an include map (see :ref:`header files `). The ``[depend]`` section ------------------------ Information about which other bundles the current package bundle depends on. * ``bundles`` (array of strings): Names of other bundles on which the bundle depends. The :sbpkg:`bundleroot::core` bundle (with the :sbpkg:`Core` package) is always an implicit dependency and does not need to be listed. * ``search_path`` (array of strings): Local paths to ``simplebuild.cfg`` files (or the directories in which they reside). This advanced variable will help simplebuild to actually locate the package bundles that are specified as dependencies in the ``bundles`` key. Any dependency not found via these search paths will be attempted to be located via the python-module-search mechanism (see :ref:`below `). The :sbpkg:`bundleroot::core` and :sbpkg:`bundleroot::core_val` bundles are always available via this latter mechanism (the same goes for the :sbpkg:`bundleroot::dgcode` and :sbpkg:`bundleroot::dgcode_val` bundles, if the ``simple-build-dgcode`` Python or Conda package has been installed). The ``[build]`` section ----------------------- Information related to the build process. Note that only the ``[build]`` section of the main ``simplebuild.cfg`` file is considered. Any ``[build]`` section in a bundle which is simply being used by another bundle, will be completely ignored. * ``cachedir`` (string): A relative path to the temporary directory in which simplebuild will place temporary build output. Defaults to ``"./simplebuild_cache"`` (see also :ref:`the SIMPLEBUILD_CACHE_ROOT variable `). * ``pkg_filter`` (array of string): A list of filter expressions, which can be used to selectively enable only some packages in the enabled bundles for consideration (the default is to enable all packages). This is obviously primarily useful for bundles with a very large number of packages. Note that this concerns packages being *explicitly* enabled. Any package needed by those, will be *implicitly* enabled. Thus, to build a package 'MyPkg', you only need to enable explicitly enable 'MyPkg', even if 'MyPkg' itself depends on many other packages. The syntax for package filter specification is rather extended, and is discussed in a dedicated section :ref:`below `. * ``mode`` (string): A string which must be either ``"release"`` (the default), ``"reldbg"``, or ``"debug"``. The ``"reldbg"`` mode is like ``"release"``, but enables all binaries to be build with debug symbols enabled, which is sometimes useful for expert-level debugging. The ``"debug"`` mode additionally reduces the level of compiler optimisations, enables ``assert(..)`` statements in C/C++ code, etc. The mode can be overridden on the command line, by the ``-r``, ``-g``, or ``-d`` flags of ``sb``, ``sbrun``, and ``sbenv`` (see :ref:`build variants `). * ``extra_cflags`` (string or array of string): Extra flags to use when compiling C and C++ code. They are added after all other flags, so they can override flags otherwise used by simplebuild (e.g. ``-O2`` will override the ``-O3`` used in ``"release"`` mode). Each string is split into individual flags in the same way as a shell would do it, so the following are equivalent: .. code-block:: toml extra_cflags = '-ffp-contract=off -DMYFLAG=1' extra_cflags = ['-ffp-contract=off','-DMYFLAG=1'] For safety, the resulting flags can only contain letters, digits, and the characters ``_+-.,:=/@%``. Builds with extra flags are carried out in separate cache directories, and the flags can be ignored with the ``-x`` flag of ``sb``, ``sbrun``, and ``sbenv`` (see :ref:`build variants `). This makes it easy to compare results with and without a given set of flags, without needing to edit the cfg file and rebuild everything each time. * ``extra_ldflags`` (string or array of string): Like ``extra_cflags``, but for flags which should be used when linking libraries and executables. * ``flagsets`` (table): Named sets of extra flags, any of which can be selected with the ``-X NAME`` flag of ``sb``, ``sbrun``, and ``sbenv`` (see :ref:`build variants `). The selected set is then used *instead of* ``extra_cflags`` and ``extra_ldflags``. Each set can have the keys ``extra_cflags`` and ``extra_ldflags``, with values as described above. The names of the build modes (``release``, ``reldbg``, and ``debug``) can not be used as names of flag sets. This makes it easy to compare results from several sets of flags, for instance to check that results do not depend on the contraction of floating point operations: .. code-block:: toml [build] mode = 'reldbg' flagsets.fma.extra_cflags = '-mfma' flagsets.nocontract.extra_cflags = '-ffp-contract=off' flagsets.ubsan.extra_cflags = '-fsanitize=undefined' flagsets.ubsan.extra_ldflags = '-fsanitize=undefined' Here, ``sb -X fma`` and ``sb -X nocontract`` build the two variants in separate cache directories, and for instance ``sbrun -X fma mycmd`` runs a command with one of them. The names of the sets can only contain letters, digits, underscores and dashes (and can not start with a dash). * ``max_cachedirs`` (integer): The maximum number of build variants (i.e. a given build mode, with or without extra flags or a flag set) for which build output is kept in the cache directory. When more are needed, the least recently used ones are removed. The value must be from 1 to 20, and the default is 5. .. _sbpkgfilterspec: Package filters --------------- The most simple package filter entries, are simply the names of packages: .. code-block:: toml pkg_filter = ['MyPkg','MyOtherPkg'] A filter can be positive (default) or negative (those prefixed with ``!``). Thus, you can exclude ``MyPkg`` (and any packages needing ``MyPkg``) by: .. code-block:: toml pkg_filter = ['!MyPkg'] If any positive filters are defined, a pkg must match at least one of them to pass. If any negative filters are defined, a pkg must match none of them, to pass. You can use fnmatch-like wildcards. The following matches both MyPkg and MyOtherPkg: .. code-block:: toml pkg_filter = ['My*'] Filters with ``/`` in them are matched on the full path relative to their ``pkg_root``, otherwise matching is just based on the package name. So if ``MyPkg`` is physically located at ``/some/where/MyPkg``, the string to match for filters with ``/`` in them would be ``some/where/MyPkg``. I.e. ``MyPkg`` would be enabled by: .. code-block:: toml pkg_filter = ['some/where/*'] Filters can use regular expressions, simply prefix them with ``RE::``. Here is an example with both a positive and a negative filter, selecting packages whose names include the word ``Util``, but does not include the word ``test``. The ``(?i)`` makes the latter case-insensitive, so ``MyUtilsTests`` and ``MyUtils_tests`` would both be excluded. ``MyUtils`` would be accepted, but ``My_utils`` would not: .. code-block:: toml pkg_filter = ['RE::.*Util.*','!RE::(?i).*test.*'] Commas (``,``) can not be used in filters, but serve to separate filter items. In fact, the two following pkg filters are identical: .. code-block:: toml pkg_filter = [ 'My*','!MyBrokenPkg' ] .. code-block:: toml pkg_filter = [ 'My*,!MyBrokenPkg' ] Advanced topics =============== The SIMPLEBUILD_CFG variable ---------------------------- Advanced users maintaining several interdependent simplebuild package bundles, might wish to override the automatic search for a main ``simplebuild.cfg`` file based on the current working directory. For that purpose, one can use the ``SIMPLEBUILD_CFG`` environment variable to point to a particular ``simplebuild.cfg`` file. In this case, the indicated file is simply taken to be the main file, and the current working directory is ignored (in fact, in this case the file does not even have to be named ``simplebuild.cfg``). For instance, they might wish to be able to freely switch their current working directory between directories inside two different interdependent bundles, and be able to always build both bundles when invoking ``sb``, irrespective of their current working directory. For such working modes, these advanced users can simply create a ``simplebuild_everything.cfg`` file (the name can be anything) with appropriate entries in the ``[depend]`` section and set:: export SIMPLEBUILD_CFG=/some/where/simplebuild_everything.cfg .. _sbcacheroot: The SIMPLEBUILD_CACHE_ROOT variable ----------------------------------- Users who prefer to keep temporary build output away from their bundles, e.g. on a dedicated scratch disk, can set the ``SIMPLEBUILD_CACHE_ROOT`` environment variable (e.g. in their ``~/.bashrc``) to the absolute path of a directory in which simplebuild will then place the cache directories:: export SIMPLEBUILD_CACHE_ROOT=/scratch/myusername/sbcache The cache directory of a bundle will then be a subdirectory with a name like ``sb__``, where the hash is calculated from the full path to the main ``simplebuild.cfg`` file. Thus, each checkout of a bundle gets its own cache directory, which is always the same as long as the checkout is not moved. The full path to the ``simplebuild.cfg`` file is recorded in a file named ``simplebuild_cfg_path.txt`` in the cache directory (which is also used to detect clashes). Note that since the cache directory of a checkout which is moved or removed is not removed automatically, it might be a good idea to occasionally clean up the directory. By default, ``SIMPLEBUILD_CACHE_ROOT`` is only used for bundles whose main ``simplebuild.cfg`` file does not specify a ``cachedir``. To use it for all bundles, add a ``force::`` prefix to the value:: export SIMPLEBUILD_CACHE_ROOT=force::/scratch/myusername/sbcache Note that the variable must be set in the same way whenever simplebuild commands like ``sb``, ``sbrun``, and ``sbenv`` are used, since they would otherwise look for the build output in different places. .. _sbpymodsearchpath: Bundle search via Python modules -------------------------------- In addition to the ``depend.search_path`` key in the ``simplebuild.cfg`` file above, simplebuild is also able to discover local bundles through a `Python plugin `_ mechanism. This is in fact how bundles like :sbpkg:`bundleroot::core` and :sbpkg:`bundleroot::core_val` are made easily available for all users without requiring them to edit their ``depend.search_path`` (the same goes for the :sbpkg:`bundleroot::dgcode` and :sbpkg:`bundleroot::dgcode_val` bundles, if the ``simple-build-dgcode`` Python or Conda package has been installed). Specifically, simplebuild will look for Python modules whose names follow the pattern ``[_]simplebuild_[anything].simplebuild_bundle_list``. Inside that module there must be a function called ``simplebuild_bundle_list()`` which returns a list of pathlib.Path objects, each being an absolute path to a ``simplebuild.cfg`` file. As an example, installing the ``simple-build-dgcode`` Python or Conda package, results in a new Python module becoming available in the environment: ``simplebuild_dgcode.simplebuild_bundle_list``, with a ``simplebuild_bundle_list()`` returning the full path to two ``simplebuild.cfg`` files: one for the :sbpkg:`bundleroot::dgcode` bundle, and one for the :sbpkg:`bundleroot::dgcode_val` bundle. Search path redirection ----------------------- Very few people will need this feature, but for completeness we mention here the possibility to add ``simplebuild_redirect.cfg`` files, which themselves essentially just contain a list of search paths. If a directory added to the ``depend.search_path`` contains such a redirection file, all of the search paths found in it will themselves be added to the search path. .. collapse:: Click to show an example of such a redirection file. .. literalinclude:: ../../simplebuild_redirect.cfg :language: toml | .. Note that the | character just above is to insert a bit more vertical space here, since the collapse box seems to not add enough space after itself.