Files
spack/lib/spack/docs/include_yaml.rst

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

66 lines
2.7 KiB
ReStructuredText
Raw Normal View History

Support independent includes with conditional, optional, and remote entries (#48784) Supersedes #46792. Closes #40018. Closes #31026. Closes #2700. There were a number of feature requests for os-specific config. This enables os-specific config without adding a lot of special sub-scopes. Support `include:` as an independent configuration schema, allowing users to include configuration scopes from files or directories. Includes can be: * conditional (similar to definitions in environments), and/or * optional (i.e., the include will be skipped if it does not exist). Includes can be paths or URLs (`ftp`, `https`, `http` or `file`). Paths can be absolute or relative . Environments can include configuration files using the same schema. Remote includes must be checked by `sha256`. Includes can also be recursive, and this modifies the config system accordingly so that we push included configuration scopes on the stack *before* their including scopes, and we remove configuration scopes from the stack when their including scopes are removed. For example, you could have an `include.yaml` file (e.g., under `$HOME/.spack`) to specify global includes: ``` include: - ./enable_debug.yaml - path: https://github.com/spack/spack-configs/blob/main/NREL/configs/mac/config.yaml sha256: 37f982915b03de18cc4e722c42c5267bf04e46b6a6d6e0ef3a67871fcb1d258b ``` Or an environment `spack.yaml`: ``` spack: include: - path: "/path/to/a/config-dir-or-file" when: os == "ventura" - ./path/relative/to/containing/file/that/is/required - path: "/path/with/spack/variables/$os/$target" optional: true - path: https://raw.githubusercontent.com/spack/spack-configs/refs/heads/main/path/to/required/raw/config.yaml sha256: 26e871804a92cd07bb3d611b31b4156ae93d35b6a6d6e0ef3a67871fcb1d258b ``` Updated TODO: - [x] Get existing unit tests to pass with Todd's changes - [x] Resolve new (or old) circular imports - [x] Ensure remote includes (global) work - [x] Ensure remote includes for environments work (note: caches remote files under user cache root) - [x] add sha256 field to include paths, validate, and require for remote includes - [x] add sha256 remote file unit tests - [x] revisit how diamond includes should work - [x] support recursive includes - [x] add recursive include unit tests - [x] update docs and unit test to indicate ordering of recursive includes with conflicting options is deferred to follow-on work --------- Signed-off-by: Todd Gamblin <tgamblin@llnl.gov> Co-authored-by: Peter Scheibel <scheibel1@llnl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2025-03-09 19:33:44 -07:00
.. Copyright Spack Project Developers. See COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
.. _include-yaml:
===============================
Include Settings (include.yaml)
===============================
Spack allows you to include configuration files through ``include.yaml``.
Using the ``include:`` heading results in pulling in external configuration
information to be used by any Spack command.
Included configuration files are required *unless* they are explicitly optional
or the entry's condition evaluates to ``false``. Optional includes are specified
with the ``optional`` clause and conditional with the ``when`` clause. For
example,
.. code-block:: yaml
include:
- /path/to/a/required/config.yaml
- path: /path/to/$os/$target/config
optional: true
- path: /path/to/os-specific/config-dir
when: os == "ventura"
shows all three. The first entry, ``/path/to/a/required/config.yaml``,
indicates that included ``config.yaml`` file is required (so must exist).
Use of ``optional: true`` for ``/path/to/$os/$target/config`` means
the path is only included if it exists. The condition ``os == "ventura"``
in the ``when`` clause for ``/path/to/os-specific/config-dir`` means the
path is only included when the operating system (``os``) is ``ventura``.
The same conditions and variables in `Spec List References
<https://spack.readthedocs.io/en/latest/environments.html#spec-list-references>`_
can be used for conditional activation in the ``when`` clauses.
Included files can be specified by path or by their parent directory.
Paths may be absolute, relative (to the configuration file including the path),
or specified as URLs. Only the ``file``, ``ftp``, ``http`` and ``https`` protocols (or
schemes) are supported. Spack-specific, environment and user path variables
can be used. (See :ref:`config-file-variables` for more information.)
A ``sha256`` is required for remote file URLs and must be specified as follows:
.. code-block:: yaml
include:
- path: https://github.com/path/to/raw/config/compilers.yaml
sha256: 26e871804a92cd07bb3d611b31b4156ae93d35b6a6d6e0ef3a67871fcb1d258b
Additionally, remote file URLs must link to the **raw** form of the file's
contents (e.g., `GitHub
<https://docs.github.com/en/repositories/working-with-files/using-files/viewing-and-understanding-files#viewing-or-copying-the-raw-file-content>`_
or `GitLab
<https://docs.gitlab.com/ee/api/repository_files.html#get-raw-file-from-repository>`_).
Support independent includes with conditional, optional, and remote entries (#48784) Supersedes #46792. Closes #40018. Closes #31026. Closes #2700. There were a number of feature requests for os-specific config. This enables os-specific config without adding a lot of special sub-scopes. Support `include:` as an independent configuration schema, allowing users to include configuration scopes from files or directories. Includes can be: * conditional (similar to definitions in environments), and/or * optional (i.e., the include will be skipped if it does not exist). Includes can be paths or URLs (`ftp`, `https`, `http` or `file`). Paths can be absolute or relative . Environments can include configuration files using the same schema. Remote includes must be checked by `sha256`. Includes can also be recursive, and this modifies the config system accordingly so that we push included configuration scopes on the stack *before* their including scopes, and we remove configuration scopes from the stack when their including scopes are removed. For example, you could have an `include.yaml` file (e.g., under `$HOME/.spack`) to specify global includes: ``` include: - ./enable_debug.yaml - path: https://github.com/spack/spack-configs/blob/main/NREL/configs/mac/config.yaml sha256: 37f982915b03de18cc4e722c42c5267bf04e46b6a6d6e0ef3a67871fcb1d258b ``` Or an environment `spack.yaml`: ``` spack: include: - path: "/path/to/a/config-dir-or-file" when: os == "ventura" - ./path/relative/to/containing/file/that/is/required - path: "/path/with/spack/variables/$os/$target" optional: true - path: https://raw.githubusercontent.com/spack/spack-configs/refs/heads/main/path/to/required/raw/config.yaml sha256: 26e871804a92cd07bb3d611b31b4156ae93d35b6a6d6e0ef3a67871fcb1d258b ``` Updated TODO: - [x] Get existing unit tests to pass with Todd's changes - [x] Resolve new (or old) circular imports - [x] Ensure remote includes (global) work - [x] Ensure remote includes for environments work (note: caches remote files under user cache root) - [x] add sha256 field to include paths, validate, and require for remote includes - [x] add sha256 remote file unit tests - [x] revisit how diamond includes should work - [x] support recursive includes - [x] add recursive include unit tests - [x] update docs and unit test to indicate ordering of recursive includes with conflicting options is deferred to follow-on work --------- Signed-off-by: Todd Gamblin <tgamblin@llnl.gov> Co-authored-by: Peter Scheibel <scheibel1@llnl.gov> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2025-03-09 19:33:44 -07:00
.. warning::
Recursive includes are not currently processed in a breadth-first manner
so the value of a configuration option that is altered by multiple included
files may not be what you expect. This will be addressed in a future
update.