Include concrete environments with include_concrete
(#33768)
Add the ability to include any number of (potentially nested) concrete environments, e.g.: ```yaml spack: specs: [] concretizer: unify: true include_concrete: - /path/to/environment1 - /path/to/environment2 ``` or, from the CLI: ```console $ spack env create myenv $ spack -e myenv add python $ spack -e myenv concretize $ spack env create --include-concrete myenv included_env ``` The contents of included concrete environments' spack.lock files are included in the environment's lock file at creation time. Any changes to included concrete environments are only reflected after the environment is re-concretized from the re-concretized included environments. - [x] Concretize included envs - [x] Save concrete specs in memory by hash - [x] Add included envs to combined env's lock file - [x] Add test - [x] Update documentation Co-authored-by: Kayla Butler <<butler59@llnl.gov> Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co m> Co-authored-by: Todd Gamblin <tgamblin@llnl.gov> Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This commit is contained in:
@@ -460,6 +460,125 @@ Sourcing that file in Bash will make the environment available to the
|
||||
user; and can be included in ``.bashrc`` files, etc. The ``loads``
|
||||
file may also be copied out of the environment, renamed, etc.
|
||||
|
||||
|
||||
.. _environment_include_concrete:
|
||||
|
||||
------------------------------
|
||||
Included Concrete Environments
|
||||
------------------------------
|
||||
|
||||
Spack environments can create an environment based off of information in already
|
||||
established environments. You can think of it as a combination of existing
|
||||
environments. It will gather information from the existing environment's
|
||||
``spack.lock`` and use that during the creation of this included concrete
|
||||
environment. When an included concrete environment is created it will generate
|
||||
a ``spack.lock`` file for the newly created environment.
|
||||
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Creating included environments
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
To create a combined concrete environment, you must have at least one existing
|
||||
concrete environment. You will use the command ``spack env create`` with the
|
||||
argument ``--include-concrete`` followed by the name or path of the environment
|
||||
you'd like to include. Here is an example of how to create a combined environment
|
||||
from the command line.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack env create myenv
|
||||
$ spack -e myenv add python
|
||||
$ spack -e myenv concretize
|
||||
$ spack env create --include-concrete myenv included_env
|
||||
|
||||
|
||||
You can also include an environment directly in the ``spack.yaml`` file. It
|
||||
involves adding the ``include_concrete`` heading in the yaml followed by the
|
||||
absolute path to the independent environments.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
spack:
|
||||
specs: []
|
||||
concretizer:
|
||||
unify: true
|
||||
include_concrete:
|
||||
- /absolute/path/to/environment1
|
||||
- /absolute/path/to/environment2
|
||||
|
||||
|
||||
Once the ``spack.yaml`` has been updated you must concretize the environment to
|
||||
get the concrete specs from the included environments.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Updating an included environment
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
If changes were made to the base environment and you want that reflected in the
|
||||
included environment you will need to reconcretize both the base environment and the
|
||||
included environment for the change to be implemented. For example:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack env create myenv
|
||||
$ spack -e myenv add python
|
||||
$ spack -e myenv concretize
|
||||
$ spack env create --include-concrete myenv included_env
|
||||
|
||||
|
||||
$ spack -e myenv find
|
||||
==> In environment myenv
|
||||
==> Root specs
|
||||
python
|
||||
|
||||
==> 0 installed packages
|
||||
|
||||
|
||||
$ spack -e included_env find
|
||||
==> In environment included_env
|
||||
==> No root specs
|
||||
==> Included specs
|
||||
python
|
||||
|
||||
==> 0 installed packages
|
||||
|
||||
Here we see that ``included_env`` has access to the python package through
|
||||
the ``myenv`` environment. But if we were to add another spec to ``myenv``,
|
||||
``included_env`` will not be able to access the new information.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack -e myenv add perl
|
||||
$ spack -e myenv concretize
|
||||
$ spack -e myenv find
|
||||
==> In environment myenv
|
||||
==> Root specs
|
||||
perl python
|
||||
|
||||
==> 0 installed packages
|
||||
|
||||
|
||||
$ spack -e included_env find
|
||||
==> In environment included_env
|
||||
==> No root specs
|
||||
==> Included specs
|
||||
python
|
||||
|
||||
==> 0 installed packages
|
||||
|
||||
It isn't until you run the ``spack concretize`` command that the combined
|
||||
environment will get the updated information from the reconcretized base environmennt.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack -e included_env concretize
|
||||
$ spack -e included_env find
|
||||
==> In environment included_env
|
||||
==> No root specs
|
||||
==> Included specs
|
||||
perl python
|
||||
|
||||
==> 0 installed packages
|
||||
|
||||
.. _environment-configuration:
|
||||
|
||||
------------------------
|
||||
@@ -811,6 +930,7 @@ For example, the following environment has three root packages:
|
||||
This allows for a much-needed reduction in redundancy between packages
|
||||
and constraints.
|
||||
|
||||
|
||||
----------------
|
||||
Filesystem Views
|
||||
----------------
|
||||
@@ -1044,7 +1164,7 @@ other targets to depend on the environment installation.
|
||||
|
||||
A typical workflow is as follows:
|
||||
|
||||
.. code:: console
|
||||
.. code-block:: console
|
||||
|
||||
spack env create -d .
|
||||
spack -e . add perl
|
||||
@@ -1137,7 +1257,7 @@ its dependencies. This can be useful when certain flags should only apply to
|
||||
dependencies. Below we show a use case where a spec is installed with verbose
|
||||
output (``spack install --verbose``) while its dependencies are installed silently:
|
||||
|
||||
.. code:: console
|
||||
.. code-block:: console
|
||||
|
||||
$ spack env depfile -o Makefile
|
||||
|
||||
@@ -1159,7 +1279,7 @@ This can be accomplished through the generated ``[<prefix>/]SPACK_PACKAGE_IDS``
|
||||
variable. Assuming we have an active and concrete environment, we generate the
|
||||
associated ``Makefile`` with a prefix ``example``:
|
||||
|
||||
.. code:: console
|
||||
.. code-block:: console
|
||||
|
||||
$ spack env depfile -o env.mk --make-prefix example
|
||||
|
||||
|
Reference in New Issue
Block a user