b1d129e681
* Module files now are generated using a template engine refers #2902 #3173 jinja2 has been hooked into Spack. The python module `modules.py` has been splitted into several modules under the python package `spack/modules`. Unit tests stressing module file generation have been refactored accordingly. The module file generator for Lmod has been extended to multi-providers and deeper hierarchies. * Improved the support for templates in module files. Added an entry in `config.yaml` (`template_dirs`) to list all the directories where Spack could find templates for `jinja2`. Module file generators have a simple override mechanism to override template selection ('modules.yaml' beats 'package.py' beats 'default'). * Added jinja2 and MarkupSafe to vendored packages. * Spec.concretize() sets mutual spec-package references The correct place to set the mutual references between spec and package objects at the end of concretization. After a call to concretize we should now be ensured that spec is the same object as spec.package.spec. Code in `build_environment.py` that was performing the same operation has been turned into an assertion to be defensive on the new behavior. * Improved code and data layout for modules and related tests. Common fixtures related to module file generation have been extracted in `conftest.py`. All the mock configurations for module files have been extracted from python code and have been put into their own yaml file. Added a `context_property` decorator for the template engine, to make it easy to define dictionaries out of properties. The default for `verbose` in `modules.yaml` is now False instead of True. * Extendable module file contexts + short description from docstring The contexts that are used in conjunction with `jinja2` templates to generate module files can now be extended from package.py and modules.yaml. Module files generators now infer the short description from package.py docstring (and as you may expect it's the first paragraph) * 'module refresh' regenerates all modules by default `module refresh` without `--module-type` specified tries to regenerate all known module types. The same holds true for `module rm` Configure options used at build time are extracted and written into the module files where possible. * Fixed python3 compatibility, tests for Lmod and Tcl. Added test for exceptional paths of execution when generating Lmod module files. Fixed a few compatibility issues with python3. Fixed a bug in Tcl with naming_scheme and autoload + unit tests * Updated module file tutorial docs. Fixed a few typos in docstrings. The reference section for module files has been reorganized. The idea is to have only three topics at the highest level: - shell support + spack load/unload use/unuse - module file generation (a.k.a. APIs + modules.yaml) - module file maintenance (spack module refresh/rm) Module file generation will cover the entries in modules.yaml Also: - Licenses have been updated to include NOTICE and extended to 2017 - docstrings have been reformatted according to Google style * Removed redundant arguments to RPackage and WafPackage. All the callbacks in `RPackage` and `WafPackage` that are not build phases have been modified not to accept a `spec` and a `prefix` argument. This permits to leverage the common `configure_args` signature to insert by default the configuration arguments into the generated module files. I think it's preferable to handling those packages differently than `AutotoolsPackage`. Besides only one package seems to override one of these methods. * Fixed broken indentation + improved resiliency of refresh Fixed broken indentation in `spack module refresh` (probably a rebase gone silently wrong?). Filter the writers for blacklisted specs before searching for name clashes. An error with a single writer will not stop regeneration, but instead will print a warning and continue the command.
985 lines
44 KiB
ReStructuredText
985 lines
44 KiB
ReStructuredText
.. _modules-tutorial:
|
|
|
|
=============================
|
|
Module Configuration Tutorial
|
|
=============================
|
|
|
|
This tutorial will guide you through the customization of both
|
|
content and naming of module files generated by Spack.
|
|
|
|
Starting from the default Spack settings you will add an increasing
|
|
number of directives to the ``modules.yaml`` configuration file to
|
|
satisfy a number of constraints that mimic those that you may encounter
|
|
in a typical production environment at HPC sites.
|
|
|
|
Even though the focus will be for the most part on customizing
|
|
TCL non-hierarchical module files, everything
|
|
you'll see applies also to other kinds of module files generated by Spack.
|
|
|
|
The generation of Lua hierarchical
|
|
module files will be addressed at the end of the tutorial,
|
|
and you'll see that with minor modifications
|
|
to an existing ``modules.yaml`` written for TCL
|
|
non-hierarchical modules you'll get almost
|
|
for free the possibility to try a hierarchical layout.
|
|
|
|
Let's start!
|
|
|
|
.. _module_file_tutorial_prerequisites:
|
|
|
|
-------------
|
|
Prerequisites
|
|
-------------
|
|
|
|
Before proceeding further ensure:
|
|
|
|
- you have LMod or Environment Modules available
|
|
- have :ref:`shell support <shell-support>` activated in Spack
|
|
|
|
If you need to install Lmod or Environment module you can refer
|
|
to the documentation :ref:`here <InstallEnvironmentModules>`.
|
|
|
|
|
|
^^^^^^^^^^^^^^^^^^
|
|
Add a new compiler
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
Spack automatically scans the environment to search for available
|
|
compilers on first use. On Ubuntu 14.04, a fresh clone will show
|
|
something like this:
|
|
|
|
.. code-block:: console
|
|
|
|
$ uname -a
|
|
Linux nuvolari 4.4.0-45-generic #66~14.04.1-Ubuntu SMP Wed Oct 19 15:05:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
|
|
|
|
$ spack compilers
|
|
==> Available compilers
|
|
-- gcc ----------------------------------------------------------
|
|
gcc@4.8
|
|
|
|
In order to showcase the capabilities of module customization, we will want to
|
|
build a limited set of packages with multiple compilers. If you do not already
|
|
have multiple compilers listed by ``spack compilers``, you should build one
|
|
with Spack:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack install gcc@6.2.0
|
|
# ...
|
|
# Wait a long time
|
|
# ...
|
|
|
|
Then we can use shell support for modules to add it to the list of known compilers:
|
|
|
|
.. code-block:: console
|
|
|
|
# The name of the generated module may vary
|
|
$ module load gcc-6.2.0-gcc-4.8-twd5nqg
|
|
|
|
$ spack compiler add
|
|
==> Added 1 new compiler to ~/.spack/linux/compilers.yaml
|
|
gcc@6.2.0
|
|
|
|
$ spack compilers
|
|
==> Available compilers
|
|
-- gcc ----------------------------------------------------------
|
|
gcc@6.2.0 gcc@4.8
|
|
|
|
Note that the 7-digit hash at the end of the generated module may vary depending
|
|
on architecture or package version.
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
Build software that will be used in the tutorial
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Next you should install a few modules that will be used in the tutorial:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack install netlib-scalapack ^openmpi ^openblas
|
|
$ spack install netlib-scalapack ^mpich ^openblas
|
|
$ spack install netlib-scalapack ^openmpi ^netlib-lapack
|
|
$ spack install netlib-scalapack ^mpich ^netlib-lapack
|
|
$ spack install py-scipy ^openblas
|
|
|
|
In the end your environment should look something like:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module avail
|
|
|
|
------------------------------------------------------------------------ ~/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
|
|
binutils-2.27-gcc-4.8-dz3xevw libpciaccess-0.13.4-gcc-6.2.0-eo2siet lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw python-2.7.12-gcc-6.2.0-qu7rc5p
|
|
bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-4.8-avb6azw m4-1.4.17-gcc-4.8-iggewke netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq sqlite-3.8.5-gcc-6.2.0-td3zfe7
|
|
cmake-3.5.2-gcc-6.2.0-6poypqg libsigsegv-2.10-gcc-6.2.0-g3qpmbi m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh tcl-8.6.5-gcc-4.8-atddxu7
|
|
curl-7.50.3-gcc-6.2.0-2ffacqm libtool-2.4.6-gcc-6.2.0-kiepac6 mpc-1.0.3-gcc-4.8-lylv7lk openblas-0.2.19-gcc-6.2.0-js33umc util-macros-1.19.0-gcc-6.2.0-uoukuqk
|
|
expat-2.2.0-gcc-6.2.0-bxqnjar libxml2-2.9.4-gcc-6.2.0-3k4ykbe mpfr-3.1.4-gcc-4.8-bldfx3w openmpi-2.0.1-gcc-6.2.0-s3qbtby xz-5.2.2-gcc-6.2.0-t5lk6in
|
|
gcc-6.2.0-gcc-4.8-twd5nqg lmod-6.4.5-gcc-4.8-7v7bh7b mpich-3.2-gcc-6.2.0-5n5xoep openssl-1.0.2j-gcc-6.2.0-hibnfda zlib-1.2.8-gcc-4.8-bds4ies
|
|
gmp-6.1.1-gcc-4.8-uq52e2n lua-5.3.2-gcc-4.8-xozf2hx ncurses-6.0-gcc-4.8-u62fit4 pkg-config-0.29.1-gcc-6.2.0-rslsgcs zlib-1.2.8-gcc-6.2.0-asydrba
|
|
gmp-6.1.1-gcc-6.2.0-3cfh3hi lua-luafilesystem-1_6_3-gcc-4.8-sbzejlz ncurses-6.0-gcc-6.2.0-7tb426s py-nose-1.3.7-gcc-6.2.0-4gl5c42
|
|
hwloc-1.11.4-gcc-6.2.0-3ostwel lua-luaposix-33.4.0-gcc-4.8-xf7y2p5 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l py-numpy-1.11.1-gcc-6.2.0-i3rpk4e
|
|
isl-0.14-gcc-4.8-cq73t5m lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy py-scipy-0.18.1-gcc-6.2.0-e6uljfi
|
|
libarchive-3.2.1-gcc-6.2.0-2b54aos lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-setuptools-25.2.0-gcc-6.2.0-hkqauaa
|
|
|
|
------------------------------------------------
|
|
Filter unwanted modifications to the environment
|
|
------------------------------------------------
|
|
|
|
The non-hierarchical TCL module files that have been generated so far
|
|
follow the default rules for module generation, which are given
|
|
:ref:`here <modules-yaml>` in the reference part of the manual. Taking a
|
|
look at the ``gcc`` module you'll see something like:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module show gcc-6.2.0-gcc-4.8-twd5nqg
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
~/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg:
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
whatis("gcc @6.2.0 ")
|
|
prepend_path("PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
|
|
prepend_path("CMAKE_PREFIX_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
|
|
prepend_path("MANPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
|
|
prepend_path("PKG_CONFIG_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
|
|
prepend_path("LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
|
|
prepend_path("LD_LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
|
|
prepend_path("CPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/include")
|
|
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
|
|
Fortran, and Java.
|
|
]])
|
|
|
|
As expected, a few environment variables representing paths will be modified
|
|
by the modules according to the default prefix inspection rules.
|
|
|
|
Consider now the case that your site has decided that e.g. ``CPATH`` and
|
|
``LIBRARY_PATH`` modifications should not be present in module files. What you can
|
|
do to abide by the rules is to create a configuration file ``~/.spack/modules.yaml``
|
|
with the following content:
|
|
|
|
.. code-block:: yaml
|
|
|
|
modules:
|
|
tcl:
|
|
all:
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
|
|
Next you should regenerate all the module files:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type tcl
|
|
==> You are about to regenerate tcl module files for:
|
|
|
|
-- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------
|
|
dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8
|
|
twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5
|
|
|
|
...
|
|
|
|
==> Do you want to proceed? [y/n] y
|
|
==> Regenerating tcl module files
|
|
|
|
If you take a look now at the module for ``gcc`` you'll see that the unwanted
|
|
paths have disappeared:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module show gcc-6.2.0-gcc-4.8-twd5nqg
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
~/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc-6.2.0-gcc-4.8-twd5nqg:
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
whatis("gcc @6.2.0 ")
|
|
prepend_path("PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
|
|
prepend_path("CMAKE_PREFIX_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
|
|
prepend_path("MANPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
|
|
prepend_path("PKG_CONFIG_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
|
|
prepend_path("LD_LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
|
|
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
|
|
Fortran, and Java.
|
|
]])
|
|
|
|
----------------------------------------------
|
|
Prevent some module files from being generated
|
|
----------------------------------------------
|
|
|
|
Another common request at many sites is to avoid exposing software that
|
|
is only needed as an intermediate step when building a newer stack.
|
|
Let's try to prevent the generation of
|
|
module files for anything that is compiled with ``gcc@4.8`` (the OS provided compiler).
|
|
|
|
To do this you should add a ``blacklist`` keyword to the configuration file:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 3,4
|
|
|
|
modules:
|
|
tcl:
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
|
|
and regenerate the module files:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type tcl --delete-tree
|
|
==> You are about to regenerate tcl module files for:
|
|
|
|
-- linux-Ubuntu14-x86_64 / gcc@4.8 ------------------------------
|
|
dz3xevw binutils@2.27 uq52e2n gmp@6.1.1 avb6azw libsigsegv@2.10 xozf2hx lua@5.3.2 xf7y2p5 lua-luaposix@33.4.0 lylv7lk mpc@1.0.3 u62fit4 ncurses@6.0 bds4ies zlib@1.2.8
|
|
twd5nqg gcc@6.2.0 cq73t5m isl@0.14 7v7bh7b lmod@6.4.5 sbzejlz lua-luafilesystem@1_6_3 iggewke m4@1.4.17 bldfx3w mpfr@3.1.4 atddxu7 tcl@8.6.5
|
|
|
|
-- linux-Ubuntu14-x86_64 / gcc@6.2.0 ----------------------------
|
|
csoc2mq bzip2@1.0.6 2b54aos libarchive@3.2.1 sfmeynw lzma@4.32.7 wnimqhw netlib-scalapack@2.0.2 s3qbtby openmpi@2.0.1 hkqauaa py-setuptools@25.2.0
|
|
6poypqg cmake@3.5.2 eo2siet libpciaccess@0.13.4 jcngz72 lzo@2.09 6bqlxqy netlib-scalapack@2.0.2 hibnfda openssl@1.0.2j qu7rc5p python@2.7.12
|
|
2ffacqm curl@7.50.3 g3qpmbi libsigsegv@2.10 lhgqa6s m4@1.4.17 wojunhq netlib-scalapack@2.0.2 rslsgcs pkg-config@0.29.1 td3zfe7 sqlite@3.8.5
|
|
bxqnjar expat@2.2.0 kiepac6 libtool@2.4.6 5n5xoep mpich@3.2 hpqb3dp netlib-scalapack@2.0.2 4gl5c42 py-nose@1.3.7 uoukuqk util-macros@1.19.0
|
|
3cfh3hi gmp@6.1.1 3k4ykbe libxml2@2.9.4 7tb426s ncurses@6.0 djdthlh nettle@3.2 i3rpk4e py-numpy@1.11.1 t5lk6in xz@5.2.2
|
|
3ostwel hwloc@1.11.4 cagoem4 lz4@131 mirer2l netlib-lapack@3.6.1 js33umc openblas@0.2.19 e6uljfi py-scipy@0.18.1 asydrba zlib@1.2.8
|
|
|
|
==> Do you want to proceed? [y/n] y
|
|
|
|
$ module avail
|
|
|
|
------------------------------------------------------------------------ ~/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
|
|
bzip2-1.0.6-gcc-6.2.0-csoc2mq libsigsegv-2.10-gcc-6.2.0-g3qpmbi ncurses-6.0-gcc-6.2.0-7tb426s openmpi-2.0.1-gcc-6.2.0-s3qbtby sqlite-3.8.5-gcc-6.2.0-td3zfe7
|
|
cmake-3.5.2-gcc-6.2.0-6poypqg libtool-2.4.6-gcc-6.2.0-kiepac6 netlib-lapack-3.6.1-gcc-6.2.0-mirer2l openssl-1.0.2j-gcc-6.2.0-hibnfda util-macros-1.19.0-gcc-6.2.0-uoukuqk
|
|
curl-7.50.3-gcc-6.2.0-2ffacqm libxml2-2.9.4-gcc-6.2.0-3k4ykbe netlib-scalapack-2.0.2-gcc-6.2.0-6bqlxqy pkg-config-0.29.1-gcc-6.2.0-rslsgcs xz-5.2.2-gcc-6.2.0-t5lk6in
|
|
expat-2.2.0-gcc-6.2.0-bxqnjar lz4-131-gcc-6.2.0-cagoem4 netlib-scalapack-2.0.2-gcc-6.2.0-hpqb3dp py-nose-1.3.7-gcc-6.2.0-4gl5c42 zlib-1.2.8-gcc-6.2.0-asydrba
|
|
gmp-6.1.1-gcc-6.2.0-3cfh3hi lzma-4.32.7-gcc-6.2.0-sfmeynw netlib-scalapack-2.0.2-gcc-6.2.0-wnimqhw py-numpy-1.11.1-gcc-6.2.0-i3rpk4e
|
|
hwloc-1.11.4-gcc-6.2.0-3ostwel lzo-2.09-gcc-6.2.0-jcngz72 netlib-scalapack-2.0.2-gcc-6.2.0-wojunhq py-scipy-0.18.1-gcc-6.2.0-e6uljfi
|
|
libarchive-3.2.1-gcc-6.2.0-2b54aos m4-1.4.17-gcc-6.2.0-lhgqa6s nettle-3.2-gcc-6.2.0-djdthlh py-setuptools-25.2.0-gcc-6.2.0-hkqauaa
|
|
libpciaccess-0.13.4-gcc-6.2.0-eo2siet mpich-3.2-gcc-6.2.0-5n5xoep openblas-0.2.19-gcc-6.2.0-js33umc python-2.7.12-gcc-6.2.0-qu7rc5p
|
|
|
|
This time it is convenient to pass the option ``--delete-tree`` to the command that
|
|
regenerates the module files to instruct it to delete the existing tree and regenerate
|
|
a new one instead of overwriting the files in the existing directory.
|
|
|
|
If you pay careful attention you'll see though that we went too far in blacklisting modules:
|
|
the module for ``gcc@6.2.0`` disappeared as it was bootstrapped with ``gcc@4.8``. To specify
|
|
exceptions to the blacklist rules you can use ``whitelist``:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 3,4
|
|
|
|
modules:
|
|
tcl:
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
|
|
``whitelist`` rules always have precedence over ``blacklist`` rules. If you regenerate the modules again:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type tcl -y
|
|
|
|
you'll see that now the module for ``gcc@6.2.0`` has reappeared:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module avail gcc-6.2.0-gcc-4.8-twd5nqg
|
|
|
|
------------------------------------------------------------------------ ~/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
|
|
gcc-6.2.0-gcc-4.8-twd5nqg
|
|
|
|
-------------------------
|
|
Change module file naming
|
|
-------------------------
|
|
|
|
The next step in making module files more user-friendly is to
|
|
improve their naming scheme.
|
|
To reduce the length of the hash or remove it altogether you can
|
|
use the ``hash_length`` keyword in the configuration file:
|
|
|
|
.. TODO: give reasons to remove hashes if they are not evident enough?
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 3
|
|
|
|
modules:
|
|
tcl:
|
|
hash_length: 0
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
|
|
If you try to regenerate the module files now you will get an error:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type tcl --delete-tree -y
|
|
==> Error: Name clashes detected in module files:
|
|
|
|
file : ~/spack/share/spack/modules/linux-Ubuntu14-x86_64/netlib-scalapack-2.0.2-gcc-6.2.0
|
|
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
|
|
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
|
|
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
|
|
spec : netlib-scalapack@2.0.2%gcc@6.2.0~fpic+shared arch=linux-Ubuntu14-x86_64
|
|
|
|
==> Error: Operation aborted
|
|
|
|
.. note::
|
|
We try to check for errors upfront!
|
|
In Spack we check for errors upfront whenever possible, so don't worry about your module files:
|
|
as a name clash was detected nothing has been changed on disk.
|
|
|
|
The problem here is that without
|
|
the hashes the four different flavors of ``netlib-scalapack`` map to the same module file
|
|
name. We have the possibility to add suffixes to differentiate them:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 9-11,14-17
|
|
|
|
modules:
|
|
tcl:
|
|
hash_length: 0
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
netlib-scalapack:
|
|
suffixes:
|
|
'^openmpi': openmpi
|
|
'^mpich': mpich
|
|
|
|
As you can see it is possible to specify rules that applies only to a
|
|
restricted set of packages using :ref:`anonymous specs <anonymous_specs>`.
|
|
Regenerating module files now we obtain:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type tcl --delete-tree -y
|
|
==> Regenerating tcl module files
|
|
$ module avail
|
|
|
|
------------------------------------------------------------------------ ~/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
|
|
bzip2-1.0.6-gcc-6.2.0 libpciaccess-0.13.4-gcc-6.2.0 mpich-3.2-gcc-6.2.0 openblas-0.2.19-gcc-6.2.0 python-2.7.12-gcc-6.2.0
|
|
cmake-3.5.2-gcc-6.2.0 libsigsegv-2.10-gcc-6.2.0 ncurses-6.0-gcc-6.2.0 openmpi-2.0.1-gcc-6.2.0 sqlite-3.8.5-gcc-6.2.0
|
|
curl-7.50.3-gcc-6.2.0 libtool-2.4.6-gcc-6.2.0 netlib-lapack-3.6.1-gcc-6.2.0 openssl-1.0.2j-gcc-6.2.0 util-macros-1.19.0-gcc-6.2.0
|
|
expat-2.2.0-gcc-6.2.0 libxml2-2.9.4-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-mpich pkg-config-0.29.1-gcc-6.2.0 xz-5.2.2-gcc-6.2.0
|
|
gcc-6.2.0-gcc-4.8 lz4-131-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-netlib-openmpi py-nose-1.3.7-gcc-6.2.0 zlib-1.2.8-gcc-6.2.0
|
|
gmp-6.1.1-gcc-6.2.0 lzma-4.32.7-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-mpich py-numpy-1.11.1-gcc-6.2.0-openblas
|
|
hwloc-1.11.4-gcc-6.2.0 lzo-2.09-gcc-6.2.0 netlib-scalapack-2.0.2-gcc-6.2.0-openblas-openmpi py-scipy-0.18.1-gcc-6.2.0-openblas
|
|
libarchive-3.2.1-gcc-6.2.0 m4-1.4.17-gcc-6.2.0 nettle-3.2-gcc-6.2.0 py-setuptools-25.2.0-gcc-6.2.0
|
|
|
|
Finally we can set a ``naming_scheme`` to prevent users from loading
|
|
modules that refer to different flavors of the same library/application:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 4,10,11
|
|
|
|
modules:
|
|
tcl:
|
|
hash_length: 0
|
|
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
conflict:
|
|
- '${PACKAGE}'
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
netlib-scalapack:
|
|
suffixes:
|
|
'^openmpi': openmpi
|
|
'^mpich': mpich
|
|
|
|
The final result should look like:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module avail
|
|
|
|
------------------------------------------------------------------------ ~/spack/share/spack/modules/linux-Ubuntu14-x86_64 ------------------------------------------------------------------------
|
|
bzip2/1.0.6-gcc-6.2.0 libpciaccess/0.13.4-gcc-6.2.0 mpich/3.2-gcc-6.2.0 openblas/0.2.19-gcc-6.2.0 python/2.7.12-gcc-6.2.0
|
|
cmake/3.5.2-gcc-6.2.0 libsigsegv/2.10-gcc-6.2.0 ncurses/6.0-gcc-6.2.0 openmpi/2.0.1-gcc-6.2.0 sqlite/3.8.5-gcc-6.2.0
|
|
curl/7.50.3-gcc-6.2.0 libtool/2.4.6-gcc-6.2.0 netlib-lapack/3.6.1-gcc-6.2.0 openssl/1.0.2j-gcc-6.2.0 util-macros/1.19.0-gcc-6.2.0
|
|
expat/2.2.0-gcc-6.2.0 libxml2/2.9.4-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-mpich pkg-config/0.29.1-gcc-6.2.0 xz/5.2.2-gcc-6.2.0
|
|
gcc/6.2.0-gcc-4.8 lz4/131-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-netlib-openmpi py-nose/1.3.7-gcc-6.2.0 zlib/1.2.8-gcc-6.2.0
|
|
gmp/6.1.1-gcc-6.2.0 lzma/4.32.7-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-mpich py-numpy/1.11.1-gcc-6.2.0-openblas
|
|
hwloc/1.11.4-gcc-6.2.0 lzo/2.09-gcc-6.2.0 netlib-scalapack/2.0.2-gcc-6.2.0-openblas-openmpi (D) py-scipy/0.18.1-gcc-6.2.0-openblas
|
|
libarchive/3.2.1-gcc-6.2.0 m4/1.4.17-gcc-6.2.0 nettle/3.2-gcc-6.2.0 py-setuptools/25.2.0-gcc-6.2.0
|
|
|
|
.. note::
|
|
TCL specific directive
|
|
The directives ``naming_scheme`` and ``conflict`` are TCL specific and do not apply
|
|
to the ``dotkit`` or ``lmod`` sections in the configuration file.
|
|
|
|
------------------------------------
|
|
Add custom environment modifications
|
|
------------------------------------
|
|
|
|
At many sites it is customary to set an environment variable in a
|
|
package's module file that points to the folder in which the package
|
|
is installed. You can achieve this with Spack by adding an
|
|
``environment`` directive to the configuration file:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 17-19
|
|
|
|
modules:
|
|
tcl:
|
|
hash_length: 0
|
|
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
conflict:
|
|
- '${PACKAGE}'
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
environment:
|
|
set:
|
|
'${PACKAGE}_ROOT': '${PREFIX}'
|
|
netlib-scalapack:
|
|
suffixes:
|
|
'^openmpi': openmpi
|
|
'^mpich': mpich
|
|
|
|
There are many variable tokens available to use in the ``environment``
|
|
and ``naming_scheme`` directives, such as ``${PACKAGE}``,
|
|
``${VERSION}``, etc. (see the :meth:`~spack.spec.Spec.format` API
|
|
documentation for the complete list).
|
|
|
|
Regenerating the module files should result in something like:
|
|
|
|
.. code-block:: console
|
|
:emphasize-lines: 14
|
|
|
|
$ spack module refresh -y --module-type tcl
|
|
==> Regenerating tcl module files
|
|
|
|
$ module show gcc
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
~/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8:
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
whatis("gcc @6.2.0 ")
|
|
prepend_path("PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
|
|
prepend_path("CMAKE_PREFIX_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
|
|
prepend_path("MANPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
|
|
prepend_path("PKG_CONFIG_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
|
|
prepend_path("LD_LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
|
|
setenv("GCC_ROOT","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u")
|
|
conflict("gcc")
|
|
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
|
|
Fortran, and Java.
|
|
]])
|
|
|
|
As you can see, the ``gcc`` module has the environment variable ``GCC_ROOT`` set.
|
|
|
|
Sometimes it's also useful to apply environment modifications selectively and target
|
|
only certain packages. You can, for instance set the common variables ``CC``, ``CXX``,
|
|
etc. in the ``gcc`` module file and apply other custom modifications to the
|
|
``openmpi`` modules as follows:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 20-32
|
|
|
|
modules:
|
|
tcl:
|
|
hash_length: 0
|
|
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
conflict:
|
|
- '${PACKAGE}'
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
environment:
|
|
set:
|
|
'${PACKAGE}_ROOT': '${PREFIX}'
|
|
gcc:
|
|
environment:
|
|
set:
|
|
CC: gcc
|
|
CXX: g++
|
|
FC: gfortran
|
|
F90: gfortran
|
|
F77: gfortran
|
|
openmpi:
|
|
environment:
|
|
set:
|
|
SLURM_MPI_TYPE: pmi2
|
|
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
|
|
netlib-scalapack:
|
|
suffixes:
|
|
'^openmpi': openmpi
|
|
'^mpich': mpich
|
|
|
|
This time we will be more selective and regenerate only the ``gcc`` and
|
|
``openmpi`` module files:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh -y --module-type tcl gcc
|
|
==> Regenerating tcl module files
|
|
|
|
$ spack module refresh -y --module-type tcl openmpi
|
|
==> Regenerating tcl module files
|
|
|
|
$ module show gcc
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
~/spack/share/spack/modules/linux-Ubuntu14-x86_64/gcc/6.2.0-gcc-4.8:
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
whatis("gcc @6.2.0 ")
|
|
prepend_path("PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/bin")
|
|
prepend_path("CMAKE_PREFIX_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/")
|
|
prepend_path("MANPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/share/man")
|
|
prepend_path("PKG_CONFIG_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64/pkgconfig")
|
|
prepend_path("LD_LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u/lib64")
|
|
setenv("GCC_ROOT","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-4.8/gcc-6.2.0-twd5nqg33hrrssqclcfi5k42eccwxz5u")
|
|
setenv("CC","gcc")
|
|
setenv("CXX","g++")
|
|
setenv("F90","gfortran")
|
|
setenv("FC","gfortran")
|
|
setenv("F77","gfortran")
|
|
conflict("gcc")
|
|
help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
|
|
Fortran, and Java.
|
|
]])
|
|
|
|
$ module show openmpi
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
~/spack/share/spack/modules/linux-Ubuntu14-x86_64/openmpi/2.0.1-gcc-6.2.0:
|
|
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
whatis("openmpi @2.0.1 ")
|
|
prepend_path("PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/bin")
|
|
prepend_path("CMAKE_PREFIX_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/")
|
|
prepend_path("LD_LIBRARY_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib")
|
|
prepend_path("PKG_CONFIG_PATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/lib/pkgconfig")
|
|
prepend_path("MANPATH","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w/share/man")
|
|
setenv("SLURM_MPI_TYPE","pmi2")
|
|
setenv("OMPI_MCA_BTL_OPENIB_WARN_DEFAULT_GID_PREFIX","0")
|
|
setenv("OPENMPI_ROOT","~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/openmpi-2.0.1-s3qbtbyh3y5y4gkchmhcuak7th44l53w")
|
|
conflict("openmpi")
|
|
help([[The Open MPI Project is an open source Message Passing Interface
|
|
implementation that is developed and maintained by a consortium of
|
|
academic, research, and industry partners. Open MPI is therefore able to
|
|
combine the expertise, technologies, and resources from all across the
|
|
High Performance Computing community in order to build the best MPI
|
|
library available. Open MPI offers advantages for system and software
|
|
vendors, application developers and computer science researchers.
|
|
]])
|
|
|
|
|
|
---------------------
|
|
Autoload dependencies
|
|
---------------------
|
|
|
|
Spack can also generate module files that contain code to load the
|
|
dependencies automatically. You can, for instance generate python
|
|
modules that load their dependencies by adding the ``autoload``
|
|
directive and assigning it the value ``direct``:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 3,38,39
|
|
|
|
modules:
|
|
tcl:
|
|
verbose: True
|
|
hash_length: 0
|
|
naming_scheme: '${PACKAGE}/${VERSION}-${COMPILERNAME}-${COMPILERVER}'
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
conflict:
|
|
- '${PACKAGE}'
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
environment:
|
|
set:
|
|
'${PACKAGE}_ROOT': '${PREFIX}'
|
|
gcc:
|
|
environment:
|
|
set:
|
|
CC: gcc
|
|
CXX: g++
|
|
FC: gfortran
|
|
F90: gfortran
|
|
F77: gfortran
|
|
openmpi:
|
|
environment:
|
|
set:
|
|
SLURM_MPI_TYPE: pmi2
|
|
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
|
|
netlib-scalapack:
|
|
suffixes:
|
|
'^openmpi': openmpi
|
|
'^mpich': mpich
|
|
^python:
|
|
autoload: 'direct'
|
|
|
|
and regenerating the module files for every package that depends on ``python``:
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh -y --module-type tcl ^python
|
|
==> Regenerating tcl module files
|
|
|
|
Now the ``py-scipy`` module will be:
|
|
|
|
.. code-block:: tcl
|
|
|
|
#%Module1.0
|
|
## Module file created by spack (https://github.com/LLNL/spack) on 2016-11-02 20:53:21.283547
|
|
##
|
|
## py-scipy@0.18.1%gcc@6.2.0 arch=linux-Ubuntu14-x86_64-e6uljfi
|
|
##
|
|
module-whatis "py-scipy @0.18.1"
|
|
|
|
proc ModulesHelp { } {
|
|
puts stderr "SciPy (pronounced "Sigh Pie") is a Scientific Library for Python. It"
|
|
puts stderr "provides many user-friendly and efficient numerical routines such as"
|
|
puts stderr "routines for numerical integration and optimization."
|
|
}
|
|
|
|
if ![ is-loaded python/2.7.12-gcc-6.2.0 ] {
|
|
puts stderr "Autoloading python/2.7.12-gcc-6.2.0"
|
|
module load python/2.7.12-gcc-6.2.0
|
|
}
|
|
|
|
if ![ is-loaded openblas/0.2.19-gcc-6.2.0 ] {
|
|
puts stderr "Autoloading openblas/0.2.19-gcc-6.2.0"
|
|
module load openblas/0.2.19-gcc-6.2.0
|
|
}
|
|
|
|
if ![ is-loaded py-numpy/1.11.1-gcc-6.2.0-openblas ] {
|
|
puts stderr "Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas"
|
|
module load py-numpy/1.11.1-gcc-6.2.0-openblas
|
|
}
|
|
|
|
prepend-path CMAKE_PREFIX_PATH "~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/"
|
|
prepend-path LD_LIBRARY_PATH "~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib"
|
|
prepend-path PYTHONPATH "~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh/lib/python2.7/site-packages"
|
|
setenv PY_SCIPY_ROOT "~/spack/opt/spack/linux-Ubuntu14-x86_64/gcc-6.2.0/py-scipy-0.18.1-e6uljfiffgym4xvj6wveevqxfqnfb3gh"
|
|
conflict py-scipy
|
|
|
|
and will contain code to autoload all the dependencies:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module load py-scipy
|
|
Autoloading python/2.7.12-gcc-6.2.0
|
|
Autoloading openblas/0.2.19-gcc-6.2.0
|
|
Autoloading py-numpy/1.11.1-gcc-6.2.0-openblas
|
|
|
|
In case messages are unwanted during the autoload procedure, it will be
|
|
sufficient to omit the line setting ``verbose: True`` in the configuration file above.
|
|
|
|
-----------------------------
|
|
Lua hierarchical module files
|
|
-----------------------------
|
|
|
|
In the final part of this tutorial you will modify ``modules.yaml`` to generate
|
|
Lua hierarchical module files. You will see that most of the directives used before
|
|
are also valid in the ``lmod`` context.
|
|
|
|
^^^^^^^^^^^^^^^^^
|
|
Core/Compiler/MPI
|
|
^^^^^^^^^^^^^^^^^
|
|
|
|
.. warning::
|
|
Only LMod supports Lua hierarchical module files
|
|
For this part of the tutorial you need to be using LMod to
|
|
manage your environment.
|
|
|
|
The most common hierarchy is the so called ``Core/Compiler/MPI``. To have an idea
|
|
how a hierarchy is organized you may refer to the
|
|
`Lmod guide <http://lmod.readthedocs.io/en/latest/080_hierarchy.html>`_.
|
|
Since ``lmod`` is not enabled by default, you need to add it to the list of
|
|
enabled module file generators. The other things you need to do are:
|
|
|
|
- change the ``tcl`` tag to ``lmod``
|
|
- remove ``tcl`` specific directives (``naming_scheme`` and ``conflict``)
|
|
- set which compilers are considered ``core``
|
|
- remove the ``mpi`` related suffixes (as they will be substituted by hierarchies)
|
|
|
|
After modifications the configuration file will be:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 2-8
|
|
|
|
modules:
|
|
enable::
|
|
- lmod
|
|
lmod:
|
|
core_compilers:
|
|
- 'gcc@4.8'
|
|
hierarchy:
|
|
- mpi
|
|
hash_length: 0
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
all:
|
|
suffixes:
|
|
'^openblas': openblas
|
|
'^netlib-lapack': netlib
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
environment:
|
|
set:
|
|
'${PACKAGE}_ROOT': '${PREFIX}'
|
|
gcc:
|
|
environment:
|
|
set:
|
|
CC: gcc
|
|
CXX: g++
|
|
FC: gfortran
|
|
F90: gfortran
|
|
F77: gfortran
|
|
openmpi:
|
|
environment:
|
|
set:
|
|
SLURM_MPI_TYPE: pmi2
|
|
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
|
|
|
|
|
|
.. note::
|
|
The double colon
|
|
The double colon after ``enable`` is intentional and it serves the
|
|
purpose of overriding the default list of enabled generators so
|
|
that only ``lmod`` will be active (see :ref:`the reference
|
|
manual <config-overrides>` for a more detailed explanation of
|
|
config scopes). If a single colon is used, it will append instead
|
|
of override.
|
|
|
|
The directive ``core_compilers`` accepts a list of compilers; everything built
|
|
using these compilers will create a module in the ``Core`` part of the hierarchy. It is
|
|
common practice to put the OS provided compilers in the list and only build common utilities
|
|
and other compilers in ``Core``.
|
|
|
|
If you regenerate the module files
|
|
|
|
.. code-block:: console
|
|
|
|
$ spack module refresh --module-type lmod --delete-tree -y
|
|
|
|
and update ``MODULEPATH`` to point to the ``Core`` folder, and
|
|
list the available modules, you'll see:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module unuse ~/spack/share/spack/modules/linux-Ubuntu14-x86_64
|
|
$ module use ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core
|
|
$ module avail
|
|
|
|
----------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
|
|
gcc/6.2.0
|
|
|
|
The only module visible now is ``gcc``. Loading that you will make
|
|
visible the ``Compiler`` part of the software stack that was built with ``gcc/6.2.0``:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module load gcc
|
|
$ module avail
|
|
|
|
-------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
|
|
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0
|
|
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2
|
|
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8
|
|
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5
|
|
|
|
----------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
|
|
gcc/6.2.0 (L)
|
|
|
|
The same holds true for the ``MPI`` part of the stack, that you can enable by loading
|
|
either ``mpich`` or ``openmpi``. The nice features of LMod will become evident
|
|
once you'll try switching among different stacks:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module load mpich
|
|
$ module avail
|
|
|
|
----------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/mpich/3.2-5n5xoep/gcc/6.2.0 ------------------------------------------------------------
|
|
netlib-scalapack/2.0.2-netlib netlib-scalapack/2.0.2-openblas (D)
|
|
|
|
-------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
|
|
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j py-scipy/0.18.1-openblas util-macros/1.19.0
|
|
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 py-setuptools/25.2.0 xz/5.2.2
|
|
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 (L) openblas/0.2.19 py-nose/1.3.7 python/2.7.12 zlib/1.2.8
|
|
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 py-numpy/1.11.1-openblas sqlite/3.8.5
|
|
|
|
----------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
|
|
gcc/6.2.0 (L)
|
|
|
|
$ module load openblas netlib-scalapack/2.0.2-openblas
|
|
$ module list
|
|
|
|
Currently Loaded Modules:
|
|
1) gcc/6.2.0 2) mpich/3.2 3) openblas/0.2.19 4) netlib-scalapack/2.0.2-openblas
|
|
|
|
$ module load openmpi
|
|
|
|
Lmod is automatically replacing "mpich/3.2" with "openmpi/2.0.1"
|
|
|
|
|
|
Due to MODULEPATH changes the following have been reloaded:
|
|
1) netlib-scalapack/2.0.2-openblas
|
|
|
|
This layout is already a great improvement over the usual non-hierarchical layout,
|
|
but it still has an asymmetry: ``LAPACK`` providers are semantically the same as ``MPI``
|
|
providers, but they are still not part of the hierarchy. We'll see a possible solution
|
|
next.
|
|
|
|
.. Activate lmod and turn the previous modifications into lmod:
|
|
Add core compilers
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
Extend the hierarchy to other virtual providers
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
.. warning::
|
|
This is an experimental feature
|
|
Having a hierarchy deeper than ``Core``/``Compiler``/``MPI`` is an experimental
|
|
feature, still not fully supported by ``module spider``,
|
|
see `here <https://github.com/TACC/Lmod/issues/114>`_. Furthermore its use
|
|
with hierarchies more complex than ``Core``/``Compiler``/``MPI``/``LAPACK``
|
|
has not been thoroughly tested in production environments.
|
|
|
|
Spack permits you to generate Lua hierarchical module files where users
|
|
can add an arbitrary list of virtual providers to the triplet
|
|
``Core``/``Compiler``/``MPI``. A configuration file like:
|
|
|
|
.. code-block:: yaml
|
|
:emphasize-lines: 9
|
|
|
|
modules:
|
|
enable::
|
|
- lmod
|
|
lmod:
|
|
core_compilers:
|
|
- 'gcc@4.8'
|
|
hierarchy:
|
|
- mpi
|
|
- lapack
|
|
hash_length: 0
|
|
whitelist:
|
|
- gcc
|
|
blacklist:
|
|
- '%gcc@4.8'
|
|
- readline
|
|
all:
|
|
filter:
|
|
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
|
environment:
|
|
set:
|
|
'${PACKAGE}_ROOT': '${PREFIX}'
|
|
gcc:
|
|
environment:
|
|
set:
|
|
CC: gcc
|
|
CXX: g++
|
|
FC: gfortran
|
|
F90: gfortran
|
|
F77: gfortran
|
|
openmpi:
|
|
environment:
|
|
set:
|
|
SLURM_MPI_TYPE: pmi2
|
|
OMPI_MCA_btl_openib_warn_default_gid_prefix: '0'
|
|
|
|
will add ``lapack`` providers to the mix. After the usual regeneration of module files:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module purge
|
|
$ spack module refresh --module-type lmod --delete-tree -y
|
|
==> Regenerating lmod module files
|
|
|
|
you will have something like:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module load gcc
|
|
$ module load openblas
|
|
$ module load openmpi
|
|
$ module avail
|
|
|
|
--------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/openmpi/2.0.1-s3qbtby/gcc/6.2.0 ----------------------------------------------
|
|
netlib-scalapack/2.0.2
|
|
|
|
-------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/openblas/0.2.19-js33umc/gcc/6.2.0 ---------------------------------------------------------
|
|
py-numpy/1.11.1 py-scipy/0.18.1
|
|
|
|
-------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/gcc/6.2.0 ---------------------------------------------------------------------
|
|
binutils/2.27 curl/7.50.3 hwloc/1.11.4 libtool/2.4.6 lzo/2.09 netlib-lapack/3.6.1 openssl/1.0.2j python/2.7.12 zlib/1.2.8
|
|
bison/3.0.4 expat/2.2.0 libarchive/3.2.1 libxml2/2.9.4 m4/1.4.17 nettle/3.2 pkg-config/0.29.1 sqlite/3.8.5
|
|
bzip2/1.0.6 flex/2.6.0 libpciaccess/0.13.4 lz4/131 mpich/3.2 openblas/0.2.19 (L) py-nose/1.3.7 util-macros/1.19.0
|
|
cmake/3.6.1 gmp/6.1.1 libsigsegv/2.10 lzma/4.32.7 ncurses/6.0 openmpi/2.0.1 (L) py-setuptools/25.2.0 xz/5.2.2
|
|
|
|
----------------------------------------------------------------------- ~/spack/share/spack/lmod/linux-Ubuntu14-x86_64/Core -----------------------------------------------------------------------
|
|
gcc/6.2.0 (L)
|
|
|
|
Now both the ``MPI`` and the ``LAPACK`` providers are handled by LMod as hierarchies:
|
|
|
|
.. code-block:: console
|
|
|
|
$ module load py-numpy netlib-scalapack
|
|
$ module load mpich
|
|
|
|
Lmod is automatically replacing "openmpi/2.0.1" with "mpich/3.2"
|
|
|
|
|
|
Due to MODULEPATH changes the following have been reloaded:
|
|
1) netlib-scalapack/2.0.2
|
|
|
|
$ module load netlib-lapack
|
|
|
|
Lmod is automatically replacing "openblas/0.2.19" with "netlib-lapack/3.6.1"
|
|
|
|
|
|
Inactive Modules:
|
|
1) py-numpy
|
|
|
|
Due to MODULEPATH changes the following have been reloaded:
|
|
1) netlib-scalapack/2.0.2
|
|
|
|
making the use of tags to differentiate them unnecessary.
|
|
Note that because we only compiled ``py-numpy`` with ``openblas`` the module
|
|
is made inactive when we switch the ``LAPACK`` provider. The user
|
|
environment will now be consistent by design!
|