zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Grouped all the module commands under spack module

Browse Source 
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.

Unit tests have been refactored to match the new command structure.
This commit is contained in:
Massimiliano Culpo
2018-07-20 10:30:36 +02:00
committed by Todd Gamblin
parent 7cbe1aedb7
commit 35cef16974
14 changed files with 272 additions and 396 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
lib/spack/docs/developer_guide.rst
View File

@@ -314,7 +314,7 @@ See the `Argparse documentation <https://docs.python.org/2.7/library/argparse.ht
for more details on how to add arguments. for more details on how to add arguments.
Some commands have a set of subcommands, like ``spack compiler find`` or Some commands have a set of subcommands, like ``spack compiler find`` or
``spack lmod refresh``. You can add subparsers to your parser to handle ``spack module lmod refresh``. You can add subparsers to your parser to handle
this. Check out ``spack edit --command compiler`` for an example of this. this. Check out ``spack edit --command compiler`` for an example of this.
A lot of commands take the same arguments and flags. These arguments should A lot of commands take the same arguments and flags. These arguments should

4
lib/spack/docs/getting_started.rst
View File

@@ -816,7 +816,7 @@ This problem is related to OpenSSL, and in some cases might be solved
by installing a new version of ``git`` and ``openssl``: by installing a new version of ``git`` and ``openssl``:
#. Run ``spack install git`` #. Run ``spack install git``
#. Add the output of ``spack tcl loads git`` to your ``.bashrc``. #. Add the output of ``spack module tcl loads git`` to your ``.bashrc``.
If this doesn't work, it is also possible to disable checking of SSL If this doesn't work, it is also possible to disable checking of SSL
certificates by using: certificates by using:
@@ -861,7 +861,7 @@ or alternately:
.. code-block:: console .. code-block:: console
$ spack tcl loads curl >>~/.bashrc $ spack module tcl loads curl >>~/.bashrc
or if environment modules don't work: or if environment modules don't work:

30
lib/spack/docs/module_file_support.rst
View File

@@ -180,9 +180,9 @@ To identify just the one built with the Intel compiler.
.. _cmd-spack-module-loads: .. _cmd-spack-module-loads:
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
``spack tcl loads`` ``spack module tcl loads``
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^
In some cases, it is desirable to load not just a module, but also all In some cases, it is desirable to load not just a module, but also all
the modules it depends on. This is not required for most modules the modules it depends on. This is not required for most modules
@@ -195,13 +195,13 @@ Scripts to load modules recursively may be made with the command:
.. code-block:: console .. code-block:: console
$ spack tcl loads --dependencies <spec> $ spack module tcl loads --dependencies <spec>
An equivalent alternative using `process substitution <http://tldp.org/LDP/abs/html/process-sub.html>`_ is: An equivalent alternative using `process substitution <http://tldp.org/LDP/abs/html/process-sub.html>`_ is:
.. code-block :: console .. code-block :: console
$ source <( spack tcl loads --dependencies <spec> ) $ source <( spack module tcl loads --dependencies <spec> )
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -211,12 +211,12 @@ Module Commands for Shell Scripts
Although Spack is flexible, the ``module`` command is much faster. Although Spack is flexible, the ``module`` command is much faster.
This could become an issue when emitting a series of ``spack load`` This could become an issue when emitting a series of ``spack load``
commands inside a shell script. By adding the ``--shell`` flag, commands inside a shell script. By adding the ``--shell`` flag,
``spack tcl find`` may also be used to generate code that can be ``spack module tcl find`` may also be used to generate code that can be
cut-and-pasted into a shell script. For example: cut-and-pasted into a shell script. For example:
.. code-block:: console .. code-block:: console
$ spack tcl loads --dependencies py-numpy git $ spack module tcl loads --dependencies py-numpy git
# bzip2@1.0.6%gcc@4.9.3=linux-x86_64 # bzip2@1.0.6%gcc@4.9.3=linux-x86_64
module load bzip2-1.0.6-gcc-4.9.3-ktnrhkrmbbtlvnagfatrarzjojmkvzsx module load bzip2-1.0.6-gcc-4.9.3-ktnrhkrmbbtlvnagfatrarzjojmkvzsx
# ncurses@6.0%gcc@4.9.3=linux-x86_64 # ncurses@6.0%gcc@4.9.3=linux-x86_64
@@ -256,9 +256,9 @@ Module Prefixes
^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^
On some systems, modules are automatically prefixed with a certain On some systems, modules are automatically prefixed with a certain
string; ``spack tcl loads`` needs to know about that prefix when it string; ``spack module tcl loads`` needs to know about that prefix when it
issues ``module load`` commands. Add the ``--prefix`` option to your issues ``module load`` commands. Add the ``--prefix`` option to your
``spack tcl loads`` commands if this is necessary. ``spack module tcl loads`` commands if this is necessary.
For example, consider the following on one system: For example, consider the following on one system:
@@ -267,11 +267,11 @@ For example, consider the following on one system:
$ module avail $ module avail
linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y
$ spack tcl loads antlr # WRONG! $ spack module tcl loads antlr # WRONG!
# antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64 # antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64
module load antlr-2.7.7-gcc-5.3.0-bdpl46y module load antlr-2.7.7-gcc-5.3.0-bdpl46y
$ spack tcl loads --prefix linux-SuSE11-x86_64/ antlr $ spack module tcl loads --prefix linux-SuSE11-x86_64/ antlr
# antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64 # antlr@2.7.7%gcc@5.3.0~csharp+cxx~java~python arch=linux-SuSE11-x86_64
module load linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y module load linux-SuSE11-x86_64/antlr-2.7.7-gcc-5.3.0-bdpl46y
@@ -626,9 +626,9 @@ Maintaining Module Files
Each type of module file has a command with the same name associated Each type of module file has a command with the same name associated
with it. The actions these commands permit are usually associated with it. The actions these commands permit are usually associated
with the maintenance of a production environment. Here's, for instance, with the maintenance of a production environment. Here's, for instance,
a sample of the features of the ``spack tcl`` command: a sample of the features of the ``spack module tcl`` command:
.. command-output:: spack tcl --help .. command-output:: spack module tcl --help
.. _cmd-spack-module-refresh: .. _cmd-spack-module-refresh:
@@ -639,7 +639,7 @@ Refresh the set of modules
The subcommand that regenerates module files to update their content or The subcommand that regenerates module files to update their content or
their layout is ``refresh``: their layout is ``refresh``:
.. command-output:: spack tcl refresh --help .. command-output:: spack module tcl refresh --help
A set of packages can be selected using anonymous specs for the optional A set of packages can be selected using anonymous specs for the optional
``constraint`` positional argument. Optionally the entire tree can be deleted ``constraint`` positional argument. Optionally the entire tree can be deleted
@@ -654,7 +654,7 @@ Delete module files
If instead what you need is just to delete a few module files, then the right If instead what you need is just to delete a few module files, then the right
subcommand is ``rm``: subcommand is ``rm``:
.. command-output:: spack tcl rm --help .. command-output:: spack module tcl rm --help
.. note:: .. note::
We care about your module files! We care about your module files!

103
lib/spack/docs/tutorial_modules.rst
View File

@@ -166,7 +166,7 @@ The fastest way to set-up your environment is to :ref:`use a Docker image <workf
.. code-block:: console .. code-block:: console
$ docker pull alalazo/spack:module_tutorial $ docker pull alalazo/spack:module_tutorial
$ docker run --rm -h module-file-tutorial -it alalazo/spack:module_tutorial $ docker run --rm -h module-file-tutorial -it spack/module-tutorial:latest
root@module-file-tutorial:/# root@module-file-tutorial:/#
If you arrived at this point you should be ready to start, as all the software needed is If you arrived at this point you should be ready to start, as all the software needed is
@@ -185,7 +185,7 @@ pre-installed in the image:
-- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 -------------------------
bzip2@1.0.6 mpich@3.2.1 netlib-scalapack@2.0.2 netlib-scalapack@2.0.2 openssl@1.0.2o py-scipy@1.1.0 readline@7.0 bzip2@1.0.6 mpich@3.2.1 netlib-scalapack@2.0.2 netlib-scalapack@2.0.2 openssl@1.0.2o py-scipy@1.1.0 readline@7.0
cmake@3.11.4 ncurses@6.1 netlib-scalapack@2.0.2 openblas@0.3.0 pkgconf@1.4.2 py-setuptools@39.2.0 sqlite@3.23.1 cmake@3.12.0 ncurses@6.1 netlib-scalapack@2.0.2 openblas@0.3.0 pkgconf@1.4.2 py-setuptools@39.2.0 sqlite@3.23.1
gdbm@1.14.1 netlib-lapack@3.8.0 netlib-scalapack@2.0.2 openmpi@1.10.2 py-numpy@1.14.3 python@2.7.15 zlib@1.2.11 gdbm@1.14.1 netlib-lapack@3.8.0 netlib-scalapack@2.0.2 openmpi@1.10.2 py-numpy@1.14.3 python@2.7.15 zlib@1.2.11
Go to :ref:`module_file_tutorial_non_hierarchical` to proceed with the tutorial. Go to :ref:`module_file_tutorial_non_hierarchical` to proceed with the tutorial.
@@ -364,17 +364,17 @@ If you arrived to this point you should have an environment that looks similar t
autoconf-2.69-gcc-5.4.0-cbvv5rj lua-luaposix-33.4.0-gcc-5.4.0-i7w7ynf perl-5.26.2-gcc-5.4.0-n2k4mza autoconf-2.69-gcc-5.4.0-cbvv5rj lua-luaposix-33.4.0-gcc-5.4.0-i7w7ynf perl-5.26.2-gcc-5.4.0-n2k4mza
automake-1.16.1-gcc-5.4.0-lykrrr4 m4-1.4.18-gcc-5.4.0-3z33ecf pkgconf-1.4.2-gcc-5.4.0-fovrh7a automake-1.16.1-gcc-5.4.0-lykrrr4 m4-1.4.18-gcc-5.4.0-3z33ecf pkgconf-1.4.2-gcc-5.4.0-fovrh7a
bzip2-1.0.6-gcc-7.2.0-mwamumj mpc-1.1.0-gcc-5.4.0-56lbd3h pkgconf-1.4.2-gcc-7.2.0-yoxwmgb bzip2-1.0.6-gcc-7.2.0-mwamumj mpc-1.1.0-gcc-5.4.0-56lbd3h pkgconf-1.4.2-gcc-7.2.0-yoxwmgb
cmake-3.11.4-gcc-7.2.0-h24ofjs mpfr-4.0.1-gcc-5.4.0-dy5r7hi py-numpy-1.14.3-gcc-7.2.0-t3loxvu cmake-3.12.0-gcc-7.2.0-6ovorxs mpfr-4.0.1-gcc-5.4.0-dy5r7hi py-numpy-1.14.3-gcc-7.2.0-t3loxvu
gcc-7.2.0-gcc-5.4.0-wwhgyej mpich-3.2.1-gcc-7.2.0-vt5xcat py-scipy-1.1.0-gcc-7.2.0-fdiryak gcc-7.2.0-gcc-5.4.0-wwhgyej mpich-3.2.1-gcc-7.2.0-vt5xcat py-scipy-1.1.0-gcc-7.2.0-fdiryak
gdbm-1.14.1-gcc-5.4.0-q4fpyuo ncurses-6.1-gcc-5.4.0-3o765ou py-setuptools-39.2.0-gcc-7.2.0-jqhycal gdbm-1.14.1-gcc-5.4.0-q4fpyuo ncurses-6.1-gcc-5.4.0-3o765ou py-setuptools-39.2.0-gcc-7.2.0-jqhycal
gdbm-1.14.1-gcc-7.2.0-zk5lhob ncurses-6.1-gcc-7.2.0-xcgzqdv python-2.7.15-gcc-7.2.0-c7pnzul gdbm-1.14.1-gcc-7.2.0-zk5lhob ncurses-6.1-gcc-7.2.0-xcgzqdv python-2.7.15-gcc-7.2.0-c7pnzul
git-2.9.4-gcc-5.4.0-mkaoyhz netlib-lapack-3.8.0-gcc-7.2.0-bcikpen readline-7.0-gcc-5.4.0-nxhwrg7 git-2.9.4-gcc-5.4.0-mkaoyhz netlib-lapack-3.8.0-gcc-7.2.0-7apabqu readline-7.0-gcc-5.4.0-nxhwrg7
gmp-6.1.2-gcc-5.4.0-qc4qcfz netlib-scalapack-2.0.2-gcc-7.2.0-d3lertf readline-7.0-gcc-7.2.0-ccruj2i gmp-6.1.2-gcc-5.4.0-qc4qcfz netlib-scalapack-2.0.2-gcc-7.2.0-3bz5rxx readline-7.0-gcc-7.2.0-ccruj2i
isl-0.19-gcc-5.4.0-hsl7f52 netlib-scalapack-2.0.2-gcc-7.2.0-jae3ilo sqlite-3.23.1-gcc-7.2.0-5ltus3a isl-0.19-gcc-5.4.0-hsl7f52 netlib-scalapack-2.0.2-gcc-7.2.0-6i5qsqx sqlite-3.23.1-gcc-7.2.0-5ltus3a
libsigsegv-2.11-gcc-5.4.0-fypapcp netlib-scalapack-2.0.2-gcc-7.2.0-lqfhvfh tcl-8.6.8-gcc-5.4.0-qhwyccy libsigsegv-2.11-gcc-5.4.0-fypapcp netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog tcl-8.6.8-gcc-5.4.0-qhwyccy
libtool-2.4.6-gcc-5.4.0-o2pfwjf netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog zlib-1.2.11-gcc-5.4.0-5nus6kn libtool-2.4.6-gcc-5.4.0-o2pfwjf netlib-scalapack-2.0.2-gcc-7.2.0-z52ltyy zlib-1.2.11-gcc-5.4.0-5nus6kn
lmod-7.7.29-gcc-5.4.0-wl6mywv openblas-0.3.0-gcc-7.2.0-pdatzbi zlib-1.2.11-gcc-7.2.0-ezuwp4p lmod-7.7.29-gcc-5.4.0-wl6mywv openblas-0.3.0-gcc-7.2.0-pdatzbi zlib-1.2.11-gcc-7.2.0-ezuwp4p
lua-5.3.4-gcc-5.4.0-izvaota openmpi-1.10.2-gcc-7.2.0-2h6xmxh lua-5.3.4-gcc-5.4.0-izvaota openmpi-1.10.2-gcc-7.2.0-6oewzwj
lua-luafilesystem-1_6_3-gcc-5.4.0-ywlmaou openssl-1.0.2o-gcc-7.2.0-cvldq3v lua-luafilesystem-1_6_3-gcc-5.4.0-ywlmaou openssl-1.0.2o-gcc-7.2.0-cvldq3v
Use "module spider" to find all possible modules. Use "module spider" to find all possible modules.
@@ -433,7 +433,7 @@ Next you should regenerate all the module files:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh root@module-file-tutorial:/# spack module tcl refresh
==> You are about to regenerate tcl module files for: ==> You are about to regenerate tcl module files for:
-- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 -------------------------
@@ -444,11 +444,11 @@ Next you should regenerate all the module files:
mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0 mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0
-- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 -------------------------
mwamumj bzip2@1.0.6 bcikpen netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 mwamumj bzip2@1.0.6 7apabqu netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11
h24ofjs cmake@3.11.4 d3lertf netlib-scalapack@2.0.2 2h6xmxh openmpi@1.10.2 jqhycal py-setuptools@39.2.0 6ovorxs cmake@3.12.0 6i5qsqx netlib-scalapack@2.0.2 6oewzwj openmpi@1.10.2 jqhycal py-setuptools@39.2.0
zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15 zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15
vt5xcat mpich@3.2.1 lqfhvfh netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 vt5xcat mpich@3.2.1 3bz5rxx netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0
xcgzqdv ncurses@6.1 jae3ilo netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 xcgzqdv ncurses@6.1 z52ltyy netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1
==> Do you want to proceed? [y/n] y ==> Do you want to proceed? [y/n] y
==> Regenerating tcl module files ==> Regenerating tcl module files
@@ -503,7 +503,7 @@ and regenerate the module files:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh --delete-tree root@module-file-tutorial:/# spack module tcl refresh --delete-tree
==> You are about to regenerate tcl module files for: ==> You are about to regenerate tcl module files for:
-- linux-ubuntu16.04-x86_64 / gcc@5.4.0 ------------------------- -- linux-ubuntu16.04-x86_64 / gcc@5.4.0 -------------------------
@@ -514,11 +514,11 @@ and regenerate the module files:
mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0 mkaoyhz git@2.9.4 wl6mywv lmod@7.7.29 56lbd3h mpc@1.1.0 nxhwrg7 readline@7.0
-- linux-ubuntu16.04-x86_64 / gcc@7.2.0 ------------------------- -- linux-ubuntu16.04-x86_64 / gcc@7.2.0 -------------------------
mwamumj bzip2@1.0.6 bcikpen netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11 mwamumj bzip2@1.0.6 7apabqu netlib-lapack@3.8.0 pdatzbi openblas@0.3.0 fdiryak py-scipy@1.1.0 ezuwp4p zlib@1.2.11
h24ofjs cmake@3.11.4 d3lertf netlib-scalapack@2.0.2 2h6xmxh openmpi@1.10.2 jqhycal py-setuptools@39.2.0 6ovorxs cmake@3.12.0 6i5qsqx netlib-scalapack@2.0.2 6oewzwj openmpi@1.10.2 jqhycal py-setuptools@39.2.0
zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15 zk5lhob gdbm@1.14.1 uhzmwog netlib-scalapack@2.0.2 cvldq3v openssl@1.0.2o c7pnzul python@2.7.15
vt5xcat mpich@3.2.1 lqfhvfh netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0 vt5xcat mpich@3.2.1 3bz5rxx netlib-scalapack@2.0.2 yoxwmgb pkgconf@1.4.2 ccruj2i readline@7.0
xcgzqdv ncurses@6.1 jae3ilo netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1 xcgzqdv ncurses@6.1 z52ltyy netlib-scalapack@2.0.2 t3loxvu py-numpy@1.14.3 5ltus3a sqlite@3.23.1
==> Do you want to proceed? [y/n] y ==> Do you want to proceed? [y/n] y
==> Regenerating tcl module files ==> Regenerating tcl module files
@@ -526,13 +526,13 @@ and regenerate the module files:
root@module-file-tutorial:/# module avail root@module-file-tutorial:/# module avail
-------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 ---------------------------------------------
bzip2-1.0.6-gcc-7.2.0-mwamumj netlib-scalapack-2.0.2-gcc-7.2.0-jae3ilo py-numpy-1.14.3-gcc-7.2.0-t3loxvu bzip2-1.0.6-gcc-7.2.0-mwamumj netlib-scalapack-2.0.2-gcc-7.2.0-6i5qsqx py-numpy-1.14.3-gcc-7.2.0-t3loxvu
cmake-3.11.4-gcc-7.2.0-h24ofjs netlib-scalapack-2.0.2-gcc-7.2.0-lqfhvfh py-scipy-1.1.0-gcc-7.2.0-fdiryak cmake-3.12.0-gcc-7.2.0-6ovorxs netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog py-scipy-1.1.0-gcc-7.2.0-fdiryak
gdbm-1.14.1-gcc-7.2.0-zk5lhob netlib-scalapack-2.0.2-gcc-7.2.0-uhzmwog py-setuptools-39.2.0-gcc-7.2.0-jqhycal gdbm-1.14.1-gcc-7.2.0-zk5lhob netlib-scalapack-2.0.2-gcc-7.2.0-z52ltyy py-setuptools-39.2.0-gcc-7.2.0-jqhycal
mpich-3.2.1-gcc-7.2.0-vt5xcat openblas-0.3.0-gcc-7.2.0-pdatzbi python-2.7.15-gcc-7.2.0-c7pnzul mpich-3.2.1-gcc-7.2.0-vt5xcat openblas-0.3.0-gcc-7.2.0-pdatzbi python-2.7.15-gcc-7.2.0-c7pnzul
ncurses-6.1-gcc-7.2.0-xcgzqdv openmpi-1.10.2-gcc-7.2.0-2h6xmxh readline-7.0-gcc-7.2.0-ccruj2i ncurses-6.1-gcc-7.2.0-xcgzqdv openmpi-1.10.2-gcc-7.2.0-6oewzwj readline-7.0-gcc-7.2.0-ccruj2i
netlib-lapack-3.8.0-gcc-7.2.0-bcikpen openssl-1.0.2o-gcc-7.2.0-cvldq3v sqlite-3.23.1-gcc-7.2.0-5ltus3a netlib-lapack-3.8.0-gcc-7.2.0-7apabqu openssl-1.0.2o-gcc-7.2.0-cvldq3v sqlite-3.23.1-gcc-7.2.0-5ltus3a
netlib-scalapack-2.0.2-gcc-7.2.0-d3lertf pkgconf-1.4.2-gcc-7.2.0-yoxwmgb zlib-1.2.11-gcc-7.2.0-ezuwp4p netlib-scalapack-2.0.2-gcc-7.2.0-3bz5rxx pkgconf-1.4.2-gcc-7.2.0-yoxwmgb zlib-1.2.11-gcc-7.2.0-ezuwp4p
Use "module spider" to find all possible modules. Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
@@ -562,7 +562,7 @@ exceptions to the blacklist rules you can use ``whitelist``:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh -y root@module-file-tutorial:/# spack module tcl refresh -y
==> Regenerating tcl module files ==> Regenerating tcl module files
@@ -626,7 +626,7 @@ If you try to regenerate the module files now you will get an error:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh --delete-tree -y root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y
==> Error: Name clashes detected in module files: ==> Error: Name clashes detected in module files:
file: /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64/netlib-scalapack-2.0.2-gcc-7.2.0 file: /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64/netlib-scalapack-2.0.2-gcc-7.2.0
@@ -673,14 +673,14 @@ Regenerating module files now we obtain:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh --delete-tree -y root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files ==> Regenerating tcl module files
root@module-file-tutorial:/# module av root@module-file-tutorial:/# module av
-------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 ---------------------------------------------
bzip2-1.0.6-gcc-7.2.0 netlib-lapack-3.8.0-gcc-7.2.0 openmpi-1.10.2-gcc-7.2.0 python-2.7.15-gcc-7.2.0 bzip2-1.0.6-gcc-7.2.0 netlib-lapack-3.8.0-gcc-7.2.0 openmpi-1.10.2-gcc-7.2.0 python-2.7.15-gcc-7.2.0
cmake-3.11.4-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-mpich openssl-1.0.2o-gcc-7.2.0 readline-7.0-gcc-7.2.0 cmake-3.12.0-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-mpich openssl-1.0.2o-gcc-7.2.0 readline-7.0-gcc-7.2.0
gcc-7.2.0-gcc-5.4.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-openmpi pkgconf-1.4.2-gcc-7.2.0 sqlite-3.23.1-gcc-7.2.0 gcc-7.2.0-gcc-5.4.0 netlib-scalapack-2.0.2-gcc-7.2.0-netlib-openmpi pkgconf-1.4.2-gcc-7.2.0 sqlite-3.23.1-gcc-7.2.0
gdbm-1.14.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-mpich py-numpy-1.14.3-gcc-7.2.0-openblas zlib-1.2.11-gcc-7.2.0 gdbm-1.14.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-mpich py-numpy-1.14.3-gcc-7.2.0-openblas zlib-1.2.11-gcc-7.2.0
mpich-3.2.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-openmpi py-scipy-1.1.0-gcc-7.2.0-openblas mpich-3.2.1-gcc-7.2.0 netlib-scalapack-2.0.2-gcc-7.2.0-openblas-openmpi py-scipy-1.1.0-gcc-7.2.0-openblas
@@ -720,13 +720,13 @@ The final result should look like:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh --delete-tree -y root@module-file-tutorial:/# spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files ==> Regenerating tcl module files
root@module-file-tutorial:/# module av root@module-file-tutorial:/# module av
-------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 --------------------------------------------- -------------------------------------------- /usr/local/share/spack/modules/linux-ubuntu16.04-x86_64 ---------------------------------------------
bzip2/1.0.6-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-openmpi py-scipy/1.1.0-gcc-7.2.0-openblas bzip2/1.0.6-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-netlib-openmpi py-scipy/1.1.0-gcc-7.2.0-openblas
cmake/3.11.4-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich py-setuptools/39.2.0-gcc-7.2.0 cmake/3.12.0-gcc-7.2.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-mpich py-setuptools/39.2.0-gcc-7.2.0
gcc/7.2.0-gcc-5.4.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-openmpi (D) python/2.7.15-gcc-7.2.0 gcc/7.2.0-gcc-5.4.0 netlib-scalapack/2.0.2-gcc-7.2.0-openblas-openmpi (D) python/2.7.15-gcc-7.2.0
gdbm/1.14.1-gcc-7.2.0 openblas/0.3.0-gcc-7.2.0 readline/7.0-gcc-7.2.0 gdbm/1.14.1-gcc-7.2.0 openblas/0.3.0-gcc-7.2.0 readline/7.0-gcc-7.2.0
mpich/3.2.1-gcc-7.2.0 openmpi/1.10.2-gcc-7.2.0 sqlite/3.23.1-gcc-7.2.0 mpich/3.2.1-gcc-7.2.0 openmpi/1.10.2-gcc-7.2.0 sqlite/3.23.1-gcc-7.2.0
@@ -794,7 +794,7 @@ Regenerating the module files results in something like:
.. code-block:: console .. code-block:: console
:emphasize-lines: 15 :emphasize-lines: 15
root@module-file-tutorial:/# spack tcl refresh -y root@module-file-tutorial:/# spack module tcl refresh -y
==> Regenerating tcl module files ==> Regenerating tcl module files
root@module-file-tutorial:/# module show gcc root@module-file-tutorial:/# module show gcc
@@ -870,10 +870,10 @@ This time we will be more selective and regenerate only the ``gcc`` and
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh -y gcc root@module-file-tutorial:/# spack module tcl refresh -y gcc
==> Regenerating tcl module files ==> Regenerating tcl module files
root@module-file-tutorial:/# spack tcl refresh -y openmpi root@module-file-tutorial:/# spack module tcl refresh -y openmpi
==> Regenerating tcl module files ==> Regenerating tcl module files
root@module-file-tutorial:/# module show gcc root@module-file-tutorial:/# module show gcc
@@ -981,7 +981,7 @@ and regenerating the module files for every package that depends on ``python``:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack tcl refresh -y ^python root@module-file-tutorial:/# spack module tcl refresh -y ^python
==> Regenerating tcl module files ==> Regenerating tcl module files
Now the ``py-scipy`` module will be: Now the ``py-scipy`` module will be:
@@ -1165,7 +1165,7 @@ If we now regenerate the module files:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack lmod refresh --delete-tree -y root@module-file-tutorial:/# spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files ==> Regenerating lmod module files
and update ``MODULEPATH`` to point to the ``Core``: and update ``MODULEPATH`` to point to the ``Core``:
@@ -1198,7 +1198,7 @@ the ``Compiler`` part of the hierarchy:
----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -----------------------------------------
bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1 bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1
cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11
gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0
------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core --------------------------------------------
@@ -1223,7 +1223,7 @@ either ``mpich`` or ``openmpi``. Let's start by loading ``mpich``:
----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -----------------------------------------
bzip2/1.0.6 mpich/3.2.1 (L) openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1 bzip2/1.0.6 mpich/3.2.1 (L) openblas/0.3.0 pkgconf/1.4.2 py-setuptools/39.2.0 sqlite/3.23.1
cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11 cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-numpy/1.14.3-openblas python/2.7.15 zlib/1.2.11
gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o py-scipy/1.1.0-openblas readline/7.0
------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core --------------------------------------------
@@ -1357,7 +1357,7 @@ After module files have been regenerated as usual:
root@module-file-tutorial:/# module purge root@module-file-tutorial:/# module purge
root@module-file-tutorial:/# spack lmod refresh --delete-tree -y root@module-file-tutorial:/# spack module lmod refresh --delete-tree -y
==> Regenerating lmod module files ==> Regenerating lmod module files
we can see that now we have additional components in the hierarchy: we can see that now we have additional components in the hierarchy:
@@ -1369,11 +1369,11 @@ we can see that now we have additional components in the hierarchy:
root@module-file-tutorial:/# module avail root@module-file-tutorial:/# module avail
----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------ ----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------
py-numpy/1.14.3-openblas py-scipy/1.1.0-openblas py-numpy/1.14.3 py-scipy/1.1.0
----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -----------------------------------------
bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0 bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0
cmake/3.11.4 ncurses/6.1 openmpi/1.10.2 py-setuptools/39.2.0 sqlite/3.23.1 cmake/3.12.0 ncurses/6.1 openmpi/1.10.2 py-setuptools/39.2.0 sqlite/3.23.1
gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o python/2.7.15 zlib/1.2.11 gdbm/1.14.1 netlib-lapack/3.8.0 openssl/1.0.2o python/2.7.15 zlib/1.2.11
------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core -------------------------------------------- ------------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/Core --------------------------------------------
@@ -1389,11 +1389,11 @@ we can see that now we have additional components in the hierarchy:
root@module-file-tutorial:/# module load openmpi root@module-file-tutorial:/# module load openmpi
root@module-file-tutorial:/# module avail root@module-file-tutorial:/# module avail
------------------ /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openmpi/1.10.2-2h6xmxh/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------ ------------------ /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openmpi/1.10.2-6oewzwj/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------
netlib-scalapack/2.0.2-openblas netlib-scalapack/2.0.2
----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------ ----------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/openblas/0.3.0-pdatzbi/gcc/7.2.0 ------------------------------
py-numpy/1.14.3-openblas py-scipy/1.1.0-openblas py-numpy/1.14.3 py-scipy/1.1.0
----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 ----------------------------------------- ----------------------------------------- /usr/local/share/spack/lmod/linux-ubuntu16.04-x86_64/gcc/7.2.0 -----------------------------------------
bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0 bzip2/1.0.6 mpich/3.2.1 openblas/0.3.0 (L) pkgconf/1.4.2 readline/7.0
@@ -1420,18 +1420,15 @@ Both ``MPI`` and ``LAPACK`` providers will now benefit from the same safety feat
Due to MODULEPATH changes, the following have been reloaded: Due to MODULEPATH changes, the following have been reloaded:
1) netlib-scalapack/2.0.2-openblas 1) netlib-scalapack/2.0.2
root@module-file-tutorial:/# module load netlib-lapack root@module-file-tutorial:/# module load mpich
Lmod is automatically replacing "openblas/0.3.0" with "netlib-lapack/3.8.0". Lmod is automatically replacing "openmpi/1.10.2" with "mpich/3.2.1".
Inactive Modules: Due to MODULEPATH changes, the following have been reloaded:
1) py-numpy 1) netlib-scalapack/2.0.2
The following have been reloaded with a version change:
1) netlib-scalapack/2.0.2-openblas => netlib-scalapack/2.0.2-netlib
Because we only compiled ``py-numpy`` with ``openblas`` the module Because we only compiled ``py-numpy`` with ``openblas`` the module
is made inactive when we switch the ``LAPACK`` provider. The user is made inactive when we switch the ``LAPACK`` provider. The user
@@ -1589,7 +1586,7 @@ If we regenerate the module files one last time:
.. code-block:: console .. code-block:: console
root@module-file-tutorial:/# spack lmod refresh -y netlib-scalapack root@module-file-tutorial:/# spack module lmod refresh -y netlib-scalapack
==> Regenerating lmod module files ==> Regenerating lmod module files
we'll find the following at the end of each ``netlib-scalapack`` module file: we'll find the following at the end of each ``netlib-scalapack`` module file:

20
lib/spack/docs/workflows.rst
View File

@@ -276,11 +276,11 @@ have some drawbacks:
2. The ``spack spec`` and ``spack install`` commands use a 2. The ``spack spec`` and ``spack install`` commands use a
sophisticated concretization algorithm that chooses the "best" sophisticated concretization algorithm that chooses the "best"
among several options, taking into account ``packages.yaml`` file. among several options, taking into account ``packages.yaml`` file.
The ``spack load`` and ``spack tcl loads`` commands, on the The ``spack load`` and ``spack module tcl loads`` commands, on the
other hand, are not very smart: if the user-supplied spec matches other hand, are not very smart: if the user-supplied spec matches
more than one installed package, then ``spack tcl loads`` will more than one installed package, then ``spack module tcl loads`` will
fail. This may change in the future. For now, the workaround is to fail. This may change in the future. For now, the workaround is to
be more specific on any ``spack tcl loads`` lines that fail. be more specific on any ``spack module tcl loads`` lines that fail.
"""""""""""""""""""""" """"""""""""""""""""""
@@ -290,7 +290,7 @@ Generated Load Scripts
Another problem with using `spack load` is, it is slow; a typical user Another problem with using `spack load` is, it is slow; a typical user
environment could take several seconds to load, and would not be environment could take several seconds to load, and would not be
appropriate to put into ``.bashrc`` directly. It is preferable to use appropriate to put into ``.bashrc`` directly. It is preferable to use
a series of ``spack tcl loads`` commands to pre-compute which a series of ``spack module tcl loads`` commands to pre-compute which
modules to load. These can be put in a script that is run whenever modules to load. These can be put in a script that is run whenever
installed Spack packages change. For example: installed Spack packages change. For example:
@@ -301,7 +301,7 @@ installed Spack packages change. For example:
# Generate module load commands in ~/env/spackenv # Generate module load commands in ~/env/spackenv
cat <<EOF | /bin/sh >$HOME/env/spackenv cat <<EOF | /bin/sh >$HOME/env/spackenv
FIND='spack tcl loads --prefix linux-SuSE11-x86_64/' FIND='spack module tcl loads --prefix linux-SuSE11-x86_64/'
\$FIND modele-utils \$FIND modele-utils
\$FIND emacs \$FIND emacs
@@ -346,14 +346,14 @@ Users may now put ``source ~/env/spackenv`` into ``.bashrc``.
Some module systems put a prefix on the names of modules created Some module systems put a prefix on the names of modules created
by Spack. For example, that prefix is ``linux-SuSE11-x86_64/`` in by Spack. For example, that prefix is ``linux-SuSE11-x86_64/`` in
the above case. If a prefix is not needed, you may omit the the above case. If a prefix is not needed, you may omit the
``--prefix`` flag from ``spack tcl loads``. ``--prefix`` flag from ``spack module tcl loads``.
""""""""""""""""""""""" """""""""""""""""""""""
Transitive Dependencies Transitive Dependencies
""""""""""""""""""""""" """""""""""""""""""""""
In the script above, each ``spack tcl loads`` command generates a In the script above, each ``spack module tcl loads`` command generates a
*single* ``module load`` line. Transitive dependencies do not usually *single* ``module load`` line. Transitive dependencies do not usually
need to be loaded, only modules the user needs in ``$PATH``. This is need to be loaded, only modules the user needs in ``$PATH``. This is
because Spack builds binaries with RPATH. Spack's RPATH policy has because Spack builds binaries with RPATH. Spack's RPATH policy has
@@ -394,13 +394,13 @@ Unfortunately, Spack's RPATH support does not work in all case. For example:
In cases where RPATH support doesn't make things "just work," it can In cases where RPATH support doesn't make things "just work," it can
be necessary to load a module's dependencies as well as the module be necessary to load a module's dependencies as well as the module
itself. This is done by adding the ``--dependencies`` flag to the itself. This is done by adding the ``--dependencies`` flag to the
``spack tcl loads`` command. For example, the following line, ``spack module tcl loads`` command. For example, the following line,
added to the script above, would be used to load SciPy, along with added to the script above, would be used to load SciPy, along with
Numpy, core Python, BLAS/LAPACK and anything else needed: Numpy, core Python, BLAS/LAPACK and anything else needed:
.. code-block:: sh .. code-block:: sh
spack tcl loads --dependencies py-scipy spack module tcl loads --dependencies py-scipy
^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
Dummy Packages Dummy Packages
@@ -630,7 +630,7 @@ environments:
and extension packages. and extension packages.
* Views and activated extensions maintain state that is semantically * Views and activated extensions maintain state that is semantically
equivalent to the information in a ``spack tcl loads`` script. equivalent to the information in a ``spack module tcl loads`` script.
Administrators might find things easier to maintain without the Administrators might find things easier to maintain without the
added "heavyweight" state of a view. added "heavyweight" state of a view.

45
lib/spack/spack/cmd/module.py Normal file
View File

@@ -0,0 +1,45 @@
##############################################################################
# Copyright (c) 2013-2018, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/spack/spack
# Please also see the NOTICE and LICENSE files for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
import spack.cmd.modules.dotkit
import spack.cmd.modules.lmod
import spack.cmd.modules.tcl
description = "manipulate module files"
section = "environment"
level = "short"
_subcommands = {}
def setup_parser(subparser):
sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_type')
spack.cmd.modules.dotkit.add_command(sp, _subcommands)
spack.cmd.modules.lmod.add_command(sp, _subcommands)
spack.cmd.modules.tcl.add_command(sp, _subcommands)
def module(parser, args):
_subcommands[args.module_type](parser, args)

13
lib/spack/spack/cmd/common/modules.py → lib/spack/spack/cmd/modules/__init__.py
View File

@@ -22,9 +22,7 @@
# License along with this program; if not, write to the Free Software # License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
"""Contains all the functions that are common to the implementation of """Implementation details of the ``spack module`` command."""
each module file command.
"""
import collections import collections
import os.path import os.path
@@ -43,15 +41,6 @@
level = "short" level = "short"
#: Dictionary that will be populated with the list of sub-commands
#: Each sub-command must be callable and accept 3 arguments:
#:
#: - mtype : the type of the module file
#: - specs : the list of specs to be processed
#: - args : namespace containing the parsed command line arguments
callbacks = {}
def setup_parser(subparser): def setup_parser(subparser):
sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='subparser_name') sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='subparser_name')

24
lib/spack/spack/cmd/dotkit.py → lib/spack/spack/cmd/modules/dotkit.py
View File

@@ -22,21 +22,17 @@
# License along with this program; if not, write to the Free Software # License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
import spack.cmd.common.modules import functools
description = "manipulate dotkit module files" import spack.cmd.modules
section = "environment"
level = "short"
#: Type of the modules managed by this command
_module_type = 'dotkit'
def setup_parser(subparser): def add_command(parser, command_dict):
spack.cmd.common.modules.setup_parser(subparser) dotkit_parser = parser.add_parser(
'dotkit', help='manipulate dotkit module files'
)
def dotkit(parser, args): spack.cmd.modules.setup_parser(dotkit_parser)
spack.cmd.common.modules.modules_cmd(
parser, args, module_type=_module_type command_dict['dotkit'] = functools.partial(
spack.cmd.modules.modules_cmd, module_type='dotkit'
) )

42
lib/spack/spack/cmd/lmod.py → lib/spack/spack/cmd/modules/lmod.py
View File

@@ -22,29 +22,33 @@
# License along with this program; if not, write to the Free Software # License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
import functools
import os import os
import llnl.util.filesystem import llnl.util.filesystem
import spack.cmd.common.arguments import spack.cmd.common.arguments
import spack.cmd.common.modules import spack.cmd.modules
description = "manipulate hierarchical module files"
section = "environment"
level = "short"
#: Type of the modules managed by this command
_module_type = 'lmod'
def setup_parser(subparser): def add_command(parser, command_dict):
sp = spack.cmd.common.modules.setup_parser(subparser) lmod_parser = parser.add_parser(
'lmod', help='manipulate hierarchical module files'
)
sp = spack.cmd.modules.setup_parser(lmod_parser)
# Set default module file for a package # Set default module file for a package
setdefault = sp.add_parser( setdefault_parser = sp.add_parser(
'setdefault', help='set the default module file for a package' 'setdefault', help='set the default module file for a package'
) )
spack.cmd.common.arguments.add_common_arguments( spack.cmd.common.arguments.add_common_arguments(
setdefault, ['constraint'] setdefault_parser, ['constraint']
)
callbacks = dict(spack.cmd.modules.callbacks.items())
callbacks['setdefault'] = setdefault
command_dict['lmod'] = functools.partial(
spack.cmd.modules.modules_cmd, module_type='lmod', callbacks=callbacks
) )
@@ -54,8 +58,8 @@ def setdefault(module_type, specs, args):
# #
# https://lmod.readthedocs.io/en/latest/060_locating.html#marking-a-version-as-default # https://lmod.readthedocs.io/en/latest/060_locating.html#marking-a-version-as-default
# #
spack.cmd.common.modules.one_spec_or_raise(specs) spack.cmd.modules.one_spec_or_raise(specs)
writer = spack.modules.module_types[_module_type](specs[0]) writer = spack.modules.module_types['lmod'](specs[0])
module_folder = os.path.dirname(writer.layout.filename) module_folder = os.path.dirname(writer.layout.filename)
module_basename = os.path.basename(writer.layout.filename) module_basename = os.path.basename(writer.layout.filename)
@@ -63,13 +67,3 @@ def setdefault(module_type, specs, args):
if os.path.exists('default') and os.path.islink('default'): if os.path.exists('default') and os.path.islink('default'):
os.remove('default') os.remove('default')
os.symlink(module_basename, 'default') os.symlink(module_basename, 'default')
callbacks = dict(spack.cmd.common.modules.callbacks.items())
callbacks['setdefault'] = setdefault
def lmod(parser, args):
spack.cmd.common.modules.modules_cmd(
parser, args, module_type=_module_type, callbacks=callbacks
)

24
lib/spack/spack/cmd/tcl.py → lib/spack/spack/cmd/modules/tcl.py
View File

@@ -22,21 +22,17 @@
# License along with this program; if not, write to the Free Software # License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
############################################################################## ##############################################################################
import spack.cmd.common.modules import functools
description = "manipulate non-hierarchical module files" import spack.cmd.modules
section = "environment"
level = "short"
#: Type of the modules managed by this command
_module_type = 'tcl'
def setup_parser(subparser): def add_command(parser, command_dict):
spack.cmd.common.modules.setup_parser(subparser) tcl_parser = parser.add_parser(
'tcl', help='manipulate non-hierarchical module files'
)
def tcl(parser, args): spack.cmd.modules.setup_parser(tcl_parser)
spack.cmd.common.modules.modules_cmd(
parser, args, module_type=_module_type command_dict['tcl'] = functools.partial(
spack.cmd.modules.modules_cmd, module_type='tcl'
) )

101
lib/spack/spack/test/cmd/dotkit.py
View File

@@ -1,101 +0,0 @@
##############################################################################
# Copyright (c) 2013-2018, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/spack/spack
# Please also see the NOTICE and LICENSE files for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
import argparse
import os.path
import pytest
import spack.cmd.dotkit
import spack.main
import spack.modules
dotkit = spack.main.SpackCommand('dotkit')
def _get_module_files(args):
specs = args.specs()
writer_cls = spack.modules.module_types['dotkit']
return [writer_cls(spec).layout.filename for spec in specs]
@pytest.fixture(
params=[
['rm', 'doesnotexist'], # Try to remove a non existing module
['find', 'mpileaks'], # Try to find a module with multiple matches
['find', 'doesnotexist'], # Try to find a module with no matches
]
)
def failure_args(request):
"""A list of arguments that will cause a failure"""
return request.param
@pytest.fixture(scope='module')
def parser():
"""Returns the parser for the module command"""
parser = argparse.ArgumentParser()
spack.cmd.dotkit.setup_parser(parser)
return parser
# TODO : test the --delete-tree option
# TODO : this requires having a separate directory for test modules
# TODO : add tests for loads and find to check the prompt format
@pytest.mark.db
@pytest.mark.usefixtures('database')
def test_exit_with_failure(database, failure_args):
with pytest.raises(spack.main.SpackCommandError):
dotkit(*failure_args)
@pytest.mark.db
@pytest.mark.usefixtures('database')
@pytest.mark.parametrize('cli_args', [
['libelf'],
['--full-path', 'libelf']
])
def test_find(cli_args):
"""Tests 'spack dotkit find' under a few common scenarios."""
dotkit(*(['find'] + cli_args))
@pytest.mark.db
@pytest.mark.usefixtures('database')
def test_remove_and_add_dotkit(parser):
"""Tests adding and removing a dotkit module file."""
rm_cli_args = ['rm', '-y', 'mpileaks']
module_files = _get_module_files(parser.parse_args(rm_cli_args))
for item in module_files:
assert os.path.exists(item)
dotkit(*rm_cli_args)
for item in module_files:
assert not os.path.exists(item)
dotkit('refresh', '-y', 'mpileaks')
for item in module_files:
assert os.path.exists(item)

117
lib/spack/spack/test/cmd/lmod.py → lib/spack/spack/test/cmd/module.py
View File

@@ -28,14 +28,14 @@
import spack.main import spack.main
import spack.modules import spack.modules
import spack.spec
lmod = spack.main.SpackCommand('lmod') module = spack.main.SpackCommand('module')
# Needed to make the fixture work
writer_cls = spack.modules.lmod.LmodModulefileWriter
# TODO : add tests for loads and find to check the prompt format def _module_files(module_type, *specs):
specs = [spack.spec.Spec(x).concretized() for x in specs]
writer_cls = spack.modules.module_types[module_type]
return [writer_cls(spec).layout.filename for spec in specs]
@pytest.fixture( @pytest.fixture(
@@ -50,15 +50,110 @@ def failure_args(request):
return request.param return request.param
def test_exit_with_failure(database, failure_args): @pytest.fixture(
params=['dotkit', 'tcl', 'lmod']
)
def module_type(request):
return request.param
# TODO : test the --delete-tree option
# TODO : this requires having a separate directory for test modules
# TODO : add tests for loads and find to check the prompt format
@pytest.mark.db
def test_exit_with_failure(database, module_type, failure_args):
with pytest.raises(spack.main.SpackCommandError): with pytest.raises(spack.main.SpackCommandError):
lmod(*failure_args) module(module_type, *failure_args)
@pytest.mark.db
def test_remove_and_add(database, module_type):
"""Tests adding and removing a tcl module file."""
if module_type == 'lmod':
# TODO: Testing this with lmod requires mocking
# TODO: the core compilers
return
rm_cli_args = ['rm', '-y', 'mpileaks']
module_files = _module_files(module_type, 'mpileaks')
for item in module_files:
assert os.path.exists(item)
module(module_type, *rm_cli_args)
for item in module_files:
assert not os.path.exists(item)
module(module_type, 'refresh', '-y', 'mpileaks')
for item in module_files:
assert os.path.exists(item)
@pytest.mark.db
@pytest.mark.parametrize('cli_args', [
['libelf'],
['--full-path', 'libelf']
])
def test_find(database, cli_args, module_type):
if module_type == 'lmod':
# TODO: Testing this with lmod requires mocking
# TODO: the core compilers
return
module(module_type, *(['find'] + cli_args))
@pytest.mark.db
@pytest.mark.usefixtures('database')
@pytest.mark.regression('2215')
def test_find_fails_on_multiple_matches():
# As we installed multiple versions of mpileaks, the command will
# fail because of multiple matches
out = module('tcl', 'find', 'mpileaks', fail_on_error=False)
assert module.returncode == 1
assert 'matches multiple packages' in out
# Passing multiple packages from the command line also results in the
# same failure
out = module(
'tcl', 'find', 'mpileaks ^mpich', 'libelf', fail_on_error=False
)
assert module.returncode == 1
assert 'matches multiple packages' in out
@pytest.mark.db
@pytest.mark.usefixtures('database')
@pytest.mark.regression('2570')
def test_find_fails_on_non_existing_packages():
# Another way the command might fail is if the package does not exist
out = module('tcl', 'find', 'doesnotexist', fail_on_error=False)
assert module.returncode == 1
assert 'matches no package' in out
@pytest.mark.db
@pytest.mark.usefixtures('database')
def test_find_recursive():
# If we call find without options it should return only one module
out = module('tcl', 'find', 'mpileaks ^zmpi')
assert len(out.split()) == 1
# If instead we call it with the recursive option the length should
# be greater
out = module('tcl', 'find', '-r', 'mpileaks ^zmpi')
assert len(out.split()) > 1
# Needed to make the 'module_configuration' fixture below work
writer_cls = spack.modules.lmod.LmodModulefileWriter
@pytest.mark.db
def test_setdefault_command( def test_setdefault_command(
mutable_database, module_configuration mutable_database, module_configuration
): ):
module_configuration('autoload_direct') module_configuration('autoload_direct')
# Install two different versions of a package # Install two different versions of a package
@@ -73,7 +168,7 @@ def test_setdefault_command(
} }
# Create two module files for the same software # Create two module files for the same software
lmod('refresh', '-y', '--delete-tree', preferred, other_spec) module('lmod', 'refresh', '-y', '--delete-tree', preferred, other_spec)
# Assert initial directory state: no link and all module files present # Assert initial directory state: no link and all module files present
link_name = os.path.join( link_name = os.path.join(
@@ -85,7 +180,7 @@ def test_setdefault_command(
assert not os.path.exists(link_name) assert not os.path.exists(link_name)
# Set the default to be the other spec # Set the default to be the other spec
lmod('setdefault', other_spec) module('lmod', 'setdefault', other_spec)
# Check that a link named 'default' exists, and points to the right file # Check that a link named 'default' exists, and points to the right file
for k in preferred, other_spec: for k in preferred, other_spec:
@@ -94,7 +189,7 @@ def test_setdefault_command(
assert os.path.realpath(link_name) == writers[other_spec].layout.filename assert os.path.realpath(link_name) == writers[other_spec].layout.filename
# Reset the default to be the preferred spec # Reset the default to be the preferred spec
lmod('setdefault', preferred) module('lmod', 'setdefault', preferred)
# Check that a link named 'default' exists, and points to the right file # Check that a link named 'default' exists, and points to the right file
for k in preferred, other_spec: for k in preferred, other_spec:

135
lib/spack/spack/test/cmd/tcl.py
View File

@@ -1,135 +0,0 @@
##############################################################################
# Copyright (c) 2013-2017, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/spack/spack
# Please also see the NOTICE and LICENSE files for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
import argparse
import os.path
import pytest
import spack.cmd.tcl
import spack.main
import spack.modules
tcl = spack.main.SpackCommand('tcl')
def _get_module_files(args):
specs = args.specs()
writer_cls = spack.modules.module_types['tcl']
return [writer_cls(spec).layout.filename for spec in specs]
@pytest.fixture(
params=[
['rm', 'doesnotexist'], # Try to remove a non existing module
['find', 'mpileaks'], # Try to find a module with multiple matches
['find', 'doesnotexist'], # Try to find a module with no matches
]
)
def failure_args(request):
"""A list of arguments that will cause a failure"""
return request.param
@pytest.fixture(scope='module')
def parser():
"""Returns the parser for the module command"""
parser = argparse.ArgumentParser()
spack.cmd.tcl.setup_parser(parser)
return parser
# TODO : test the --delete-tree option
# TODO : this requires having a separate directory for test modules
# TODO : add tests for loads and find to check the prompt format
def test_exit_with_failure(database, failure_args):
with pytest.raises(spack.main.SpackCommandError):
tcl(*failure_args)
def test_remove_and_add_tcl(database, parser):
"""Tests adding and removing a tcl module file."""
rm_cli_args = ['rm', '-y', 'mpileaks']
module_files = _get_module_files(parser.parse_args(rm_cli_args))
for item in module_files:
assert os.path.exists(item)
tcl(*rm_cli_args)
for item in module_files:
assert not os.path.exists(item)
tcl('refresh', '-y', 'mpileaks')
for item in module_files:
assert os.path.exists(item)
@pytest.mark.parametrize('cli_args', [
['libelf'],
['--full-path', 'libelf']
])
def test_find(database, cli_args):
"""Tests 'spack tcl find' under a few common scenarios."""
tcl(*(['find'] + cli_args))
@pytest.mark.db
@pytest.mark.usefixtures('database')
@pytest.mark.regression('2215')
def test_find_fails_on_multiple_matches():
# As we installed multiple versions of mpileaks, the command will
# fail because of multiple matches
out = tcl('find', 'mpileaks', fail_on_error=False)
assert tcl.returncode == 1
assert 'matches multiple packages' in out
# Passing multiple packages from the command line also results in the
# same failure
out = tcl('find', 'mpileaks ^mpich', 'libelf', fail_on_error=False)
assert tcl.returncode == 1
assert 'matches multiple packages' in out
@pytest.mark.db
@pytest.mark.usefixtures('database')
@pytest.mark.regression('2570')
def test_find_fails_on_non_existing_packages():
# Another way the command might fail is if the package does not exist
out = tcl('find', 'doesnotexist', fail_on_error=False)
assert tcl.returncode == 1
assert 'matches no package' in out
@pytest.mark.db
@pytest.mark.usefixtures('database')
def test_find_recursive():
# If we call find without options it should return only one module
out = tcl('find', 'mpileaks ^zmpi')
assert len(out.split()) == 1
# If instead we call it with the recursive option the length should
# be greater
out = tcl('find', '-r', 'mpileaks ^zmpi')
assert len(out.split()) > 1

8
share/spack/setup-env.sh
View File

@@ -121,25 +121,25 @@ function spack {
# If spack module command comes back with an error, do nothing. # If spack module command comes back with an error, do nothing.
case $_sp_subcommand in case $_sp_subcommand in
"use") "use")
if _sp_full_spec=$(command spack $_sp_flags dotkit find $_sp_subcommand_args "${_sp_spec[@]}"); then if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "${_sp_spec[@]}"); then
use $_sp_module_args $_sp_full_spec use $_sp_module_args $_sp_full_spec
else else
$(exit 1) $(exit 1)
fi ;; fi ;;
"unuse") "unuse")
if _sp_full_spec=$(command spack $_sp_flags dotkit find $_sp_subcommand_args "${_sp_spec[@]}"); then if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "${_sp_spec[@]}"); then
unuse $_sp_module_args $_sp_full_spec unuse $_sp_module_args $_sp_full_spec
else else
$(exit 1) $(exit 1)
fi ;; fi ;;
"load") "load")
if _sp_full_spec=$(command spack $_sp_flags tcl find $_sp_subcommand_args "${_sp_spec[@]}"); then if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "${_sp_spec[@]}"); then
module load $_sp_module_args $_sp_full_spec module load $_sp_module_args $_sp_full_spec
else else
$(exit 1) $(exit 1)
fi ;; fi ;;
"unload") "unload")
if _sp_full_spec=$(command spack $_sp_flags tcl find $_sp_subcommand_args "${_sp_spec[@]}"); then if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "${_sp_spec[@]}"); then
module unload $_sp_module_args $_sp_full_spec module unload $_sp_module_args $_sp_full_spec
else else
$(exit 1) $(exit 1)