Remove support for generating dotkit files (#11986)
Dotkit is being used only at a few sites and has been deprecated on new machines. This commit removes all the code that provide support for the generation of dotkit module files. A new validator named "deprecatedProperties" has been added to the jsonschema validators. It permits to prompt a warning message or exit with an error if a property that has been marked as deprecated is encountered. * Removed references to dotkit in the docs * Removed references to dotkit in setup-env-test.sh * Added a unit test for the 'deprecatedProperties' schema validator
This commit is contained in:
parent
b11984767b
commit
76b9c56110
@ -32,7 +32,6 @@ config:
|
||||
module_roots:
|
||||
tcl: $spack/share/spack/modules
|
||||
lmod: $spack/share/spack/lmod
|
||||
dotkit: $spack/share/spack/dotkit
|
||||
|
||||
|
||||
# Temporary locations Spack can try to use for builds.
|
||||
|
@ -16,7 +16,6 @@
|
||||
modules:
|
||||
enable:
|
||||
- tcl
|
||||
- dotkit
|
||||
prefix_inspections:
|
||||
bin:
|
||||
- PATH
|
||||
|
@ -1190,24 +1190,15 @@ Using Extensions
|
||||
^^^^^^^^^^^^^^^^
|
||||
|
||||
There are three ways to get ``numpy`` working in Python. The first is
|
||||
to use :ref:`shell-support`. You can simply ``use`` or ``load`` the
|
||||
to use :ref:`shell-support`. You can simply ``load`` the
|
||||
module for the extension, and it will be added to the ``PYTHONPATH``
|
||||
in your current shell.
|
||||
|
||||
For tcl modules:
|
||||
in your current shell:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack load python
|
||||
$ spack load py-numpy
|
||||
|
||||
or, for dotkit:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack use python
|
||||
$ spack use py-numpy
|
||||
|
||||
Now ``import numpy`` will succeed for as long as you keep your current
|
||||
session open.
|
||||
|
||||
|
@ -74,7 +74,6 @@ the location for each type of module. e.g.:
|
||||
module_roots:
|
||||
tcl: $spack/share/spack/modules
|
||||
lmod: $spack/share/spack/lmod
|
||||
dotkit: $spack/share/spack/dotkit
|
||||
|
||||
See :ref:`modules` for details.
|
||||
|
||||
|
@ -463,7 +463,6 @@ account all scopes. For example, to see the fully merged
|
||||
module_roots:
|
||||
tcl: $spack/share/spack/modules
|
||||
lmod: $spack/share/spack/lmod
|
||||
dotkit: $spack/share/spack/dotkit
|
||||
build_stage:
|
||||
- $tempdir/$user/spack-stage
|
||||
- ~/.spack/stage
|
||||
@ -514,7 +513,6 @@ down the problem:
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:32 module_roots:
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:33 tcl: $spack/share/spack/modules
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:34 lmod: $spack/share/spack/lmod
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:35 dotkit: $spack/share/spack/dotkit
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:49 build_stage:
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:50 - $tempdir/$user/spack-stage
|
||||
/home/myuser/spack/etc/spack/defaults/config.yaml:51 - ~/.spack/stage
|
||||
|
@ -71,7 +71,7 @@ This automatically adds Spack to your ``PATH`` and allows the ``spack``
|
||||
command to be used to execute spack :ref:`commands <shell-support>` and
|
||||
:ref:`useful packaging commands <packaging-shell-support>`.
|
||||
|
||||
If :ref:`environment-modules or dotkit <InstallEnvironmentModules>` is
|
||||
If :ref:`environment-modules <InstallEnvironmentModules>` is
|
||||
installed and available, the ``spack`` command can also load and unload
|
||||
:ref:`modules <modules>`.
|
||||
|
||||
|
@ -13,8 +13,8 @@ The use of module systems to manage user environment in a controlled way
|
||||
is a common practice at HPC centers that is often embraced also by individual
|
||||
programmers on their development machines. To support this common practice
|
||||
Spack integrates with `Environment Modules
|
||||
<http://modules.sourceforge.net/>`_ , `LMod
|
||||
<http://lmod.readthedocs.io/en/latest/>`_ and `Dotkit <https://computing.llnl.gov/?set=jobs&page=dotkit>`_ by
|
||||
<http://modules.sourceforge.net/>`_ and `LMod
|
||||
<http://lmod.readthedocs.io/en/latest/>`_ by
|
||||
providing post-install hooks that generate module files and commands to manipulate them.
|
||||
|
||||
.. note::
|
||||
@ -67,7 +67,7 @@ to load the ``cmake`` module:
|
||||
$ module load cmake-3.7.2-gcc-6.3.0-fowuuby
|
||||
|
||||
Neither of these is particularly pretty, easy to remember, or
|
||||
easy to type. Luckily, Spack has its own interface for using modules and dotkits.
|
||||
easy to type. Luckily, Spack has its own interface for using modules.
|
||||
|
||||
^^^^^^^^^^^^^
|
||||
Shell support
|
||||
@ -108,20 +108,10 @@ that the startup time may be slightly increased because of that.
|
||||
^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Once you have shell support enabled you can use the same spec syntax
|
||||
you're used to:
|
||||
you're used to and you can use the same shortened names you use
|
||||
everywhere else in Spack.
|
||||
|
||||
========================= ==========================
|
||||
Modules Dotkit
|
||||
========================= ==========================
|
||||
``spack load <spec>`` ``spack use <spec>``
|
||||
``spack unload <spec>`` ``spack unuse <spec>``
|
||||
========================= ==========================
|
||||
|
||||
And you can use the same shortened names you use everywhere else in
|
||||
Spack.
|
||||
|
||||
For example, if you are using dotkit, this will add the ``mpich``
|
||||
package built with ``gcc`` to your path:
|
||||
For example this will add the ``mpich`` package built with ``gcc`` to your path:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@ -129,17 +119,10 @@ package built with ``gcc`` to your path:
|
||||
|
||||
# ... wait for install ...
|
||||
|
||||
$ spack use mpich %gcc@4.4.7 # dotkit
|
||||
Prepending: mpich@3.0.4%gcc@4.4.7 (ok)
|
||||
$ spack load mpich %gcc@4.4.7 # modules
|
||||
$ which mpicc
|
||||
~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc
|
||||
|
||||
Or, similarly if you are using modules, you could type:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ spack load mpich %gcc@4.4.7 # modules
|
||||
|
||||
These commands will add appropriate directories to your ``PATH``,
|
||||
``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer
|
||||
want to use a package, you can type unload or unuse similarly:
|
||||
@ -147,13 +130,12 @@ want to use a package, you can type unload or unuse similarly:
|
||||
.. code-block:: console
|
||||
|
||||
$ spack unload mpich %gcc@4.4.7 # modules
|
||||
$ spack unuse mpich %gcc@4.4.7 # dotkit
|
||||
|
||||
.. note::
|
||||
|
||||
These ``use``, ``unuse``, ``load``, and ``unload`` subcommands are
|
||||
The ``load`` and ``unload`` subcommands are
|
||||
only available if you have enabled Spack's shell support *and* you
|
||||
have dotkit or modules installed on your machine.
|
||||
have environment-modules installed on your machine.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
Ambiguous module names
|
||||
@ -292,8 +274,6 @@ that can be generated by Spack:
|
||||
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
|
||||
| | **Hook name** | **Default root directory** | **Default template file** | **Compatible tools** |
|
||||
+=============================+====================+===============================+==============================================+======================+
|
||||
| **Dotkit** | ``dotkit`` | share/spack/dotkit | share/spack/templates/modules/modulefile.dk | DotKit |
|
||||
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
|
||||
| **TCL - Non-Hierarchical** | ``tcl`` | share/spack/modules | share/spack/templates/modules/modulefile.tcl | Env. Modules/LMod |
|
||||
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
|
||||
| **Lua - Hierarchical** | ``lmod`` | share/spack/lmod | share/spack/templates/modules/modulefile.lua | LMod |
|
||||
@ -377,7 +357,7 @@ are named ``modules.yaml``. The default configuration:
|
||||
.. literalinclude:: _spack_root/etc/spack/defaults/modules.yaml
|
||||
:language: yaml
|
||||
|
||||
activates the hooks to generate ``tcl`` and ``dotkit`` module files and inspects
|
||||
activates the hooks to generate ``tcl`` module files and inspects
|
||||
the installation folder of each package for the presence of a set of subdirectories
|
||||
(``bin``, ``man``, ``share/man``, etc.). If any is found its full path is prepended
|
||||
to the environment variables listed below the folder name.
|
||||
@ -399,12 +379,9 @@ to the generator being customized:
|
||||
modules:
|
||||
enable:
|
||||
- tcl
|
||||
- dotkit
|
||||
- lmod
|
||||
tcl:
|
||||
# contains environment modules specific customizations
|
||||
dotkit:
|
||||
# contains dotkit specific customizations
|
||||
lmod:
|
||||
# contains lmod specific customizations
|
||||
|
||||
@ -590,15 +567,14 @@ do so by using the environment blacklist:
|
||||
.. code-block:: yaml
|
||||
|
||||
modules:
|
||||
dotkit:
|
||||
tcl:
|
||||
all:
|
||||
filter:
|
||||
# Exclude changes to any of these variables
|
||||
environment_blacklist: ['CPATH', 'LIBRARY_PATH']
|
||||
|
||||
The configuration above will generate dotkit module files that will not contain
|
||||
modifications to either ``CPATH`` or ``LIBRARY_PATH`` and environment module
|
||||
files that instead will contain these modifications.
|
||||
The configuration above will generate module files that will not contain
|
||||
modifications to either ``CPATH`` or ``LIBRARY_PATH``.
|
||||
|
||||
|
||||
.. _autoloading-dependencies:
|
||||
|
@ -7,7 +7,6 @@
|
||||
|
||||
import llnl.util.tty as tty
|
||||
|
||||
import spack.cmd.modules.dotkit
|
||||
import spack.cmd.modules.lmod
|
||||
import spack.cmd.modules.tcl
|
||||
|
||||
@ -23,7 +22,6 @@
|
||||
|
||||
def setup_parser(subparser):
|
||||
sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_command')
|
||||
spack.cmd.modules.dotkit.add_command(sp, _subcommands)
|
||||
spack.cmd.modules.lmod.add_command(sp, _subcommands)
|
||||
spack.cmd.modules.tcl.add_command(sp, _subcommands)
|
||||
|
||||
|
@ -137,7 +137,6 @@ def loads(module_type, specs, args, out=sys.stdout):
|
||||
module_commands = {
|
||||
'tcl': 'module load ',
|
||||
'lmod': 'module load ',
|
||||
'dotkit': 'use '
|
||||
}
|
||||
|
||||
d = {
|
||||
|
@ -1,19 +0,0 @@
|
||||
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import functools
|
||||
|
||||
import spack.cmd.modules
|
||||
|
||||
|
||||
def add_command(parser, command_dict):
|
||||
dotkit_parser = parser.add_parser(
|
||||
'dotkit', help='manipulate dotkit module files'
|
||||
)
|
||||
spack.cmd.modules.setup_parser(dotkit_parser)
|
||||
|
||||
command_dict['dotkit'] = functools.partial(
|
||||
spack.cmd.modules.modules_cmd, module_type='dotkit'
|
||||
)
|
@ -1,23 +0,0 @@
|
||||
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import argparse
|
||||
from spack.cmd.common import print_module_placeholder_help
|
||||
|
||||
description = "remove package from environment using dotkit"
|
||||
section = "modules"
|
||||
level = "long"
|
||||
|
||||
|
||||
def setup_parser(subparser):
|
||||
"""Parser is only constructed so that this prints a nice help
|
||||
message with -h. """
|
||||
subparser.add_argument(
|
||||
'spec', nargs=argparse.REMAINDER,
|
||||
help='spec of package to unuse with dotkit')
|
||||
|
||||
|
||||
def unuse(parser, args):
|
||||
print_module_placeholder_help()
|
@ -1,24 +0,0 @@
|
||||
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
import argparse
|
||||
from spack.cmd.common import print_module_placeholder_help, arguments
|
||||
|
||||
description = "add package to environment using dotkit"
|
||||
section = "modules"
|
||||
level = "long"
|
||||
|
||||
|
||||
def setup_parser(subparser):
|
||||
"""Parser is only constructed so that this prints a nice help
|
||||
message with -h. """
|
||||
subparser.add_argument(
|
||||
'spec', nargs=argparse.REMAINDER,
|
||||
help='spec of package to use with dotkit')
|
||||
arguments.add_common_arguments(subparser, ['recurse_dependencies'])
|
||||
|
||||
|
||||
def use(parser, args):
|
||||
print_module_placeholder_help()
|
@ -19,7 +19,7 @@
|
||||
* post_uninstall(spec)
|
||||
|
||||
This can be used to implement support for things like module
|
||||
systems (e.g. modules, dotkit, etc.) or to add other custom
|
||||
systems (e.g. modules, lmod, etc.) or to add other custom
|
||||
features.
|
||||
"""
|
||||
import os.path
|
||||
|
@ -591,7 +591,6 @@ def shell_set(var, value):
|
||||
module_roots = spack.config.get('config:module_roots')
|
||||
module_to_roots = {
|
||||
'tcl': list(),
|
||||
'dotkit': list(),
|
||||
'lmod': list()
|
||||
}
|
||||
for name, path in module_roots.items():
|
||||
|
@ -4,24 +4,20 @@
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
"""This package contains code for creating environment modules, which can
|
||||
include dotkits, TCL non-hierarchical modules, LUA hierarchical modules, and
|
||||
others.
|
||||
include TCL non-hierarchical modules, LUA hierarchical modules, and others.
|
||||
"""
|
||||
|
||||
from __future__ import absolute_import
|
||||
|
||||
from .dotkit import DotkitModulefileWriter
|
||||
from .tcl import TclModulefileWriter
|
||||
from .lmod import LmodModulefileWriter
|
||||
|
||||
__all__ = [
|
||||
'DotkitModulefileWriter',
|
||||
'TclModulefileWriter',
|
||||
'LmodModulefileWriter'
|
||||
]
|
||||
|
||||
module_types = {
|
||||
'dotkit': DotkitModulefileWriter,
|
||||
'tcl': TclModulefileWriter,
|
||||
'lmod': LmodModulefileWriter
|
||||
}
|
||||
|
@ -1,60 +0,0 @@
|
||||
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
"""This module implements the classes necessary to generate dotkit modules."""
|
||||
import os.path
|
||||
|
||||
import spack.config
|
||||
from .common import BaseConfiguration, BaseFileLayout
|
||||
from .common import BaseContext, BaseModuleFileWriter
|
||||
|
||||
#: Dotkit specific part of the configuration
|
||||
configuration = spack.config.get('modules:dotkit', {})
|
||||
|
||||
#: Caches the configuration {spec_hash: configuration}
|
||||
configuration_registry = {}
|
||||
|
||||
|
||||
def make_configuration(spec):
|
||||
"""Returns the dotkit configuration for spec"""
|
||||
key = spec.dag_hash()
|
||||
try:
|
||||
return configuration_registry[key]
|
||||
except KeyError:
|
||||
return configuration_registry.setdefault(
|
||||
key, DotkitConfiguration(spec)
|
||||
)
|
||||
|
||||
|
||||
def make_layout(spec):
|
||||
"""Returns the layout information for spec """
|
||||
conf = make_configuration(spec)
|
||||
return DotkitFileLayout(conf)
|
||||
|
||||
|
||||
def make_context(spec):
|
||||
"""Returns the context information for spec"""
|
||||
conf = make_configuration(spec)
|
||||
return DotkitContext(conf)
|
||||
|
||||
|
||||
class DotkitConfiguration(BaseConfiguration):
|
||||
"""Configuration class for dotkit module files."""
|
||||
|
||||
|
||||
class DotkitFileLayout(BaseFileLayout):
|
||||
"""File layout for dotkit module files."""
|
||||
|
||||
#: file extension of dotkit module files
|
||||
extension = 'dk'
|
||||
|
||||
|
||||
class DotkitContext(BaseContext):
|
||||
"""Context class for dotkit module files."""
|
||||
|
||||
|
||||
class DotkitModulefileWriter(BaseModuleFileWriter):
|
||||
"""Writer class for dotkit module files."""
|
||||
default_template = os.path.join('modules', 'modulefile.dk')
|
@ -11,6 +11,7 @@
|
||||
import six
|
||||
|
||||
import llnl.util.lang
|
||||
import llnl.util.tty
|
||||
import spack.spec
|
||||
|
||||
|
||||
@ -72,11 +73,35 @@ def _validate_spec(validator, is_spec, instance, schema):
|
||||
'"{0}" is an invalid spec [{1}]'.format(spec_str, str(e))
|
||||
)
|
||||
|
||||
def _deprecated_properties(validator, deprecated, instance, schema):
|
||||
if not (validator.is_type(instance, "object") or
|
||||
validator.is_type(instance, "array")):
|
||||
return
|
||||
|
||||
# Get a list of the deprecated properties, return if there is none
|
||||
deprecated_properties = [
|
||||
x for x in instance if x in deprecated['properties']
|
||||
]
|
||||
if not deprecated_properties:
|
||||
return
|
||||
|
||||
# Retrieve the template message
|
||||
msg = deprecated['message']
|
||||
is_error = deprecated['error']
|
||||
if not is_error:
|
||||
for entry in deprecated_properties:
|
||||
llnl.util.tty.warn(msg.format(property=entry))
|
||||
else:
|
||||
import jsonschema
|
||||
for entry in deprecated_properties:
|
||||
yield jsonschema.ValidationError(msg.format(property=entry))
|
||||
|
||||
return jsonschema.validators.extend(
|
||||
jsonschema.Draft4Validator, {
|
||||
"validate_spec": _validate_spec,
|
||||
"properties": _set_defaults,
|
||||
"patternProperties": _set_pp_defaults
|
||||
"patternProperties": _set_pp_defaults,
|
||||
"deprecatedProperties": _deprecated_properties
|
||||
}
|
||||
)
|
||||
|
||||
|
@ -41,6 +41,13 @@
|
||||
'lmod': {'type': 'string'},
|
||||
'dotkit': {'type': 'string'},
|
||||
},
|
||||
'deprecatedProperties': {
|
||||
'properties': ['dotkit'],
|
||||
'message': 'specifying a "{property}" module root has no '
|
||||
'effect [support for {property} module files'
|
||||
' has been dropped]',
|
||||
'error': False
|
||||
},
|
||||
},
|
||||
'source_cache': {'type': 'string'},
|
||||
'misc_cache': {'type': 'string'},
|
||||
|
@ -135,7 +135,16 @@
|
||||
'default': [],
|
||||
'items': {
|
||||
'type': 'string',
|
||||
'enum': ['tcl', 'dotkit', 'lmod']}},
|
||||
'enum': ['tcl', 'dotkit', 'lmod']
|
||||
},
|
||||
'deprecatedProperties': {
|
||||
'properties': ['dotkit'],
|
||||
'message': 'cannot enable "{property}" in modules.yaml '
|
||||
'[support for {property} module files has been'
|
||||
' dropped]',
|
||||
'error': False
|
||||
},
|
||||
},
|
||||
'lmod': {
|
||||
'allOf': [
|
||||
# Base configuration
|
||||
@ -164,6 +173,13 @@
|
||||
]
|
||||
},
|
||||
},
|
||||
'deprecatedProperties': {
|
||||
'properties': ['dotkit'],
|
||||
'message': 'the section "{property}" in modules.yaml has no effect'
|
||||
' [support for {property} module files has been '
|
||||
'dropped]',
|
||||
'error': False
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
@ -20,8 +20,7 @@
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'tcl': {'type': 'string'},
|
||||
'lmod': {'type': 'string'},
|
||||
'dotkit': {'type': 'string'}
|
||||
'lmod': {'type': 'string'}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
@ -21,7 +21,6 @@ def _module_files(module_type, *specs):
|
||||
|
||||
@pytest.fixture(scope='module', autouse=True)
|
||||
def ensure_module_files_are_there(database):
|
||||
module('dotkit', 'refresh', '-y')
|
||||
module('tcl', 'refresh', '-y')
|
||||
|
||||
|
||||
@ -39,7 +38,7 @@ def failure_args(request):
|
||||
|
||||
|
||||
@pytest.fixture(
|
||||
params=['dotkit', 'tcl', 'lmod']
|
||||
params=['tcl', 'lmod']
|
||||
)
|
||||
def module_type(request):
|
||||
return request.param
|
||||
|
@ -16,7 +16,6 @@
|
||||
modules:
|
||||
enable:
|
||||
- tcl
|
||||
- dotkit
|
||||
prefix_inspections:
|
||||
bin:
|
||||
- PATH
|
||||
|
@ -1,5 +0,0 @@
|
||||
enable:
|
||||
- dotkit
|
||||
dotkit:
|
||||
all:
|
||||
autoload: 'direct'
|
@ -1,5 +0,0 @@
|
||||
enable:
|
||||
- dotkit
|
||||
dotkit:
|
||||
all:
|
||||
template: 'override_from_modules.txt'
|
@ -1,51 +0,0 @@
|
||||
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
|
||||
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||
#
|
||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||
|
||||
|
||||
import pytest
|
||||
import spack.modules.dotkit
|
||||
|
||||
#: Class of the writer tested in this module
|
||||
writer_cls = spack.modules.dotkit.DotkitModulefileWriter
|
||||
|
||||
|
||||
@pytest.mark.usefixtures('config', 'mock_packages')
|
||||
class TestDotkit(object):
|
||||
|
||||
def test_dotkit(self, modulefile_content, module_configuration):
|
||||
"""Tests the generation of a dotkit file that loads dependencies
|
||||
automatically.
|
||||
"""
|
||||
|
||||
module_configuration('autoload_direct')
|
||||
content = modulefile_content('mpileaks arch=x86-linux')
|
||||
|
||||
assert '#c spack' in content
|
||||
assert '#d mpileaks @2.3' in content
|
||||
assert len([x for x in content if 'dk_op' in x]) == 2
|
||||
|
||||
def test_override_template_in_package(
|
||||
self, modulefile_content, module_configuration
|
||||
):
|
||||
"""Tests overriding a template from and attribute in the package."""
|
||||
|
||||
module_configuration('autoload_direct')
|
||||
content = modulefile_content('override-module-templates')
|
||||
|
||||
assert 'Override successful!' in content
|
||||
|
||||
def test_override_template_in_modules_yaml(
|
||||
self, modulefile_content, module_configuration
|
||||
):
|
||||
"""Tests overriding a template from `modules.yaml`"""
|
||||
|
||||
module_configuration('override_template')
|
||||
|
||||
# Check that this takes precedence over an attribute in the package
|
||||
content = modulefile_content('override-module-templates')
|
||||
assert 'Override even better!' in content
|
||||
|
||||
content = modulefile_content('mpileaks arch=x86-linux')
|
||||
assert 'Override even better!' in content
|
@ -114,3 +114,28 @@ def test_schema_validation(meta_schema, config_name):
|
||||
|
||||
# If this validation throws the test won't pass
|
||||
jsonschema.validate(schema, meta_schema)
|
||||
|
||||
|
||||
def test_deprecated_properties(module_suffixes_schema):
|
||||
# Test that an error is reported when 'error: True'
|
||||
module_suffixes_schema['deprecatedProperties'] = {
|
||||
'properties': ['tcl'],
|
||||
'message': '{property} not allowed',
|
||||
'error': True
|
||||
}
|
||||
v = spack.schema.Validator(module_suffixes_schema)
|
||||
data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}}
|
||||
|
||||
with pytest.raises(jsonschema.ValidationError, match='tcl not allowed'):
|
||||
v.validate(data)
|
||||
|
||||
# Test that just a warning is reported when 'error: False'
|
||||
module_suffixes_schema['deprecatedProperties'] = {
|
||||
'properties': ['tcl'],
|
||||
'message': '{property} not allowed',
|
||||
'error': False
|
||||
}
|
||||
v = spack.schema.Validator(module_suffixes_schema)
|
||||
data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}}
|
||||
# The next validation doesn't raise anymore
|
||||
v.validate(data)
|
||||
|
@ -5,27 +5,27 @@
|
||||
|
||||
########################################################################
|
||||
# This is a wrapper around the spack command that forwards calls to
|
||||
# 'spack use' and 'spack unuse' to shell functions. This in turn
|
||||
# allows them to be used to invoke dotkit functions.
|
||||
# 'spack load' and 'spack unload' to shell functions. This in turn
|
||||
# allows them to be used to invoke environment-modules functions.
|
||||
#
|
||||
# 'spack use' is smarter than just 'use' because it converts its
|
||||
# arguments into a unique spack spec that is then passed to dotkit
|
||||
# 'spack load' is smarter than just 'load' because it converts its
|
||||
# arguments into a unique Spack spec that is then passed to environment-modules
|
||||
# commands. This allows the user to use packages without knowing all
|
||||
# their installation details.
|
||||
#
|
||||
# e.g., rather than requring a full spec for libelf, the user can type:
|
||||
# e.g., rather than requiring a full spec for libelf, the user can type:
|
||||
#
|
||||
# spack use libelf
|
||||
# spack load libelf
|
||||
#
|
||||
# This will first find the available libelf dotkits and use a
|
||||
# This will first find the available libelf module file and use a
|
||||
# matching one. If there are two versions of libelf, the user would
|
||||
# need to be more specific, e.g.:
|
||||
#
|
||||
# spack use libelf@0.8.13
|
||||
# spack load libelf@0.8.13
|
||||
#
|
||||
# This is very similar to how regular spack commands work and it
|
||||
# avoids the need to come up with a user-friendly naming scheme for
|
||||
# spack dotfiles.
|
||||
# spack module files.
|
||||
########################################################################
|
||||
# accumulate initial flags for main spack command
|
||||
set _sp_flags = ""
|
||||
@ -104,8 +104,6 @@ case env:
|
||||
breaksw
|
||||
endsw
|
||||
endif
|
||||
case use:
|
||||
case unuse:
|
||||
case load:
|
||||
case unload:
|
||||
set _sp_module_args=""""
|
||||
@ -115,22 +113,10 @@ case unload:
|
||||
set _sp_spec = ($_sp_spec)
|
||||
endif
|
||||
|
||||
# Here the user has run use or unuse with a spec. Find a matching
|
||||
# Here the user has run load or unload with a spec. Find a matching
|
||||
# spec using 'spack module find', then use the appropriate module
|
||||
# tool's commands to add/remove the result from the environment.
|
||||
switch ($_sp_subcommand)
|
||||
case "use":
|
||||
set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" )
|
||||
if ( $? == 0 ) then
|
||||
use $_sp_module_args $_sp_full_spec
|
||||
endif
|
||||
breaksw
|
||||
case "unuse":
|
||||
set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" )
|
||||
if ( $? == 0 ) then
|
||||
unuse $_sp_module_args $_sp_full_spec
|
||||
endif
|
||||
breaksw
|
||||
case "load":
|
||||
# _sp_module_args may be "-r" for recursive spec retrieval
|
||||
set _sp_full_spec = ( "`\spack $_sp_flags module tcl find $_sp_module_args $_sp_spec`" )
|
||||
|
@ -167,7 +167,7 @@ is_not_set() {
|
||||
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Instead of invoking the module/use/dotkit commands, we print the
|
||||
# Instead of invoking the module commands, we print the
|
||||
# arguments that Spack invokes the command with, so we can check that
|
||||
# Spack passes the expected arguments in the tests below.
|
||||
#
|
||||
@ -177,14 +177,6 @@ module() {
|
||||
echo module "$@"
|
||||
}
|
||||
|
||||
use() {
|
||||
echo use "$@"
|
||||
}
|
||||
|
||||
unuse() {
|
||||
echo unuse "$@"
|
||||
}
|
||||
|
||||
# -----------------------------------------------------------------------
|
||||
# Setup test environment and do some preliminary checks
|
||||
# -----------------------------------------------------------------------
|
||||
@ -219,11 +211,9 @@ echo "Creating a mock package installation"
|
||||
spack -m install --fake a
|
||||
a_install=$(spack location -i a)
|
||||
a_module=$(spack -m module tcl find a)
|
||||
a_dotkit=$(spack -m module dotkit find a)
|
||||
|
||||
b_install=$(spack location -i b)
|
||||
b_module=$(spack -m module tcl find b)
|
||||
b_dotkit=$(spack -m module dotkit find b)
|
||||
|
||||
# create a test environment for tesitng environment commands
|
||||
echo "Creating a mock environment"
|
||||
@ -304,26 +294,6 @@ contains "usage: spack unload " spack -m unload -h
|
||||
contains "usage: spack unload " spack -m unload -h d
|
||||
contains "usage: spack unload " spack -m unload --help
|
||||
|
||||
title 'Testing `spack use`'
|
||||
contains "use $b_dotkit" spack -m use b
|
||||
fails spack -m use -l
|
||||
contains "use -l --arg $b_dotkit" spack -m use -l --arg b
|
||||
contains "use $b_dotkit $a_dotkit" spack -m use -r a
|
||||
contains "use $b_dotkit $a_dotkit" spack -m use --dependencies a
|
||||
fails spack -m use d
|
||||
contains "usage: spack use " spack -m use -h
|
||||
contains "usage: spack use " spack -m use -h d
|
||||
contains "usage: spack use " spack -m use --help
|
||||
|
||||
title 'Testing `spack unuse`'
|
||||
contains "unuse $b_dotkit" spack -m unuse b
|
||||
fails spack -m unuse -l
|
||||
contains "unuse -l --arg $b_dotkit" spack -m unuse -l --arg b
|
||||
fails spack -m unuse d
|
||||
contains "usage: spack unuse " spack -m unuse -h
|
||||
contains "usage: spack unuse " spack -m unuse -h d
|
||||
contains "usage: spack unuse " spack -m unuse --help
|
||||
|
||||
title 'Testing `spack env`'
|
||||
contains "usage: spack env " spack env -h
|
||||
contains "usage: spack env " spack env --help
|
||||
|
@ -6,7 +6,7 @@
|
||||
|
||||
#
|
||||
# This file is part of Spack and sets up the spack environment for
|
||||
# csh and tcsh. This includes dotkit support, module support, and
|
||||
# csh and tcsh. This includes environment modules and lmod support, and
|
||||
# it also puts spack in your path. Source it like this:
|
||||
#
|
||||
# setenv SPACK_ROOT /path/to/spack
|
||||
@ -24,7 +24,7 @@ if ($?SPACK_ROOT) then
|
||||
_spack_pathadd PATH "$SPACK_ROOT/bin"
|
||||
eval `spack --print-shell-vars csh`
|
||||
|
||||
# Set up modules and dotkit search paths in the user environment
|
||||
# Set up module search paths in the user environment
|
||||
set tcl_roots = `echo $_sp_tcl_roots:q | sed 's/:/ /g'`
|
||||
set compatible_sys_types = `echo $_sp_compatible_sys_types:q | sed 's/:/ /g'`
|
||||
foreach tcl_root ($tcl_roots:q)
|
||||
@ -33,10 +33,6 @@ if ($?SPACK_ROOT) then
|
||||
end
|
||||
end
|
||||
|
||||
set dotkit_roots = `echo $_sp_dotkit_roots:q | sed 's/:/ /g'`
|
||||
foreach dotkit_root ($dotkit_roots)
|
||||
_spack_pathadd DK_NODE "$dotkit_root/$_sp_sys_type"
|
||||
end
|
||||
else
|
||||
echo "ERROR: Sourcing spack setup-env.csh requires setting SPACK_ROOT to "
|
||||
echo " the root of your spack installation."
|
||||
|
@ -7,8 +7,8 @@
|
||||
########################################################################
|
||||
#
|
||||
# This file is part of Spack and sets up the spack environment for bash,
|
||||
# zsh, and dash (sh). This includes dotkit support, module support, and
|
||||
# it also puts spack in your path. The script also checks that at least
|
||||
# zsh, and dash (sh). This includes environment modules and lmod support,
|
||||
# and it also puts spack in your path. The script also checks that at least
|
||||
# module support exists, and provides suggestions if it doesn't. Source
|
||||
# it like this:
|
||||
#
|
||||
@ -16,27 +16,27 @@
|
||||
#
|
||||
########################################################################
|
||||
# This is a wrapper around the spack command that forwards calls to
|
||||
# 'spack use' and 'spack unuse' to shell functions. This in turn
|
||||
# allows them to be used to invoke dotkit functions.
|
||||
# 'spack load' and 'spack unload' to shell functions. This in turn
|
||||
# allows them to be used to invoke environment modules functions.
|
||||
#
|
||||
# 'spack use' is smarter than just 'use' because it converts its
|
||||
# arguments into a unique spack spec that is then passed to dotkit
|
||||
# 'spack load' is smarter than just 'load' because it converts its
|
||||
# arguments into a unique Spack spec that is then passed to module
|
||||
# commands. This allows the user to use packages without knowing all
|
||||
# their installation details.
|
||||
#
|
||||
# e.g., rather than requiring a full spec for libelf, the user can type:
|
||||
#
|
||||
# spack use libelf
|
||||
# spack load libelf
|
||||
#
|
||||
# This will first find the available libelf dotkits and use a
|
||||
# This will first find the available libelf module file and use a
|
||||
# matching one. If there are two versions of libelf, the user would
|
||||
# need to be more specific, e.g.:
|
||||
#
|
||||
# spack use libelf@0.8.13
|
||||
# spack load libelf@0.8.13
|
||||
#
|
||||
# This is very similar to how regular spack commands work and it
|
||||
# avoids the need to come up with a user-friendly naming scheme for
|
||||
# spack dotfiles.
|
||||
# spack module files.
|
||||
########################################################################
|
||||
|
||||
spack() {
|
||||
@ -140,7 +140,7 @@ spack() {
|
||||
fi
|
||||
return
|
||||
;;
|
||||
"use"|"unuse"|"load"|"unload")
|
||||
"load"|"unload")
|
||||
# Shift any other args for use off before parsing spec.
|
||||
_sp_subcommand_args=""
|
||||
_sp_module_args=""
|
||||
@ -161,20 +161,6 @@ spack() {
|
||||
# tool's commands to add/remove the result from the environment.
|
||||
# If spack module command comes back with an error, do nothing.
|
||||
case $_sp_subcommand in
|
||||
"use")
|
||||
if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then
|
||||
use $_sp_module_args $_sp_full_spec
|
||||
else
|
||||
$(exit 1)
|
||||
fi
|
||||
;;
|
||||
"unuse")
|
||||
if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then
|
||||
unuse $_sp_module_args $_sp_full_spec
|
||||
else
|
||||
$(exit 1)
|
||||
fi
|
||||
;;
|
||||
"load")
|
||||
if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "$@"); then
|
||||
module load $_sp_module_args $_sp_full_spec
|
||||
@ -356,7 +342,6 @@ _sp_multi_pathadd() {
|
||||
done
|
||||
}
|
||||
_sp_multi_pathadd MODULEPATH "$_sp_tcl_roots"
|
||||
_sp_multi_pathadd DK_NODE "$_sp_dotkit_roots"
|
||||
|
||||
# Add programmable tab completion for Bash
|
||||
#
|
||||
|
@ -760,7 +760,7 @@ function _spack_module {
|
||||
then
|
||||
compgen -W "-h --help" -- "$cur"
|
||||
else
|
||||
compgen -W "lmod tcl dotkit" -- "$cur"
|
||||
compgen -W "lmod tcl" -- "$cur"
|
||||
fi
|
||||
}
|
||||
|
||||
@ -812,53 +812,6 @@ function _spack_module_tcl_rm {
|
||||
fi
|
||||
}
|
||||
|
||||
function _spack_module_dotkit {
|
||||
if $list_options
|
||||
then
|
||||
compgen -W "-h --help" -- "$cur"
|
||||
else
|
||||
compgen -W "refresh find rm loads" -- "$cur"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
function _spack_module_dotkit_find {
|
||||
if $list_options
|
||||
then
|
||||
compgen -W "-h --help --full-path -r --dependencies" -- "$cur"
|
||||
else
|
||||
compgen -W "$(_installed_packages)" -- "$cur"
|
||||
fi
|
||||
}
|
||||
|
||||
function _spack_module_dotkit_loads {
|
||||
if $list_options
|
||||
then
|
||||
compgen -W "-h --help --input-only -p --prefix -x --exclude
|
||||
-r --dependencies" -- "$cur"
|
||||
else
|
||||
compgen -W "$(_installed_packages)" -- "$cur"
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
function _spack_module_dotkit_refresh {
|
||||
if $list_options
|
||||
then
|
||||
compgen -W "-h --help --delete-tree -y --yes-to-all" -- "$cur"
|
||||
else
|
||||
compgen -W "$(_installed_packages)" -- "$cur"
|
||||
fi
|
||||
}
|
||||
|
||||
function _spack_module_dotkit_rm {
|
||||
if $list_options
|
||||
then
|
||||
compgen -W "-h --help -y --yes-to-all" -- "$cur"
|
||||
else
|
||||
compgen -W "$(_installed_packages)" -- "$cur"
|
||||
fi
|
||||
}
|
||||
|
||||
function _spack_module_lmod {
|
||||
if $list_options
|
||||
|
@ -1,31 +0,0 @@
|
||||
{% block header %}
|
||||
{% if category %}
|
||||
#c {{ category }}
|
||||
{% endif %}
|
||||
{% if short_description %}
|
||||
#d {{ short_description }}
|
||||
{% endif %}
|
||||
{% if long_description %}
|
||||
{{ long_description| textwrap(72)| prepend_to_line('#h ')| join() }}
|
||||
{% endif %}
|
||||
{% endblock %}
|
||||
|
||||
{% block autoloads %}
|
||||
{% for module in autoload %}
|
||||
dk_op {{ module }}
|
||||
{% endfor %}
|
||||
{% endblock %}
|
||||
|
||||
{% block environment %}
|
||||
{% for command_name, cmd in environment_modifications %}
|
||||
{% if command_name == 'PrependPath' %}
|
||||
dk_alter {{ cmd.name }} {{ cmd.value }}
|
||||
{% endif %}
|
||||
{% if command_name == 'RemovePath' %}
|
||||
dk_unalter {{ cmd.name }} {{ cmd.value }}
|
||||
{% endif %}
|
||||
{% if command_name == 'SetEnv' %}
|
||||
dk_setenv {{ cmd.name }} {{ cmd.value }}
|
||||
{% endif %}
|
||||
{% endfor %}
|
||||
{% endblock %}
|
@ -12,7 +12,6 @@ class OverrideModuleTemplates(Package):
|
||||
|
||||
version('1.0', 'foobarbaz')
|
||||
|
||||
dotkit_template = 'override.txt'
|
||||
tcl_template = 'override.txt'
|
||||
lmod_template = 'override.txt'
|
||||
|
||||
|
@ -134,6 +134,6 @@ def setup_dependent_environment(self, spack_env, run_env, dependent_spec):
|
||||
# This *MUST* be first, this is where new code is installed
|
||||
spack_env.set('GOPATH', ':'.join(path_components))
|
||||
|
||||
# Allow packages to find this when using module or dotkit
|
||||
# Allow packages to find this when using module files
|
||||
run_env.prepend_path('GOPATH', ':'.join(
|
||||
[dependent_spec.prefix] + path_components))
|
||||
|
Loading…
Reference in New Issue
Block a user