Remove support for generating dotkit files (#11986)

Dotkit is being used only at a few sites and has been deprecated on new
machines. This commit removes all the code that provide support for the
generation of dotkit module files.

A new validator named "deprecatedProperties" has been added to the
jsonschema validators. It permits to prompt a warning message or exit
with an error if a property that has been marked as deprecated is
encountered.

* Removed references to dotkit in the docs
* Removed references to dotkit in setup-env-test.sh
* Added a unit test for the 'deprecatedProperties' schema validator
This commit is contained in:
Massimiliano Culpo 2019-10-03 07:15:01 +02:00 committed by Todd Gamblin
parent b11984767b
commit 76b9c56110
34 changed files with 121 additions and 426 deletions

View File

@ -32,7 +32,6 @@ config:
module_roots: module_roots:
tcl: $spack/share/spack/modules tcl: $spack/share/spack/modules
lmod: $spack/share/spack/lmod lmod: $spack/share/spack/lmod
dotkit: $spack/share/spack/dotkit
# Temporary locations Spack can try to use for builds. # Temporary locations Spack can try to use for builds.

View File

@ -16,7 +16,6 @@
modules: modules:
enable: enable:
- tcl - tcl
- dotkit
prefix_inspections: prefix_inspections:
bin: bin:
- PATH - PATH

View File

@ -1190,24 +1190,15 @@ Using Extensions
^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^
There are three ways to get ``numpy`` working in Python. The first is There are three ways to get ``numpy`` working in Python. The first is
to use :ref:`shell-support`. You can simply ``use`` or ``load`` the to use :ref:`shell-support`. You can simply ``load`` the
module for the extension, and it will be added to the ``PYTHONPATH`` module for the extension, and it will be added to the ``PYTHONPATH``
in your current shell. in your current shell:
For tcl modules:
.. code-block:: console .. code-block:: console
$ spack load python $ spack load python
$ spack load py-numpy $ spack load py-numpy
or, for dotkit:
.. code-block:: console
$ spack use python
$ spack use py-numpy
Now ``import numpy`` will succeed for as long as you keep your current Now ``import numpy`` will succeed for as long as you keep your current
session open. session open.

View File

@ -74,7 +74,6 @@ the location for each type of module. e.g.:
module_roots: module_roots:
tcl: $spack/share/spack/modules tcl: $spack/share/spack/modules
lmod: $spack/share/spack/lmod lmod: $spack/share/spack/lmod
dotkit: $spack/share/spack/dotkit
See :ref:`modules` for details. See :ref:`modules` for details.

View File

@ -463,7 +463,6 @@ account all scopes. For example, to see the fully merged
module_roots: module_roots:
tcl: $spack/share/spack/modules tcl: $spack/share/spack/modules
lmod: $spack/share/spack/lmod lmod: $spack/share/spack/lmod
dotkit: $spack/share/spack/dotkit
build_stage: build_stage:
- $tempdir/$user/spack-stage - $tempdir/$user/spack-stage
- ~/.spack/stage - ~/.spack/stage
@ -514,7 +513,6 @@ down the problem:
/home/myuser/spack/etc/spack/defaults/config.yaml:32 module_roots: /home/myuser/spack/etc/spack/defaults/config.yaml:32 module_roots:
/home/myuser/spack/etc/spack/defaults/config.yaml:33 tcl: $spack/share/spack/modules /home/myuser/spack/etc/spack/defaults/config.yaml:33 tcl: $spack/share/spack/modules
/home/myuser/spack/etc/spack/defaults/config.yaml:34 lmod: $spack/share/spack/lmod /home/myuser/spack/etc/spack/defaults/config.yaml:34 lmod: $spack/share/spack/lmod
/home/myuser/spack/etc/spack/defaults/config.yaml:35 dotkit: $spack/share/spack/dotkit
/home/myuser/spack/etc/spack/defaults/config.yaml:49 build_stage: /home/myuser/spack/etc/spack/defaults/config.yaml:49 build_stage:
/home/myuser/spack/etc/spack/defaults/config.yaml:50 - $tempdir/$user/spack-stage /home/myuser/spack/etc/spack/defaults/config.yaml:50 - $tempdir/$user/spack-stage
/home/myuser/spack/etc/spack/defaults/config.yaml:51 - ~/.spack/stage /home/myuser/spack/etc/spack/defaults/config.yaml:51 - ~/.spack/stage

View File

@ -71,7 +71,7 @@ This automatically adds Spack to your ``PATH`` and allows the ``spack``
command to be used to execute spack :ref:`commands <shell-support>` and command to be used to execute spack :ref:`commands <shell-support>` and
:ref:`useful packaging commands <packaging-shell-support>`. :ref:`useful packaging commands <packaging-shell-support>`.
If :ref:`environment-modules or dotkit <InstallEnvironmentModules>` is If :ref:`environment-modules <InstallEnvironmentModules>` is
installed and available, the ``spack`` command can also load and unload installed and available, the ``spack`` command can also load and unload
:ref:`modules <modules>`. :ref:`modules <modules>`.

View File

@ -13,8 +13,8 @@ The use of module systems to manage user environment in a controlled way
is a common practice at HPC centers that is often embraced also by individual is a common practice at HPC centers that is often embraced also by individual
programmers on their development machines. To support this common practice programmers on their development machines. To support this common practice
Spack integrates with `Environment Modules Spack integrates with `Environment Modules
<http://modules.sourceforge.net/>`_ , `LMod <http://modules.sourceforge.net/>`_ and `LMod
<http://lmod.readthedocs.io/en/latest/>`_ and `Dotkit <https://computing.llnl.gov/?set=jobs&page=dotkit>`_ by <http://lmod.readthedocs.io/en/latest/>`_ by
providing post-install hooks that generate module files and commands to manipulate them. providing post-install hooks that generate module files and commands to manipulate them.
.. note:: .. note::
@ -67,7 +67,7 @@ to load the ``cmake`` module:
$ module load cmake-3.7.2-gcc-6.3.0-fowuuby $ module load cmake-3.7.2-gcc-6.3.0-fowuuby
Neither of these is particularly pretty, easy to remember, or Neither of these is particularly pretty, easy to remember, or
easy to type. Luckily, Spack has its own interface for using modules and dotkits. easy to type. Luckily, Spack has its own interface for using modules.
^^^^^^^^^^^^^ ^^^^^^^^^^^^^
Shell support Shell support
@ -108,20 +108,10 @@ that the startup time may be slightly increased because of that.
^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^
Once you have shell support enabled you can use the same spec syntax Once you have shell support enabled you can use the same spec syntax
you're used to: you're used to and you can use the same shortened names you use
everywhere else in Spack.
========================= ========================== For example this will add the ``mpich`` package built with ``gcc`` to your path:
Modules Dotkit
========================= ==========================
``spack load <spec>`` ``spack use <spec>``
``spack unload <spec>`` ``spack unuse <spec>``
========================= ==========================
And you can use the same shortened names you use everywhere else in
Spack.
For example, if you are using dotkit, this will add the ``mpich``
package built with ``gcc`` to your path:
.. code-block:: console .. code-block:: console
@ -129,17 +119,10 @@ package built with ``gcc`` to your path:
# ... wait for install ... # ... wait for install ...
$ spack use mpich %gcc@4.4.7 # dotkit $ spack load mpich %gcc@4.4.7 # modules
Prepending: mpich@3.0.4%gcc@4.4.7 (ok)
$ which mpicc $ which mpicc
~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc ~/spack/opt/linux-debian7-x86_64/gcc@4.4.7/mpich@3.0.4/bin/mpicc
Or, similarly if you are using modules, you could type:
.. code-block:: console
$ spack load mpich %gcc@4.4.7 # modules
These commands will add appropriate directories to your ``PATH``, These commands will add appropriate directories to your ``PATH``,
``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer ``MANPATH``, ``CPATH``, and ``LD_LIBRARY_PATH``. When you no longer
want to use a package, you can type unload or unuse similarly: want to use a package, you can type unload or unuse similarly:
@ -147,13 +130,12 @@ want to use a package, you can type unload or unuse similarly:
.. code-block:: console .. code-block:: console
$ spack unload mpich %gcc@4.4.7 # modules $ spack unload mpich %gcc@4.4.7 # modules
$ spack unuse mpich %gcc@4.4.7 # dotkit
.. note:: .. note::
These ``use``, ``unuse``, ``load``, and ``unload`` subcommands are The ``load`` and ``unload`` subcommands are
only available if you have enabled Spack's shell support *and* you only available if you have enabled Spack's shell support *and* you
have dotkit or modules installed on your machine. have environment-modules installed on your machine.
^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^
Ambiguous module names Ambiguous module names
@ -292,8 +274,6 @@ that can be generated by Spack:
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+ +-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
| | **Hook name** | **Default root directory** | **Default template file** | **Compatible tools** | | | **Hook name** | **Default root directory** | **Default template file** | **Compatible tools** |
+=============================+====================+===============================+==============================================+======================+ +=============================+====================+===============================+==============================================+======================+
| **Dotkit** | ``dotkit`` | share/spack/dotkit | share/spack/templates/modules/modulefile.dk | DotKit |
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
| **TCL - Non-Hierarchical** | ``tcl`` | share/spack/modules | share/spack/templates/modules/modulefile.tcl | Env. Modules/LMod | | **TCL - Non-Hierarchical** | ``tcl`` | share/spack/modules | share/spack/templates/modules/modulefile.tcl | Env. Modules/LMod |
+-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+ +-----------------------------+--------------------+-------------------------------+----------------------------------------------+----------------------+
| **Lua - Hierarchical** | ``lmod`` | share/spack/lmod | share/spack/templates/modules/modulefile.lua | LMod | | **Lua - Hierarchical** | ``lmod`` | share/spack/lmod | share/spack/templates/modules/modulefile.lua | LMod |
@ -377,7 +357,7 @@ are named ``modules.yaml``. The default configuration:
.. literalinclude:: _spack_root/etc/spack/defaults/modules.yaml .. literalinclude:: _spack_root/etc/spack/defaults/modules.yaml
:language: yaml :language: yaml
activates the hooks to generate ``tcl`` and ``dotkit`` module files and inspects activates the hooks to generate ``tcl`` module files and inspects
the installation folder of each package for the presence of a set of subdirectories the installation folder of each package for the presence of a set of subdirectories
(``bin``, ``man``, ``share/man``, etc.). If any is found its full path is prepended (``bin``, ``man``, ``share/man``, etc.). If any is found its full path is prepended
to the environment variables listed below the folder name. to the environment variables listed below the folder name.
@ -399,12 +379,9 @@ to the generator being customized:
modules: modules:
enable: enable:
- tcl - tcl
- dotkit
- lmod - lmod
tcl: tcl:
# contains environment modules specific customizations # contains environment modules specific customizations
dotkit:
# contains dotkit specific customizations
lmod: lmod:
# contains lmod specific customizations # contains lmod specific customizations
@ -590,15 +567,14 @@ do so by using the environment blacklist:
.. code-block:: yaml .. code-block:: yaml
modules: modules:
dotkit: tcl:
all: all:
filter: filter:
# Exclude changes to any of these variables # Exclude changes to any of these variables
environment_blacklist: ['CPATH', 'LIBRARY_PATH'] environment_blacklist: ['CPATH', 'LIBRARY_PATH']
The configuration above will generate dotkit module files that will not contain The configuration above will generate module files that will not contain
modifications to either ``CPATH`` or ``LIBRARY_PATH`` and environment module modifications to either ``CPATH`` or ``LIBRARY_PATH``.
files that instead will contain these modifications.
.. _autoloading-dependencies: .. _autoloading-dependencies:

View File

@ -7,7 +7,6 @@
import llnl.util.tty as tty import llnl.util.tty as tty
import spack.cmd.modules.dotkit
import spack.cmd.modules.lmod import spack.cmd.modules.lmod
import spack.cmd.modules.tcl import spack.cmd.modules.tcl
@ -23,7 +22,6 @@
def setup_parser(subparser): def setup_parser(subparser):
sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_command') sp = subparser.add_subparsers(metavar='SUBCOMMAND', dest='module_command')
spack.cmd.modules.dotkit.add_command(sp, _subcommands)
spack.cmd.modules.lmod.add_command(sp, _subcommands) spack.cmd.modules.lmod.add_command(sp, _subcommands)
spack.cmd.modules.tcl.add_command(sp, _subcommands) spack.cmd.modules.tcl.add_command(sp, _subcommands)

View File

@ -137,7 +137,6 @@ def loads(module_type, specs, args, out=sys.stdout):
module_commands = { module_commands = {
'tcl': 'module load ', 'tcl': 'module load ',
'lmod': 'module load ', 'lmod': 'module load ',
'dotkit': 'use '
} }
d = { d = {

View File

@ -1,19 +0,0 @@
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import functools
import spack.cmd.modules
def add_command(parser, command_dict):
dotkit_parser = parser.add_parser(
'dotkit', help='manipulate dotkit module files'
)
spack.cmd.modules.setup_parser(dotkit_parser)
command_dict['dotkit'] = functools.partial(
spack.cmd.modules.modules_cmd, module_type='dotkit'
)

View File

@ -1,23 +0,0 @@
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import argparse
from spack.cmd.common import print_module_placeholder_help
description = "remove package from environment using dotkit"
section = "modules"
level = "long"
def setup_parser(subparser):
"""Parser is only constructed so that this prints a nice help
message with -h. """
subparser.add_argument(
'spec', nargs=argparse.REMAINDER,
help='spec of package to unuse with dotkit')
def unuse(parser, args):
print_module_placeholder_help()

View File

@ -1,24 +0,0 @@
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import argparse
from spack.cmd.common import print_module_placeholder_help, arguments
description = "add package to environment using dotkit"
section = "modules"
level = "long"
def setup_parser(subparser):
"""Parser is only constructed so that this prints a nice help
message with -h. """
subparser.add_argument(
'spec', nargs=argparse.REMAINDER,
help='spec of package to use with dotkit')
arguments.add_common_arguments(subparser, ['recurse_dependencies'])
def use(parser, args):
print_module_placeholder_help()

View File

@ -19,7 +19,7 @@
* post_uninstall(spec) * post_uninstall(spec)
This can be used to implement support for things like module This can be used to implement support for things like module
systems (e.g. modules, dotkit, etc.) or to add other custom systems (e.g. modules, lmod, etc.) or to add other custom
features. features.
""" """
import os.path import os.path

View File

@ -591,7 +591,6 @@ def shell_set(var, value):
module_roots = spack.config.get('config:module_roots') module_roots = spack.config.get('config:module_roots')
module_to_roots = { module_to_roots = {
'tcl': list(), 'tcl': list(),
'dotkit': list(),
'lmod': list() 'lmod': list()
} }
for name, path in module_roots.items(): for name, path in module_roots.items():

View File

@ -4,24 +4,20 @@
# SPDX-License-Identifier: (Apache-2.0 OR MIT) # SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""This package contains code for creating environment modules, which can """This package contains code for creating environment modules, which can
include dotkits, TCL non-hierarchical modules, LUA hierarchical modules, and include TCL non-hierarchical modules, LUA hierarchical modules, and others.
others.
""" """
from __future__ import absolute_import from __future__ import absolute_import
from .dotkit import DotkitModulefileWriter
from .tcl import TclModulefileWriter from .tcl import TclModulefileWriter
from .lmod import LmodModulefileWriter from .lmod import LmodModulefileWriter
__all__ = [ __all__ = [
'DotkitModulefileWriter',
'TclModulefileWriter', 'TclModulefileWriter',
'LmodModulefileWriter' 'LmodModulefileWriter'
] ]
module_types = { module_types = {
'dotkit': DotkitModulefileWriter,
'tcl': TclModulefileWriter, 'tcl': TclModulefileWriter,
'lmod': LmodModulefileWriter 'lmod': LmodModulefileWriter
} }

View File

@ -1,60 +0,0 @@
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""This module implements the classes necessary to generate dotkit modules."""
import os.path
import spack.config
from .common import BaseConfiguration, BaseFileLayout
from .common import BaseContext, BaseModuleFileWriter
#: Dotkit specific part of the configuration
configuration = spack.config.get('modules:dotkit', {})
#: Caches the configuration {spec_hash: configuration}
configuration_registry = {}
def make_configuration(spec):
"""Returns the dotkit configuration for spec"""
key = spec.dag_hash()
try:
return configuration_registry[key]
except KeyError:
return configuration_registry.setdefault(
key, DotkitConfiguration(spec)
)
def make_layout(spec):
"""Returns the layout information for spec """
conf = make_configuration(spec)
return DotkitFileLayout(conf)
def make_context(spec):
"""Returns the context information for spec"""
conf = make_configuration(spec)
return DotkitContext(conf)
class DotkitConfiguration(BaseConfiguration):
"""Configuration class for dotkit module files."""
class DotkitFileLayout(BaseFileLayout):
"""File layout for dotkit module files."""
#: file extension of dotkit module files
extension = 'dk'
class DotkitContext(BaseContext):
"""Context class for dotkit module files."""
class DotkitModulefileWriter(BaseModuleFileWriter):
"""Writer class for dotkit module files."""
default_template = os.path.join('modules', 'modulefile.dk')

View File

@ -11,6 +11,7 @@
import six import six
import llnl.util.lang import llnl.util.lang
import llnl.util.tty
import spack.spec import spack.spec
@ -72,11 +73,35 @@ def _validate_spec(validator, is_spec, instance, schema):
'"{0}" is an invalid spec [{1}]'.format(spec_str, str(e)) '"{0}" is an invalid spec [{1}]'.format(spec_str, str(e))
) )
def _deprecated_properties(validator, deprecated, instance, schema):
if not (validator.is_type(instance, "object") or
validator.is_type(instance, "array")):
return
# Get a list of the deprecated properties, return if there is none
deprecated_properties = [
x for x in instance if x in deprecated['properties']
]
if not deprecated_properties:
return
# Retrieve the template message
msg = deprecated['message']
is_error = deprecated['error']
if not is_error:
for entry in deprecated_properties:
llnl.util.tty.warn(msg.format(property=entry))
else:
import jsonschema
for entry in deprecated_properties:
yield jsonschema.ValidationError(msg.format(property=entry))
return jsonschema.validators.extend( return jsonschema.validators.extend(
jsonschema.Draft4Validator, { jsonschema.Draft4Validator, {
"validate_spec": _validate_spec, "validate_spec": _validate_spec,
"properties": _set_defaults, "properties": _set_defaults,
"patternProperties": _set_pp_defaults "patternProperties": _set_pp_defaults,
"deprecatedProperties": _deprecated_properties
} }
) )

View File

@ -41,6 +41,13 @@
'lmod': {'type': 'string'}, 'lmod': {'type': 'string'},
'dotkit': {'type': 'string'}, 'dotkit': {'type': 'string'},
}, },
'deprecatedProperties': {
'properties': ['dotkit'],
'message': 'specifying a "{property}" module root has no '
'effect [support for {property} module files'
' has been dropped]',
'error': False
},
}, },
'source_cache': {'type': 'string'}, 'source_cache': {'type': 'string'},
'misc_cache': {'type': 'string'}, 'misc_cache': {'type': 'string'},

View File

@ -135,7 +135,16 @@
'default': [], 'default': [],
'items': { 'items': {
'type': 'string', 'type': 'string',
'enum': ['tcl', 'dotkit', 'lmod']}}, 'enum': ['tcl', 'dotkit', 'lmod']
},
'deprecatedProperties': {
'properties': ['dotkit'],
'message': 'cannot enable "{property}" in modules.yaml '
'[support for {property} module files has been'
' dropped]',
'error': False
},
},
'lmod': { 'lmod': {
'allOf': [ 'allOf': [
# Base configuration # Base configuration
@ -164,6 +173,13 @@
] ]
}, },
}, },
'deprecatedProperties': {
'properties': ['dotkit'],
'message': 'the section "{property}" in modules.yaml has no effect'
' [support for {property} module files has been '
'dropped]',
'error': False
},
}, },
} }

View File

@ -20,8 +20,7 @@
'type': 'object', 'type': 'object',
'properties': { 'properties': {
'tcl': {'type': 'string'}, 'tcl': {'type': 'string'},
'lmod': {'type': 'string'}, 'lmod': {'type': 'string'}
'dotkit': {'type': 'string'}
} }
} }
} }

View File

@ -21,7 +21,6 @@ def _module_files(module_type, *specs):
@pytest.fixture(scope='module', autouse=True) @pytest.fixture(scope='module', autouse=True)
def ensure_module_files_are_there(database): def ensure_module_files_are_there(database):
module('dotkit', 'refresh', '-y')
module('tcl', 'refresh', '-y') module('tcl', 'refresh', '-y')
@ -39,7 +38,7 @@ def failure_args(request):
@pytest.fixture( @pytest.fixture(
params=['dotkit', 'tcl', 'lmod'] params=['tcl', 'lmod']
) )
def module_type(request): def module_type(request):
return request.param return request.param

View File

@ -16,7 +16,6 @@
modules: modules:
enable: enable:
- tcl - tcl
- dotkit
prefix_inspections: prefix_inspections:
bin: bin:
- PATH - PATH

View File

@ -1,5 +0,0 @@
enable:
- dotkit
dotkit:
all:
autoload: 'direct'

View File

@ -1,5 +0,0 @@
enable:
- dotkit
dotkit:
all:
template: 'override_from_modules.txt'

View File

@ -1,51 +0,0 @@
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import pytest
import spack.modules.dotkit
#: Class of the writer tested in this module
writer_cls = spack.modules.dotkit.DotkitModulefileWriter
@pytest.mark.usefixtures('config', 'mock_packages')
class TestDotkit(object):
def test_dotkit(self, modulefile_content, module_configuration):
"""Tests the generation of a dotkit file that loads dependencies
automatically.
"""
module_configuration('autoload_direct')
content = modulefile_content('mpileaks arch=x86-linux')
assert '#c spack' in content
assert '#d mpileaks @2.3' in content
assert len([x for x in content if 'dk_op' in x]) == 2
def test_override_template_in_package(
self, modulefile_content, module_configuration
):
"""Tests overriding a template from and attribute in the package."""
module_configuration('autoload_direct')
content = modulefile_content('override-module-templates')
assert 'Override successful!' in content
def test_override_template_in_modules_yaml(
self, modulefile_content, module_configuration
):
"""Tests overriding a template from `modules.yaml`"""
module_configuration('override_template')
# Check that this takes precedence over an attribute in the package
content = modulefile_content('override-module-templates')
assert 'Override even better!' in content
content = modulefile_content('mpileaks arch=x86-linux')
assert 'Override even better!' in content

View File

@ -114,3 +114,28 @@ def test_schema_validation(meta_schema, config_name):
# If this validation throws the test won't pass # If this validation throws the test won't pass
jsonschema.validate(schema, meta_schema) jsonschema.validate(schema, meta_schema)
def test_deprecated_properties(module_suffixes_schema):
# Test that an error is reported when 'error: True'
module_suffixes_schema['deprecatedProperties'] = {
'properties': ['tcl'],
'message': '{property} not allowed',
'error': True
}
v = spack.schema.Validator(module_suffixes_schema)
data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}}
with pytest.raises(jsonschema.ValidationError, match='tcl not allowed'):
v.validate(data)
# Test that just a warning is reported when 'error: False'
module_suffixes_schema['deprecatedProperties'] = {
'properties': ['tcl'],
'message': '{property} not allowed',
'error': False
}
v = spack.schema.Validator(module_suffixes_schema)
data = {'tcl': {'all': {'suffixes': {'^python': 'py'}}}}
# The next validation doesn't raise anymore
v.validate(data)

View File

@ -5,27 +5,27 @@
######################################################################## ########################################################################
# This is a wrapper around the spack command that forwards calls to # This is a wrapper around the spack command that forwards calls to
# 'spack use' and 'spack unuse' to shell functions. This in turn # 'spack load' and 'spack unload' to shell functions. This in turn
# allows them to be used to invoke dotkit functions. # allows them to be used to invoke environment-modules functions.
# #
# 'spack use' is smarter than just 'use' because it converts its # 'spack load' is smarter than just 'load' because it converts its
# arguments into a unique spack spec that is then passed to dotkit # arguments into a unique Spack spec that is then passed to environment-modules
# commands. This allows the user to use packages without knowing all # commands. This allows the user to use packages without knowing all
# their installation details. # their installation details.
# #
# e.g., rather than requring a full spec for libelf, the user can type: # e.g., rather than requiring a full spec for libelf, the user can type:
# #
# spack use libelf # spack load libelf
# #
# This will first find the available libelf dotkits and use a # This will first find the available libelf module file and use a
# matching one. If there are two versions of libelf, the user would # matching one. If there are two versions of libelf, the user would
# need to be more specific, e.g.: # need to be more specific, e.g.:
# #
# spack use libelf@0.8.13 # spack load libelf@0.8.13
# #
# This is very similar to how regular spack commands work and it # This is very similar to how regular spack commands work and it
# avoids the need to come up with a user-friendly naming scheme for # avoids the need to come up with a user-friendly naming scheme for
# spack dotfiles. # spack module files.
######################################################################## ########################################################################
# accumulate initial flags for main spack command # accumulate initial flags for main spack command
set _sp_flags = "" set _sp_flags = ""
@ -104,8 +104,6 @@ case env:
breaksw breaksw
endsw endsw
endif endif
case use:
case unuse:
case load: case load:
case unload: case unload:
set _sp_module_args="""" set _sp_module_args=""""
@ -115,22 +113,10 @@ case unload:
set _sp_spec = ($_sp_spec) set _sp_spec = ($_sp_spec)
endif endif
# Here the user has run use or unuse with a spec. Find a matching # Here the user has run load or unload with a spec. Find a matching
# spec using 'spack module find', then use the appropriate module # spec using 'spack module find', then use the appropriate module
# tool's commands to add/remove the result from the environment. # tool's commands to add/remove the result from the environment.
switch ($_sp_subcommand) switch ($_sp_subcommand)
case "use":
set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" )
if ( $? == 0 ) then
use $_sp_module_args $_sp_full_spec
endif
breaksw
case "unuse":
set _sp_full_spec = ( "`\spack $_sp_flags module dotkit find $_sp_spec`" )
if ( $? == 0 ) then
unuse $_sp_module_args $_sp_full_spec
endif
breaksw
case "load": case "load":
# _sp_module_args may be "-r" for recursive spec retrieval # _sp_module_args may be "-r" for recursive spec retrieval
set _sp_full_spec = ( "`\spack $_sp_flags module tcl find $_sp_module_args $_sp_spec`" ) set _sp_full_spec = ( "`\spack $_sp_flags module tcl find $_sp_module_args $_sp_spec`" )

View File

@ -167,7 +167,7 @@ is_not_set() {
# ----------------------------------------------------------------------- # -----------------------------------------------------------------------
# Instead of invoking the module/use/dotkit commands, we print the # Instead of invoking the module commands, we print the
# arguments that Spack invokes the command with, so we can check that # arguments that Spack invokes the command with, so we can check that
# Spack passes the expected arguments in the tests below. # Spack passes the expected arguments in the tests below.
# #
@ -177,14 +177,6 @@ module() {
echo module "$@" echo module "$@"
} }
use() {
echo use "$@"
}
unuse() {
echo unuse "$@"
}
# ----------------------------------------------------------------------- # -----------------------------------------------------------------------
# Setup test environment and do some preliminary checks # Setup test environment and do some preliminary checks
# ----------------------------------------------------------------------- # -----------------------------------------------------------------------
@ -219,11 +211,9 @@ echo "Creating a mock package installation"
spack -m install --fake a spack -m install --fake a
a_install=$(spack location -i a) a_install=$(spack location -i a)
a_module=$(spack -m module tcl find a) a_module=$(spack -m module tcl find a)
a_dotkit=$(spack -m module dotkit find a)
b_install=$(spack location -i b) b_install=$(spack location -i b)
b_module=$(spack -m module tcl find b) b_module=$(spack -m module tcl find b)
b_dotkit=$(spack -m module dotkit find b)
# create a test environment for tesitng environment commands # create a test environment for tesitng environment commands
echo "Creating a mock environment" echo "Creating a mock environment"
@ -304,26 +294,6 @@ contains "usage: spack unload " spack -m unload -h
contains "usage: spack unload " spack -m unload -h d contains "usage: spack unload " spack -m unload -h d
contains "usage: spack unload " spack -m unload --help contains "usage: spack unload " spack -m unload --help
title 'Testing `spack use`'
contains "use $b_dotkit" spack -m use b
fails spack -m use -l
contains "use -l --arg $b_dotkit" spack -m use -l --arg b
contains "use $b_dotkit $a_dotkit" spack -m use -r a
contains "use $b_dotkit $a_dotkit" spack -m use --dependencies a
fails spack -m use d
contains "usage: spack use " spack -m use -h
contains "usage: spack use " spack -m use -h d
contains "usage: spack use " spack -m use --help
title 'Testing `spack unuse`'
contains "unuse $b_dotkit" spack -m unuse b
fails spack -m unuse -l
contains "unuse -l --arg $b_dotkit" spack -m unuse -l --arg b
fails spack -m unuse d
contains "usage: spack unuse " spack -m unuse -h
contains "usage: spack unuse " spack -m unuse -h d
contains "usage: spack unuse " spack -m unuse --help
title 'Testing `spack env`' title 'Testing `spack env`'
contains "usage: spack env " spack env -h contains "usage: spack env " spack env -h
contains "usage: spack env " spack env --help contains "usage: spack env " spack env --help

View File

@ -6,7 +6,7 @@
# #
# This file is part of Spack and sets up the spack environment for # This file is part of Spack and sets up the spack environment for
# csh and tcsh. This includes dotkit support, module support, and # csh and tcsh. This includes environment modules and lmod support, and
# it also puts spack in your path. Source it like this: # it also puts spack in your path. Source it like this:
# #
# setenv SPACK_ROOT /path/to/spack # setenv SPACK_ROOT /path/to/spack
@ -24,7 +24,7 @@ if ($?SPACK_ROOT) then
_spack_pathadd PATH "$SPACK_ROOT/bin" _spack_pathadd PATH "$SPACK_ROOT/bin"
eval `spack --print-shell-vars csh` eval `spack --print-shell-vars csh`
# Set up modules and dotkit search paths in the user environment # Set up module search paths in the user environment
set tcl_roots = `echo $_sp_tcl_roots:q | sed 's/:/ /g'` set tcl_roots = `echo $_sp_tcl_roots:q | sed 's/:/ /g'`
set compatible_sys_types = `echo $_sp_compatible_sys_types:q | sed 's/:/ /g'` set compatible_sys_types = `echo $_sp_compatible_sys_types:q | sed 's/:/ /g'`
foreach tcl_root ($tcl_roots:q) foreach tcl_root ($tcl_roots:q)
@ -33,10 +33,6 @@ if ($?SPACK_ROOT) then
end end
end end
set dotkit_roots = `echo $_sp_dotkit_roots:q | sed 's/:/ /g'`
foreach dotkit_root ($dotkit_roots)
_spack_pathadd DK_NODE "$dotkit_root/$_sp_sys_type"
end
else else
echo "ERROR: Sourcing spack setup-env.csh requires setting SPACK_ROOT to " echo "ERROR: Sourcing spack setup-env.csh requires setting SPACK_ROOT to "
echo " the root of your spack installation." echo " the root of your spack installation."

View File

@ -7,8 +7,8 @@
######################################################################## ########################################################################
# #
# This file is part of Spack and sets up the spack environment for bash, # This file is part of Spack and sets up the spack environment for bash,
# zsh, and dash (sh). This includes dotkit support, module support, and # zsh, and dash (sh). This includes environment modules and lmod support,
# it also puts spack in your path. The script also checks that at least # and it also puts spack in your path. The script also checks that at least
# module support exists, and provides suggestions if it doesn't. Source # module support exists, and provides suggestions if it doesn't. Source
# it like this: # it like this:
# #
@ -16,27 +16,27 @@
# #
######################################################################## ########################################################################
# This is a wrapper around the spack command that forwards calls to # This is a wrapper around the spack command that forwards calls to
# 'spack use' and 'spack unuse' to shell functions. This in turn # 'spack load' and 'spack unload' to shell functions. This in turn
# allows them to be used to invoke dotkit functions. # allows them to be used to invoke environment modules functions.
# #
# 'spack use' is smarter than just 'use' because it converts its # 'spack load' is smarter than just 'load' because it converts its
# arguments into a unique spack spec that is then passed to dotkit # arguments into a unique Spack spec that is then passed to module
# commands. This allows the user to use packages without knowing all # commands. This allows the user to use packages without knowing all
# their installation details. # their installation details.
# #
# e.g., rather than requiring a full spec for libelf, the user can type: # e.g., rather than requiring a full spec for libelf, the user can type:
# #
# spack use libelf # spack load libelf
# #
# This will first find the available libelf dotkits and use a # This will first find the available libelf module file and use a
# matching one. If there are two versions of libelf, the user would # matching one. If there are two versions of libelf, the user would
# need to be more specific, e.g.: # need to be more specific, e.g.:
# #
# spack use libelf@0.8.13 # spack load libelf@0.8.13
# #
# This is very similar to how regular spack commands work and it # This is very similar to how regular spack commands work and it
# avoids the need to come up with a user-friendly naming scheme for # avoids the need to come up with a user-friendly naming scheme for
# spack dotfiles. # spack module files.
######################################################################## ########################################################################
spack() { spack() {
@ -140,7 +140,7 @@ spack() {
fi fi
return return
;; ;;
"use"|"unuse"|"load"|"unload") "load"|"unload")
# Shift any other args for use off before parsing spec. # Shift any other args for use off before parsing spec.
_sp_subcommand_args="" _sp_subcommand_args=""
_sp_module_args="" _sp_module_args=""
@ -161,20 +161,6 @@ spack() {
# tool's commands to add/remove the result from the environment. # tool's commands to add/remove the result from the environment.
# If spack module command comes back with an error, do nothing. # If spack module command comes back with an error, do nothing.
case $_sp_subcommand in case $_sp_subcommand in
"use")
if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then
use $_sp_module_args $_sp_full_spec
else
$(exit 1)
fi
;;
"unuse")
if _sp_full_spec=$(command spack $_sp_flags module dotkit find $_sp_subcommand_args "$@"); then
unuse $_sp_module_args $_sp_full_spec
else
$(exit 1)
fi
;;
"load") "load")
if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "$@"); then if _sp_full_spec=$(command spack $_sp_flags module tcl find $_sp_subcommand_args "$@"); then
module load $_sp_module_args $_sp_full_spec module load $_sp_module_args $_sp_full_spec
@ -356,7 +342,6 @@ _sp_multi_pathadd() {
done done
} }
_sp_multi_pathadd MODULEPATH "$_sp_tcl_roots" _sp_multi_pathadd MODULEPATH "$_sp_tcl_roots"
_sp_multi_pathadd DK_NODE "$_sp_dotkit_roots"
# Add programmable tab completion for Bash # Add programmable tab completion for Bash
# #

View File

@ -760,7 +760,7 @@ function _spack_module {
then then
compgen -W "-h --help" -- "$cur" compgen -W "-h --help" -- "$cur"
else else
compgen -W "lmod tcl dotkit" -- "$cur" compgen -W "lmod tcl" -- "$cur"
fi fi
} }
@ -812,53 +812,6 @@ function _spack_module_tcl_rm {
fi fi
} }
function _spack_module_dotkit {
if $list_options
then
compgen -W "-h --help" -- "$cur"
else
compgen -W "refresh find rm loads" -- "$cur"
fi
}
function _spack_module_dotkit_find {
if $list_options
then
compgen -W "-h --help --full-path -r --dependencies" -- "$cur"
else
compgen -W "$(_installed_packages)" -- "$cur"
fi
}
function _spack_module_dotkit_loads {
if $list_options
then
compgen -W "-h --help --input-only -p --prefix -x --exclude
-r --dependencies" -- "$cur"
else
compgen -W "$(_installed_packages)" -- "$cur"
fi
}
function _spack_module_dotkit_refresh {
if $list_options
then
compgen -W "-h --help --delete-tree -y --yes-to-all" -- "$cur"
else
compgen -W "$(_installed_packages)" -- "$cur"
fi
}
function _spack_module_dotkit_rm {
if $list_options
then
compgen -W "-h --help -y --yes-to-all" -- "$cur"
else
compgen -W "$(_installed_packages)" -- "$cur"
fi
}
function _spack_module_lmod { function _spack_module_lmod {
if $list_options if $list_options

View File

@ -1,31 +0,0 @@
{% block header %}
{% if category %}
#c {{ category }}
{% endif %}
{% if short_description %}
#d {{ short_description }}
{% endif %}
{% if long_description %}
{{ long_description| textwrap(72)| prepend_to_line('#h ')| join() }}
{% endif %}
{% endblock %}
{% block autoloads %}
{% for module in autoload %}
dk_op {{ module }}
{% endfor %}
{% endblock %}
{% block environment %}
{% for command_name, cmd in environment_modifications %}
{% if command_name == 'PrependPath' %}
dk_alter {{ cmd.name }} {{ cmd.value }}
{% endif %}
{% if command_name == 'RemovePath' %}
dk_unalter {{ cmd.name }} {{ cmd.value }}
{% endif %}
{% if command_name == 'SetEnv' %}
dk_setenv {{ cmd.name }} {{ cmd.value }}
{% endif %}
{% endfor %}
{% endblock %}

View File

@ -12,7 +12,6 @@ class OverrideModuleTemplates(Package):
version('1.0', 'foobarbaz') version('1.0', 'foobarbaz')
dotkit_template = 'override.txt'
tcl_template = 'override.txt' tcl_template = 'override.txt'
lmod_template = 'override.txt' lmod_template = 'override.txt'

View File

@ -134,6 +134,6 @@ def setup_dependent_environment(self, spack_env, run_env, dependent_spec):
# This *MUST* be first, this is where new code is installed # This *MUST* be first, this is where new code is installed
spack_env.set('GOPATH', ':'.join(path_components)) spack_env.set('GOPATH', ':'.join(path_components))
# Allow packages to find this when using module or dotkit # Allow packages to find this when using module files
run_env.prepend_path('GOPATH', ':'.join( run_env.prepend_path('GOPATH', ':'.join(
[dependent_spec.prefix] + path_components)) [dependent_spec.prefix] + path_components))