b1d129e681
* Module files now are generated using a template engine refers #2902 #3173 jinja2 has been hooked into Spack. The python module `modules.py` has been splitted into several modules under the python package `spack/modules`. Unit tests stressing module file generation have been refactored accordingly. The module file generator for Lmod has been extended to multi-providers and deeper hierarchies. * Improved the support for templates in module files. Added an entry in `config.yaml` (`template_dirs`) to list all the directories where Spack could find templates for `jinja2`. Module file generators have a simple override mechanism to override template selection ('modules.yaml' beats 'package.py' beats 'default'). * Added jinja2 and MarkupSafe to vendored packages. * Spec.concretize() sets mutual spec-package references The correct place to set the mutual references between spec and package objects at the end of concretization. After a call to concretize we should now be ensured that spec is the same object as spec.package.spec. Code in `build_environment.py` that was performing the same operation has been turned into an assertion to be defensive on the new behavior. * Improved code and data layout for modules and related tests. Common fixtures related to module file generation have been extracted in `conftest.py`. All the mock configurations for module files have been extracted from python code and have been put into their own yaml file. Added a `context_property` decorator for the template engine, to make it easy to define dictionaries out of properties. The default for `verbose` in `modules.yaml` is now False instead of True. * Extendable module file contexts + short description from docstring The contexts that are used in conjunction with `jinja2` templates to generate module files can now be extended from package.py and modules.yaml. Module files generators now infer the short description from package.py docstring (and as you may expect it's the first paragraph) * 'module refresh' regenerates all modules by default `module refresh` without `--module-type` specified tries to regenerate all known module types. The same holds true for `module rm` Configure options used at build time are extracted and written into the module files where possible. * Fixed python3 compatibility, tests for Lmod and Tcl. Added test for exceptional paths of execution when generating Lmod module files. Fixed a few compatibility issues with python3. Fixed a bug in Tcl with naming_scheme and autoload + unit tests * Updated module file tutorial docs. Fixed a few typos in docstrings. The reference section for module files has been reorganized. The idea is to have only three topics at the highest level: - shell support + spack load/unload use/unuse - module file generation (a.k.a. APIs + modules.yaml) - module file maintenance (spack module refresh/rm) Module file generation will cover the entries in modules.yaml Also: - Licenses have been updated to include NOTICE and extended to 2017 - docstrings have been reformatted according to Google style * Removed redundant arguments to RPackage and WafPackage. All the callbacks in `RPackage` and `WafPackage` that are not build phases have been modified not to accept a `spec` and a `prefix` argument. This permits to leverage the common `configure_args` signature to insert by default the configuration arguments into the generated module files. I think it's preferable to handling those packages differently than `AutotoolsPackage`. Besides only one package seems to override one of these methods. * Fixed broken indentation + improved resiliency of refresh Fixed broken indentation in `spack module refresh` (probably a rebase gone silently wrong?). Filter the writers for blacklisted specs before searching for name clashes. An error with a single writer will not stop regeneration, but instead will print a warning and continue the command.
88 lines
3.2 KiB
Python
88 lines
3.2 KiB
Python
# -*- coding: utf-8 -*-
|
|
"""
|
|
jinja2.visitor
|
|
~~~~~~~~~~~~~~
|
|
|
|
This module implements a visitor for the nodes.
|
|
|
|
:copyright: (c) 2017 by the Jinja Team.
|
|
:license: BSD.
|
|
"""
|
|
from jinja2.nodes import Node
|
|
|
|
|
|
class NodeVisitor(object):
|
|
"""Walks the abstract syntax tree and call visitor functions for every
|
|
node found. The visitor functions may return values which will be
|
|
forwarded by the `visit` method.
|
|
|
|
Per default the visitor functions for the nodes are ``'visit_'`` +
|
|
class name of the node. So a `TryFinally` node visit function would
|
|
be `visit_TryFinally`. This behavior can be changed by overriding
|
|
the `get_visitor` function. If no visitor function exists for a node
|
|
(return value `None`) the `generic_visit` visitor is used instead.
|
|
"""
|
|
|
|
def get_visitor(self, node):
|
|
"""Return the visitor function for this node or `None` if no visitor
|
|
exists for this node. In that case the generic visit function is
|
|
used instead.
|
|
"""
|
|
method = 'visit_' + node.__class__.__name__
|
|
return getattr(self, method, None)
|
|
|
|
def visit(self, node, *args, **kwargs):
|
|
"""Visit a node."""
|
|
f = self.get_visitor(node)
|
|
if f is not None:
|
|
return f(node, *args, **kwargs)
|
|
return self.generic_visit(node, *args, **kwargs)
|
|
|
|
def generic_visit(self, node, *args, **kwargs):
|
|
"""Called if no explicit visitor function exists for a node."""
|
|
for node in node.iter_child_nodes():
|
|
self.visit(node, *args, **kwargs)
|
|
|
|
|
|
class NodeTransformer(NodeVisitor):
|
|
"""Walks the abstract syntax tree and allows modifications of nodes.
|
|
|
|
The `NodeTransformer` will walk the AST and use the return value of the
|
|
visitor functions to replace or remove the old node. If the return
|
|
value of the visitor function is `None` the node will be removed
|
|
from the previous location otherwise it's replaced with the return
|
|
value. The return value may be the original node in which case no
|
|
replacement takes place.
|
|
"""
|
|
|
|
def generic_visit(self, node, *args, **kwargs):
|
|
for field, old_value in node.iter_fields():
|
|
if isinstance(old_value, list):
|
|
new_values = []
|
|
for value in old_value:
|
|
if isinstance(value, Node):
|
|
value = self.visit(value, *args, **kwargs)
|
|
if value is None:
|
|
continue
|
|
elif not isinstance(value, Node):
|
|
new_values.extend(value)
|
|
continue
|
|
new_values.append(value)
|
|
old_value[:] = new_values
|
|
elif isinstance(old_value, Node):
|
|
new_node = self.visit(old_value, *args, **kwargs)
|
|
if new_node is None:
|
|
delattr(node, field)
|
|
else:
|
|
setattr(node, field, new_node)
|
|
return node
|
|
|
|
def visit_list(self, node, *args, **kwargs):
|
|
"""As transformers may return lists in some places this method
|
|
can be used to enforce a list as return value.
|
|
"""
|
|
rv = self.visit(node, *args, **kwargs)
|
|
if not isinstance(rv, list):
|
|
rv = [rv]
|
|
return rv
|