Backport operator precedence algorithm from here:
397b96f6d7
This eliminates unnecessary parentheses from our unparsed output and makes Spack's unparser
consistent with the one in upstream Python 3.9+, with one exception.
Our parser normalizes argument order when `py_ver_consistent` is set, so that star arguments
in function calls come last. We have to do this because Python 2's AST doesn't have information
about their actual order.
If we ever support only Python 3.9 and higher, we can easily switch over to `ast.unparse`, as
the unparsing is consistent except for this detail (modulo future changes to `ast.unparse`)
Add `astunparse` as `spack_astunparse`. This library unparses Python ASTs and we're
adding it under our own name so that we can make modifications to it.
Ultimately this will be used to make `package_hash` consistent across Python versions.
Our `jsonschema` external won't support Python 3.10, so we need to upgrade it.
It currently generates this warning:
lib/spack/external/jsonschema/compat.py:6: DeprecationWarning: Using or importing the ABCs
from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and
in 3.10 it will stop working
This upgrades `jsonschema` to 3.2.0, the latest version with support for Python 2.7. The next
version after this (4.0.0) drops support for 2.7 and 3.6, so we'll have to wait to upgrade to it.
Dependencies have been added in prior commits.
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code
Updating the vendored dependencies will be done in separate PRs
* Use gnuconfig package for config file replacement for RISC-V.
This extends the changes in #26035 to handle RISC-V. Before this change,
many packages fail to configure on riscv64 due to config.guess being too
old to know about RISC-V. This is seen out of the box when clingo fails
to build from source due to pkgconfig failing to configure, throwing
error: "configure: error: cannot guess build type; you must specify one".
* Add riscv64 architecture
* Update vendored archspec from upstream project.
These archspec updates include changes needed to support riscv64.
* Update archspec's __init__.py to reflect the commit hash of archspec being used.
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files
- [x] update all licensed files to say 2013-2021 using
`spack license update-copyright-year`
- [x] appease mypy with some additions to package.py that needed
for oneapi.py
If the Python used by Spack does not include Setuptools, then
'spack test' will fail because Spack's vendored pytest dependency
imports and uses Setuptools in some of its functions. It turns out
that Spack doesn't use the functionality those methods enable, so
this PR removes those functions and thereby allows 'spack test' to
run without Setuptools.
This commit removes the `python_version.py` unit test module
and the vendored dependencies `pyqver2.py` and `pyqver3.py`.
It substitutes them with an equivalent check done using
`vermin` that is run as a separate workflow via Github Actions.
This allows us to delete 2 vendored dependencies that are unmaintained
and substitutes them with a maintained tool.
Also, updates the list of vendored dependencies.
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
- core and most packages are (Apache-2.0 OR MIT)
- a very small number of remaining packages are LGPL-2.1-only
* Vendor ordereddict for python2.6 only
This commit substitutes the custom module 'ordereddict_backport' with
the more known 'ordereddict' and vendors it only for python 2.6. Other
supported versions of python will use 'collections.OrderedDict'.
* Use absolute imports also for python 2.6
See PEP-328 for more information on the subject
* Added provenance of vendored ordereddict
* Keep track of source and versions for external libraries
* Note source of more obscure libraries
* We aren't upgrading jsonschema after all
* Add note on modifications made to pytest
* Module files now are generated using a template engine refers #2902#3173
jinja2 has been hooked into Spack.
The python module `modules.py` has been splitted into several modules
under the python package `spack/modules`. Unit tests stressing module
file generation have been refactored accordingly.
The module file generator for Lmod has been extended to multi-providers
and deeper hierarchies.
* Improved the support for templates in module files.
Added an entry in `config.yaml` (`template_dirs`) to list all the
directories where Spack could find templates for `jinja2`.
Module file generators have a simple override mechanism to override
template selection ('modules.yaml' beats 'package.py' beats 'default').
* Added jinja2 and MarkupSafe to vendored packages.
* Spec.concretize() sets mutual spec-package references
The correct place to set the mutual references between spec and package
objects at the end of concretization. After a call to concretize we
should now be ensured that spec is the same object as spec.package.spec.
Code in `build_environment.py` that was performing the same operation
has been turned into an assertion to be defensive on the new behavior.
* Improved code and data layout for modules and related tests.
Common fixtures related to module file generation have been extracted
in `conftest.py`. All the mock configurations for module files have been
extracted from python code and have been put into their own yaml file.
Added a `context_property` decorator for the template engine, to make
it easy to define dictionaries out of properties.
The default for `verbose` in `modules.yaml` is now False instead of True.
* Extendable module file contexts + short description from docstring
The contexts that are used in conjunction with `jinja2` templates to
generate module files can now be extended from package.py and
modules.yaml.
Module files generators now infer the short description from package.py
docstring (and as you may expect it's the first paragraph)
* 'module refresh' regenerates all modules by default
`module refresh` without `--module-type` specified tries to
regenerate all known module types. The same holds true for `module rm`
Configure options used at build time are extracted and written into the
module files where possible.
* Fixed python3 compatibility, tests for Lmod and Tcl.
Added test for exceptional paths of execution when generating Lmod
module files.
Fixed a few compatibility issues with python3.
Fixed a bug in Tcl with naming_scheme and autoload + unit tests
* Updated module file tutorial docs. Fixed a few typos in docstrings.
The reference section for module files has been reorganized. The idea is
to have only three topics at the highest level:
- shell support + spack load/unload use/unuse
- module file generation (a.k.a. APIs + modules.yaml)
- module file maintenance (spack module refresh/rm)
Module file generation will cover the entries in modules.yaml
Also:
- Licenses have been updated to include NOTICE and extended to 2017
- docstrings have been reformatted according to Google style
* Removed redundant arguments to RPackage and WafPackage.
All the callbacks in `RPackage` and `WafPackage` that are not build
phases have been modified not to accept a `spec` and a `prefix`
argument. This permits to leverage the common `configure_args` signature
to insert by default the configuration arguments into the generated
module files. I think it's preferable to handling those packages
differently than `AutotoolsPackage`. Besides only one package seems
to override one of these methods.
* Fixed broken indentation + improved resiliency of refresh
Fixed broken indentation in `spack module refresh` (probably a rebase
gone silently wrong?). Filter the writers for blacklisted specs before
searching for name clashes. An error with a single writer will not
stop regeneration, but instead will print a warning and continue
the command.
* Porting: substitute nose with ytest
This huge commit substitutes nose with pytest as a testing system. Things done here:
* deleted external/nose as it is no longer used
* moved mock resources in their own directory 'test/mock/'
* ported two tests (cmd/find, build_system) to pytest native syntax as an example
* build_environment, log: used monkeypatch instead of try/catch
* moved global mocking of fetch_cache to an auto-used fixture
* moved global mocking from test/__init__.py to conftest.py
* made `spack test` a wrapper around pytest
* run-unit-tests: avoid running python 2.6 tests under coverage to speed them up
* use `pytest --cov` instead of coverage run to cut down testing time
* mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals
* concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls
* conftest.py, fixtures: added docstrings
* concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest
* directory_layout.py: uses fixtures instead of subclassing MockPackagesTest
* install.py: uses fixtures instead of subclassing MockPackagesTest
* optional_deps.py: uses fixtures instead of subclassing MockPackagesTest
optional_deps.py: uses fixtures instead of subclassing MockPackagesTest
* packages.py: uses fixtures instead of subclassing MockPackagesTest
* provider_index.py: uses fixtures instead of subclassing MockPackagesTest
* spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest
* multimethod.py: uses fixtures instead of subclassing MockPackagesTest
* install.py: now uses mock_archive_url
* git_fetch.py: uses fixtures instead of subclassing MockPackagesTest
* hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest
* svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest
repo.py: deleted
* test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest
* cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase
* database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database
* pytest: uncluttering fixture implementations
* database: changing the scope to 'module'
* config.py: uses fixtures instead of subclassing MockPackagesTest
* spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest
* stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory
* pytest: added docstrings to all the fixtures
* pytest: final cleanup
* build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp
* spec_syntax.py: added expected failure on parsing multiple specs closes#1976
* Add pytest and pytest-cov to Spack externals.
* Make `spack flake8` ignore externals.
* run-unit-tests runs spack test and not pytest.
* Remove all the special stuff for `spack test`
- Remove `conftest.py` magic and all the special case stuff in `bin/spack`
- Spack commands can optionally take unknown arguments, if they want to
handle them.
- `spack test` is now a command like the others.
- `spack test` now just delegates its arguments to `pytest`, but it does
it by receiving unknown arguments and NOT taking an explicit
help argument.
* Fix error in fixtures.
* Improve `spack test` command a bit.
- Now supports an approximation of the old simple interface
- Also supports full pytest options if you want them.
* Use external coverage instead of pytest-cov
* Make coverage use parallel-mode.
* change __init__.py docs to include pytest
* Ensure that every package has a license
Also fixes URLs with http://http:// doubled.
This is a continuation of #2656.
* Add license to every file in Spack
* Make sure Todd is the author of all packages
* Fix flake8 tests
* Don't license external Sphinx docs
* Don't display licenses in tutorial example packages
Also fixes typos and converts command-line examples
from tcsh to bash, which is more common
