Using "compilers" with the "s" is an invalid config section and throws an error.
Traceback (most recent call last):
File "spack/bin/spack", line 48, in <module>
sys.exit(spack.main.main())
File "/home/omsai/src/libkmap/spack/lib/spack/spack/main.py", line 633, in main
env = ev.find_environment(args)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 263, in find_environment
return Environment(env)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 534, in __init__
self._read_manifest(f)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 561, in _read_manifest
self.yaml = _read_yaml(f)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 402, in _read_yaml
validate(data, filename)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 395, in validate
e, data, filename, e.instance.lc.line + 1)
spack.config.ConfigFormatError: /home/omsai/src/libkmap/spack.yaml:15: Additional properties are not allowed ('compilers' was unexpected)
Fixes#11781
* Rename build log to spack-build-log.txt
* Rename environment variables file to spack-build-env.txt
* The name of the log and env files is now the same during the build
and after the build completes
* Update packages which referred to the build log/env files
* For packages installed before this commit using older names for the
build and env files, search for the older names
Add an example of a 'modules:' entry for an external package in
packages.yaml. The 'External Packages' section of 'Build
Customization' mentions 'paths:' and 'modules:' and gives an
example of paths, but not modules.
* config:build_jobs now controls the number of parallel jobs to spawn during
builds, but cannot ever exceed the number of cores on the machine.
* The default is set to 16 or the number of available cores, whatever
is lowest.
* Updated docs to reflect the changes done to limit parallel builds
- `gettext_uuid=True` makes every commit update every .pot file in spack/localized-docs,
and speeds up the internationalized doc build slightly.
- Optimize for less repository churn, and use `python-levenshtein` to accelerate
the build instead.
- make all Spack paths relative to a `_spack_root` symlink, so that we
can easily relocate the docs build *outside* lib/spack/docs
- set some useful defaults for gettext translation variables in conf.py
- update `relativeinclude` and other references to the spack root in the
RST files to use _spack_root
- Add a `--update FILE` option to `spack list`
- Output is written to the file only if any package is newer than the file
- Simplify the code in docs/conf.py using this new option
The Spack documentation currently hard-codes some functionality in
`conf.py`, which makes the doc build less "pluggable" for things like
localized doc builds.
In particular, we unconditionally generate an index of commands and a
package list as part of the docs, but those should really only be done if
things are not up to date.
This commit does the following:
- Add `--header` option to `spack commands` so that it can do the work of
prepending text to its output.
- Add `--update FILE` option to `spack commands` that makes it generate a
new command index *only* if FILE is out of date w.r.t. commands in the
Spack source.
- Simplify code in `conf.py` to use these options and only update the
command index when needed.
#8612 added command extensions to Spack: a command implemented in a
separate directory. This improves the implementation by allowing
the command to import additional utility code stored within the
established directory structure for commands.
This also:
* Adds tests for command extensions
* Documents command extensions (including the expected directory
layout)
* extend Version class so that 2.0 > 1.develop > 1.1
* add concretization tests, with preferences and preferred version.
* add master, head, trunk as develop-like versions, develop > master > head > trunk
* update documentation on version comparison
#11152 added documentation for #8772 but some details were based on
an earlier implementation that had changed by the time #8772 was
merged. In particular, #11152 mentioned that upstream Spack instances
were configured in config.yaml, when in fact they should be placed in
a separate upstreams.yaml config file; this PR updates the
documentation accordingly.
* Update spec format to simpler syntax, maintain backwards compatibility
* Switch to new spec.format method throughout internals
* update package files for new format strings
* documentation and minor code cleanup. removed nonsensical variant sigils
- add CombinatorialSpecSet in spack.util.spec_set module.
- class is iterable and encaspulated YAML parsing and validation.
- Adjust YAML format to be more generic
- YAML spec-set format now has a `matrix` section, which can contain
multiple lists of specs, generated different ways. Including:
- specs: a raw list of specs.
- packages: a list of package names and versions
- compilers: a list of compiler names and versions
- All of the elements of `matrix` are dimensions for the build matrix;
we take the cartesian product of these lists of specs to generate a
build matrix. This means we can add things like [^mpich, ^openmpi]
to get builds with different MPI versions. It also means we can
multiply the build matrix out with lots of different parameters.
- Add a schema format for spec-sets
* Note that `none` is the default for lmod autoload
Save a bit of confusion by *explicitly* pointing out that `none` is
the default value for autoload in the lmod module file generator.
* Add a tip re building software externally
Add a tip about using `autoload: all` when building packages outside
of the tree that use artifacts (e.g. libraries, includes) within the
tree.
Allow customizing views with Spec-formatted directory structure
Allow views to specify projections that are more complicated than
merging every package into a single shared prefix. This will allow
sites to configure a view for the way they want to present packages
to their users; for example this can be used to create a prefix for
each package but omit the DAG hash from the path.
This includes a new YAML format file for specifying the simplified
prefix for a spec in a view. This configuration allows the use of
different prefix formats for different specs (i.e. specs depending
on MPI can include the MPI implementation in the prefix).
Documentation on usage of the view projection configuration is
included.
Depending on the projection configuration, paths are not guaranteed
to be unique and it may not be possible to add multiple installs of
a package to a view.
This enforces conventions that allow for correct handling of
multi-valued variants where specifying no value is an option,
and adds convenience functionality for specifying multi-valued
variants with conflicting sets of values. This also adds a notion
of "feature values" for variants, which are those that are understood
by the build system (e.g. those that would appear as configure
options). In more detail:
* Add documentation on variants to the packaging guide
* Forbid usage of '' or None as a possible variant value, in
particular as a default. To indicate choosing no value, the user
must explicitly define an option like 'none'. Without this,
multi-valued variants with default set to None were not parsable
from the command line (Fixes#6314)
* Add "disjoint_sets" function to support the declaration of
multi-valued variants with conflicting sets of options. For example
a variant "foo" with possible values "a", "b", and "c" where "c"
is exclusive of the other values ("foo=a,b" and "foo=c" are
valid but "foo=a,c" is not).
* Add "any_combination_of" function to support the declaration of
multi-valued variants where it is valid to choose none of the
values. This automatically defines "none" as an option (exclusive
with all other choices); this value does not appear when iterating
over the variant's values, for example in "with_or_without" (which
constructs autotools option strings from variant values).
* The "disjoint_sets" and "any_combination_of" methods return an
object which tracks the possible values. It is also possible to
indicate that some of these values do not correspond to options
understood by the package's build system, such that methods like
"with_or_without" will not define options for those values (this
occurs automatically for "none")
* Add documentation for usage of new functions for specifying
multi-valued variants
- currently just looks at patches
- allows you to find out which package applied a patch to a spec
- intended to work with tarballs and resources in the future.
- add tab completion for `spack resource` and subcommands
* Remove /nfs/tmp2 from default configuration
* /nfs/tmp2 is going away from LC... and doesn’t exist for the rest of the world.
* update documentation to remove /nfs/tmp2 as well
* "spack install" now uses cache by default, update examples accordingly
* Replace some example packages with others
* Packing tutorial reference to "spack env" replaced with "spack build-env"
* Command line prompts in examples are shortened
* Example output (including paths) are updated to be more relevant to training environment
Update all examples that need an MPI provider to build with MPICH; reorganize so that fixing MPICH (as part of environment section) comes first in the tutorial (most examples in the tutorial use an MPI provider).
* Update Makefile to use property methods ("build_targets"/"install_targets")
to demonstrate their usage
* Fix highlighting
* Change cbench example to ESMF:
CBench package file was changed and no longer uses the example shown in
the old docs
Scopes added with -C are now referred to as "custom scopes"
rather than "command line scopes". "command line scope" now refers
to specific config options that are set on the command line (like
"--insecure")
- `spack.util.environment` is the new home for routines that modify
environment variables.
- This is to make room for `spack.environment` to contain new routines
for dealing with spack environments
* modified tutorial packages
* update hint in hdf5 tutorial file (typo for suggested argument)
* add repo.yaml to tutorial repository
* update tutorial docs to refer user to tutorial package repository
* flake edits
* recommend site scope vs. defaults
* you don't specify the repo's name when adding a repo, just the path
* Unite Dockerfiles - add build/run/push scripts
* update docker documentation
* update .travis.yml
* switch to using a preprocessor on Dockerfiles
* skip building docker images on pull requests
* update files with copyright info
* tweak when travis builds for docker files are done
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
- core and most packages are (Apache-2.0 OR MIT)
- a very small number of remaining packages are LGPL-2.1-only
compilers.yaml can track a module that is needed for a compiler, but
Spack does not fill this in automatically. This adds a note to the
documentation informing the user how to do this.
Spack can now be configured to assign permissions to the files installed by a package.
In the `packages.yaml` file under `permissions`, the attributes `read`, `write`, and `group` control the package permissions. These attributes can be set per-package, or for all packages under `all`. If permissions are set under `all` and for a specific package, the package-specific settings take precedence. The `read` and `write` attributes take one of `user`, `group`, and `world`.
packages:
all:
permissions:
write: group
group: spack
my_app:
permissions:
read: group
group: my_team
* Push default flag handlers into module scope
* Preserve backwards compatibility of builtin flag handler names
Ensure Spack continues to work for packages using the `Package.env_flags` idiom and equivalent.
* update docs and tests to match
* Update packages to match new syntax
Consolidate prefix calculation logic for intel packages into the
IntelPackage class.
Add documentation on installing Intel packages with Spack an
(alternatively) adding them as external packages in Spack.
- Support for Python 3.3 isn't really needed, as nothing uses it as the
default system Python, and nearly everyone will have a newer Python 3
version installed.
* Branch with the meson build-system
* Fix build_environment for dual loads and add create code
* Add documentation
* Fixed option list
* Update build_system_guess for meson
* Fixed documentation errors
* Added meson to build and configure and updated documentation
* fix typos
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.
Unit tests have been refactored to match the new command structure.
fixes#4400
The feature requested in #4400 was already part of the module file
configuration, but it was neither tested nor documented. This
commit takes care of adding a few lines in the documentation and a
regression test.
'spack module' has been split into multiple commands, each one tied to a
specific module type. This permits the specialization of the new
commands with features that are module type specific (e.g. set the
default module file in lmod when multiple versions of the same package
are installed at the same time).
Spack provides a number of classes based on commonly-used build systems
that users can extend when writing packages; the classes provide functionality
to perform the actions relevant to the build system (e.g. running "configure" for
an Autotools-based package). This adds documentation for classes supporting the
following build systems:
* Makefile
* Autotools
* CMake
* QMake
* SCons
* Waf
This includes build systems for managing extensions of the following packages:
* Perl
* Python
* R
* Octave
This also adds documentation on implementing packages that use a custom build
system (e.g. Perl/CMake).
Spack also provides extendable classes which aggregate functionality for related
sets of packages, e.g. those using CUDA. Documentation is added for
CudaPackage.
If the user sets "ccache: true" in spack's config.yaml, Spack will use an available
ccache executable when compiling c/c++ code. This feature is disabled by default
(i.e. "ccache: false") and the documentation is updated with how to enable
ccache support
Functional updates:
- `python` now creates a copy of the `python` binaries when it is added
to a view
- Python extensions (packages which subclass `PythonPackage`) rewrite
their shebang lines to refer to python in the view
- Python packages in the same namespace will not generate conflicts if
both have `...lib/site-packages/namespace-example/__init__.py`
- These `__init__` files will also remain when removing any package in
the namespace until the last package in the namespace is removed
Generally (Updated 2/16):
- Any package can define `add_files_to_view` to customize how it is added
to a view (and at the moment custom definitions are included for
`python` and `PythonPackage`)
- Likewise any package can define `remove_files_from_view` to customize
which files are removed (e.g. you don't always want to remove the
namespace `__init__`)
- Any package can define `view_file_conflicts` to customize what it
considers a merge conflict
- Global activations are handled like views (where the view root is the
spec prefix of the extendee)
- Benefit: filesystem-management aspects of activating extensions are
now placed in views (e.g. now one can hardlink a global activation)
- Benefit: overriding `Package.activate` is more straightforward (see
`Python.activate`)
- Complication: extension packages which have special-purpose logic
*only* when activated outside of the extendee prefix must check for
this in their `add_files_to_view` method (see `PythonPackage`)
- `LinkTree` is refactored to have separate methods for copying a
directory structure and for copying files (since it was found that
generally packages may want to alter how files are copied but still
wanted to copy directories in the same way)
TODOs (updated 2/20):
- [x] additional testing (there is some unit testing added at this point
but more would be useful)
- [x] refactor or reorganize `LinkTree` methods: currently there is a
separate set of methods for replicating just the directory structure
without the files, and a set for replicating everything
- [x] Right now external views (i.e. those not used for global
activations) call `view.add_extension`, but global activations do not
to avoid some extra work that goes into maintaining external views. I'm
not sure if addressing that needs to be done here but I'd like to
clarify it in the comments (UPDATE: for now I have added a TODO and in
my opinion this can be merged now and the refactor handled later)
- [x] Several method descriptions (e.g. for `Package.activate`) are out
of date and reference a distinction between global activations and
views, they need to be updated
- [x] Update aspell package activations
The following improvements are made to cxx standard support
(e.g. compiler.cxxNN_flag functions) in compilers:
* Add cxx98_flag property
* Add support for throwing an exception when a flag is not supported (previously
if a flag was not supported the application was terminated with tty.die)
* The name of the flag associated with e.g. c++14 standard support changes for
different compiler versions (e.g. c++1y vs c++14). This makes a few corrections
on what flag to return for which version.
* Added tests to confirm that versions report expected flags for various c++
standards (or raise an exception for versions that don't provide a given cxx
standard)
Note that if a given cxx standard is the default, the associated flag property will
return ""; cxx98 is assumed to be the default standard so this is the behavior for
the associated property in the base compiler class.
Package changes:
* Improvements to the boost spec to take advantage of the improved standard
flag facility.
* Update the clingo spec to catch the new exception rather than look for an
empty flag to indicate non-support (which is not part of the compiler flag API)
* extend Prefix class with join() member to support dynamic directories
* add more tests for Prefix.join()
* more tests for Prefix.join()
* add docstring
* add example to docstring of Prefix class
* cleanup Prefix.join() tests
* use Prefix.join() in Packaging Guide
- Spack packages were originally expected to call `from spack import *`
themselves, but it has become difficult to manage imports in the
Spack core.
- the top-level namespace polluted by package symbols, and it's not
possible to avoid circular dependencies and unnecessary module loads in
the core, given all the stuff the packages need.
- This makes the top-level `spack` package essentially empty, save for a
version tuple and a version string, and `from spack import *` is now
essentially a no-op.
- The common routines and directives that packages need are now in
`spack.pkgkit`, and the import system forces packages to automatically
include this so that old packages that call `from spack import *`
will continue to work without modification.
- Since `from spack import *` is no longer required, we could consider
removing ``from spack import *`` from packages in the future and
shifting to ``from spack.pkgkit import *``, but we can wait a while to
do this.
- spack.util.lock behaves the same as llnl.util.lock, but Lock._lock and
Lock._unlock do nothing.
- can be disabled with a control variable.
- configuration options can enable/disable locking:
- `locks` option in spack configuration controls whether Spack will use filesystem locks or not.
- `-l` and `-L` command-line options can force-disable or force-enable locking.
- Spack will check for group- and world-writability before disabling
locks, and it will not allow a group- or world-writable instance to
have locks disabled.
- update documentation
- `spack.cmd.all_commands` does a directory listing on
`lib/spack/spack/cmd`, regardless of whether it is needed
- make this lazy so that the directory listing won't happen unless it's
necessary.
- spack.repository module is now spack.repo
- `spack.repo` is now `spack.repo.path()` and loaded lazily
- Added `spack.repo.get()` and `spack.repo.all_package_names()` as
convenience functions to simplify the new lazy interface.
- updated tests and code
- no longer require `spack_version` to be a Version (it isn't used that
way anyway)
- use a simple tuple `spack_version_info` with major, minor, patch
versions
- generate `spack_version` from the tuple
- command reference now includes usage for all Spack commands as output
by `spack help`. Each command usage links to any related section in
the docs.
- added `spack commands` command which can list command names,
subcommands, and generate RST docs for commands.
- added `llnl.util.argparsewriter`, which analyzes an argparse parser and
calls hooks for description, usage, options, and subcommands
This reorganizes most sections and rewords a significant portion of
the content (including all introductions) but keeps all the examples.
* Remove section 'What happens at subscript time' from tutorial:
it is too detailed for a tutorial
* Move the 'Extra query parameters' and 'Attach attributes to other
packages' sections into a separate grouping 'Other packaging topics'
* move the 'Set variables at build time yourself' section after
'Set environment variables in dependents' section since the latter
is more motivating
* start the 'set environment variables at build-time for yourself'
section with qt as an example
* renamed section 'specs build interface' to 'retrieving library
information' and updated section introduction
* renamed section 'a motivating example' to 'accessing library
dependencies'; split out the material which deals with implementing
.libs for netlib-lapack into a separate section called 'providing
libraries to dependents'. consolidated in material from the section
'single package providing multiple virtual specs' since
netlib-lapack is an example of this (this removes the material
about intel-parallel studio)
- Generating the HTML from for >2300 packages from RST in Sphinx seems to
take forever.
- Add an option to `spack list` to generate straight HTML instead.
- This reduces the doc build time to about a minute (from 5 minutes on a mac laptop).
* add OctavePackage
1. remove import CudaPackage which is not needed anymore
2. mention CudaPackage and OctavePackage in packaging guide
3. adjust OctavePackageTemplate
4. add clue file for Octave build
5. sanity check on self.prefix
* use setup_environment
This adds the ability for packages to apply compiler flags in one of
three ways: by injecting them into the compiler wrapper calls (the
default in this PR and previously the only automated choice);
exporting environment variable definitions for variables with
corresponding names (e.g. CPPFLAGS=...); providing them as arguments
to the build system (e.g. configure).
When applying compiler flags using build system arguments, a package
must implement the 'flags_to_build_system_args" function. This is
provided for CMake and autotools packages, so for packages which
subclass those build systems, they need only update their flag
handler method specify which compiler flags should be specified as
arguments to the build system.
Convenience methods are provided to specify that all flags be applied
in one of the 3 available ways, so a custom implementation is only
required if more than one method of applying compiler flags is
needed.
This also removes redundant build system definitions from tutorial
examples
Fixes#2440
The "Getting started" guide should be short and sweet. This commit
simplifies the "Environment-Modules" section pruning:
- outdated / wrong suggestions as noted in #2440
- uncommon setups that are better treated in a reference guide
* First draft for SC17 build systems portion
Added tutorial_buildsystems.rst file as well as example files under
the tutorial/ directory.
* Remove floating `
* Add requested changes, and examples of subclasses
Added in the requested changes to the documentation. Also added in
information about the subclasses and the defaults that they provide.
Also fixed some phrasing issues, formatting and punctuation.
* Flake8 fixes and new files for classes
Made flake8 fixes to pass tests and also added files to demonstrate code
in the classes.
* Minor edits
Edits in formatting and made some sentence changes
* Flake8 fixes
More flake8 fixes
* Flake8 fix
* Change section order on tutorial and minor edits
Placed the section at the appropriate section for the tutorial and then
added some minor edits that were requested.
* Add requested changes and more details
Added more details to Cmake, Makefile and Python Packages.
* Fixed formatting and minor edits
* Fix doc build error
* Allow types and 'any' in variant definitions.
- Previously variant values had to be a tuple or a callable predicate.
- This allows 'any' as shorthand for `lambda x: True` and type objects
as shorthand for "any value of this type".
- Makes variant definitions more readable, keeps lambdas out of
packages for common cases.
* Update packaging tutorial
* Fix bad file reference in packaging tutorial
* First draft of the advanced packaging tutorial
* advanced packaging tutorial: improved phrasing
Thanks Denis and Hartzell!
* Fixed typos + reworded a couple of sentences
* Reworked module file tutorial section
First draft for the SC17 update. This includes:
- adding an introduction on module files + Spack's module
generation blueprints
- adding a set-up section and provide a docker image for easy set-up
- updating all the relevant snippets
- extending a bit some of the concepts that were already touched
* Added reference to #5582 + committed Dockerfiles
Also fixed a couple of typos spotted by Denis.
* module file tutorial: added section on template customization
* module file tutorial: fixed minor typos + rephrased a sentence
* module file tutorial: made explicit that Docker image comes with software
* module file tutorial: improved phrasing and layout.
Thanks Hartzell!
* module file tutorial: added vim and nano to editors
* module file tutorial: fixed typo
* Fixed typos
Thanks Adam!
* module file tutorial: updated Dockerfile + minor changes in introduction
- This isn't one of those autogenerated SVGs from a drawing program!
- This is a completely re-traced, minimalist SVG file with clearly
delineated pieces so that your favorite renderer can draw a Spack logo
at whatever resolution you want.
- Included versions with text, as well.
* When creating a tar of a package for a build cache, symlinks are
preserved (the corresponding path in the newly-created tarfile will
be a symlink rather than a copy of the file)
* Dont add external packages to a build cache
* When installing from binary cache, don't create install prefix until
verification is complete
* Update Getting Started docs to clarify that full Xcode suite is required for qt
* Better error message when only the command-line tools are installed
- The shell script uses arrays and hence only works on sophisticated shells and not the default `sh`. For clarity the shebang `#!/bin/bash` has been used instead.
This adds a workflow section on how to use spack on Docker.
It provides an example on the best-practices I collected over the
last months and circumvents the common pitfalls I tapped in.
Works with MPI, CUDA, Modules, execution as root, etc.
Background: Developed initially for PIConGPU.
* Module files now are generated using a template engine refers #2902#3173
jinja2 has been hooked into Spack.
The python module `modules.py` has been splitted into several modules
under the python package `spack/modules`. Unit tests stressing module
file generation have been refactored accordingly.
The module file generator for Lmod has been extended to multi-providers
and deeper hierarchies.
* Improved the support for templates in module files.
Added an entry in `config.yaml` (`template_dirs`) to list all the
directories where Spack could find templates for `jinja2`.
Module file generators have a simple override mechanism to override
template selection ('modules.yaml' beats 'package.py' beats 'default').
* Added jinja2 and MarkupSafe to vendored packages.
* Spec.concretize() sets mutual spec-package references
The correct place to set the mutual references between spec and package
objects at the end of concretization. After a call to concretize we
should now be ensured that spec is the same object as spec.package.spec.
Code in `build_environment.py` that was performing the same operation
has been turned into an assertion to be defensive on the new behavior.
* Improved code and data layout for modules and related tests.
Common fixtures related to module file generation have been extracted
in `conftest.py`. All the mock configurations for module files have been
extracted from python code and have been put into their own yaml file.
Added a `context_property` decorator for the template engine, to make
it easy to define dictionaries out of properties.
The default for `verbose` in `modules.yaml` is now False instead of True.
* Extendable module file contexts + short description from docstring
The contexts that are used in conjunction with `jinja2` templates to
generate module files can now be extended from package.py and
modules.yaml.
Module files generators now infer the short description from package.py
docstring (and as you may expect it's the first paragraph)
* 'module refresh' regenerates all modules by default
`module refresh` without `--module-type` specified tries to
regenerate all known module types. The same holds true for `module rm`
Configure options used at build time are extracted and written into the
module files where possible.
* Fixed python3 compatibility, tests for Lmod and Tcl.
Added test for exceptional paths of execution when generating Lmod
module files.
Fixed a few compatibility issues with python3.
Fixed a bug in Tcl with naming_scheme and autoload + unit tests
* Updated module file tutorial docs. Fixed a few typos in docstrings.
The reference section for module files has been reorganized. The idea is
to have only three topics at the highest level:
- shell support + spack load/unload use/unuse
- module file generation (a.k.a. APIs + modules.yaml)
- module file maintenance (spack module refresh/rm)
Module file generation will cover the entries in modules.yaml
Also:
- Licenses have been updated to include NOTICE and extended to 2017
- docstrings have been reformatted according to Google style
* Removed redundant arguments to RPackage and WafPackage.
All the callbacks in `RPackage` and `WafPackage` that are not build
phases have been modified not to accept a `spec` and a `prefix`
argument. This permits to leverage the common `configure_args` signature
to insert by default the configuration arguments into the generated
module files. I think it's preferable to handling those packages
differently than `AutotoolsPackage`. Besides only one package seems
to override one of these methods.
* Fixed broken indentation + improved resiliency of refresh
Fixed broken indentation in `spack module refresh` (probably a rebase
gone silently wrong?). Filter the writers for blacklisted specs before
searching for name clashes. An error with a single writer will not
stop regeneration, but instead will print a warning and continue
the command.
Renames the existing bootstrap command to 'clone'. Repurposes
'spack bootstrap' to install packages that are useful to the
operation of Spack (for now this is just environment-modules).
For bash and ksh users running setup-env.sh, if a Spack-installed
instance of environment-modules is detected and environment modules
and dotkit are not externally available, Spack will define the
'module' command in the user's shell to use the environment-modules
built by Spack.
* Docs: Travis-CI Workflow
Add a workflow how to use spack on Travis-CI.
Future Work:
depending if and how we can simplify 5101:
add a multi-compiler, multi-C++-standard, multi-software
build matrix example
* Fix Typos
* Refactor IntelInstaller into IntelPackage base class
* Move license attributes from __init__ to class-level
* Flake8 fixes: remove unused imports
* Fix logic that writes the silent.cfg file
* More specific version numbers for Intel MPI
* Rework logic that selects components to install
* Final changes necessary to get intel package working
* Various updates to intel-parallel-studio
* Add latest version of every Intel package
* Add environment variables for Intel packages
* Update env vars for intel package
* Finalize components for intel-parallel-studio package
Adds a +tbb variant to intel-parallel-studio.
The tbb package was renamed to intel-tbb.
Now both intel-tbb and intel-parallel-studio+tbb
provide tbb.
* Overhaul environment variables set by intel-parallel-studio
* Point dependent packages to the correct MPI wrappers
* Never default to intel-parallel-studio
* Gather env vars by sourcing setup scripts
* Use mpiicc instead of mpicc when using Intel compiler
* Undo change to ARCH
* Add changes from intel-mpi to intel-parallel-studio
* Add comment explaining mpicc vs mpiicc
* Prepend env vars containing 'PATH' or separators
* Flake8 fix
* Fix bugs in from_sourcing_file
* Indentation fix
* Prepend, not set if contains separator
* Fix license symlinking broken by changes to intel-parallel-studio
* Use comments instead of docstrings to document attributes
* Flake8 fixes
* Use a set instead of a list to prevent duplicate components
* Fix MKL and MPI library linking directories
* Remove +all variant from intel-parallel-studio
* It is not possible to build with MKL, GCC, and OpenMP at this time
* Found a workaround for locating GCC libraries
* Typos and variable names
* Fix initialization of empty LibraryList
Adds the "buildcache" command to spack. The buildcache command is
used to create gpg signatures for archives of installed spack
packages; the signatures and archives are placed together in a
directory that can be added to a spack mirror. A user can retrieve
the archives from a mirror and verify their integrity using the
buildcache command. It is often the case that the user's Spack
instance is located in a different path compared to the Spack
instance used to generate the package archive and signature, so
this includes logic to relocate the RPATHs generated by Spack.
* Merged 'purge' command with 'clean'. Deleted 'purge'. fixes#2942
'spack purge' has been merged with 'spack clean'. Documentation has been
updated accordingly. The 'clean' and 'purge' behavior are not mutually
exclusive, and they log brief information to tty while they go.
* Fixed a wrong reference to spack clean in the docs
* Added tests for 'spack clean'. Updated bash completion.
* Add a QMakePackage base class
* Fix sqlite linking bug in qt-creator
* Add latest version of qt-creator
* Add latest version of qwt
* Use raw strings for regular expressions
* Increase minimum required version of qt
* Add comment about specific version of sqlite required
* Fixes for latest version of qwt and qt-creator
* Older versions of Qwt only work with older versions of Qt
* Change version.up_to() to return Version() object
* Add unit tests for Version.up_to()
* Fix packages that expected up_to() to return a string
* Ensure that up_to() preserves separator characters
* Use version indexing instead of up_to
* Make all Version formatting properties return Version objects
* Update docs
* Tests need to test string representation
* Initial work on flag trapping using functions called <flag>_handler and default_flag_handler
* Update packages so they do not obliterate flags
* Added append to EnvironmentModifications class
* changed EnvironmentModifications to have append_flags method
* changed flag_val to be a tuple
* Increased test coverage
* added documentation of flag handling
* Code changes to enable system config scope in /etc
Files will go in either /etc/spack or /etc/spack/<platform>
Required minor changes to conftest.
* Updated documentation to match new config scope
- Add a `spack gpg` subcommand in anticipation of signed binaries.
- GPG keys are stored in var/spack/gpg, and the spack gpg command manages them.
- Docs are included on the command.
* Sphinx no longer supports Python 2.6
* Update vendored sphinxcontrib.programoutput from 0.9.0 to 0.10.0
* Documentation cannot be built in parallel
* Let Travis install programoutput for us
* Remove vendored sphinxcontrib-programoutput
Recent updates to the sphinx package prevent the vendored version
from being found in sys.path. We don't vendor sphinx, so it doesn't
make sense to vendor sphinxcontrib-programoutput either.
* fetch git submodules recursively
This is useful if the submodules have submodules themselves. On
the other hand doing a recursive update doesn't hurt if there
is only one level.
* fetch submodules with depth=1 as well (fix#2190)
* use git submodule with depth only for git>=1.8.4
* Remove fake URLs from Spack
* Ignore long lines for URLs that start with ftp:
* Preliminary changes to version regexes
* New redesign of version regexes
* Allow letters in version-only
* Fix detection of versions that end in Final
* Rearrange a few regexes and add examples
* Add tests for common download repositories
* Add test cases for common tarball naming schemes
* Finalize version regexes
* spack url test -> spack url summary
* Clean up comments
* Rearrange suffix checks
* Use query strings for name detection
* Remove no longer necessary url_for_version functions
* Strip off extraneous information after package name
* Add one more test
* Dot in square brackets does not need to be escaped
* Move renaming outside of parse_name_offset
* Fix versions for a couple more packages
* Fix flake8 and doc tests
* Correctly parse Python, Lua, and Bio++ package names
* Use effective URLs for mfem
* Add checksummed version to mitos
* Remove url_for_version from STAR-CCM+ package
* Revert changes to version numbers with underscores and dashes
* Fix name detection for tbb
* Correctly parse Ruby gems
* Reverted mfem back to shortened URLs.
* Updated instructions for better security
* Remove preferred=True from newest version
* Add tests for new `spack url list` flags
* Add tests for strip_name_suffixes
* Add unit tests for version separators
* Fix bugs related to parseable name but in parseable version
* Remove dead code, update docstring
* Ignore 'binary' at end of version string
* Remove platform from version
* Flip libedit version numbers
* Re-support weird NCO alpha/beta versions
* Rebase and remove one new fake URL
* Add / to beginning of regex to avoid picking up similarly named packages
* Ignore weird tar versions
* Fix bug in url parse --spider when no versions found
* Less strict version matching for spack versions
* Don't rename Python packages
* Be a little more selective, version must begin with a digit
* Re-add fake URLs
* Fix up several other packages
* Ignore more file endings
* Add parsing support for Miniconda
* Update tab completion
* XFAILS are now PASSES for 2 web tests
- Update YAML version to support Python 3
- Python 3 support for ordereddict backport
- Exclude Python3 YAML from version tests.
- Vendor six into Spack.
- Make Python version-check tests work with Python 3
- Add ability to add version check exceptions with '# nopyqver' line
comments.
* perl: make extendable and add Module::Build package
* perl: allow 'spack create' to identify perl packages from their contents
* perl-module-build: fix indenting of package docstring
* perl: split install() method for extensions into phases
* perl: auto-detect build method (Makefile.PL vs Build.PL) and define a 'check' method
* PerlPackage: use import statements similar to those in AutotoolsPackage
* PerlModule: fix detection of Build.PL
* PerlPackageTemplate: remove extraneous lines to avoid flake8 warnings
* PerlPackageTemplate: split into separate templates for Makefile.PL and Build.PL
* PerlPackage: add cross-references to docstrings
* AutotoolsPackage: fix ambiguous cross-references to avoid errors in doc tests
* PerlbuildPackageTemplate: depend on perl-module-build if Build.PL exists
* Replace `spack urls` and `spack url-parse` with `spack url`
* Allow spack url list to only list incorrect parsings
* Add spack url test reporting
* Add unit tests for new URL commands
* PackageMeta: `run_before` is an alias of `precondition`, `run_after` an alias of `sanity_check`
* PackageMeta: removed `precondition` and `sanity_check`
* PackageMeta: decorators are now free-standing
* package: modified/added docstrings. Fixed the semantics of `on_package_attributes`.
* package: added unit test assertion as side effects of install
* build_systems: factored build-time test running into base class
* r: updated decorators in package.py
* docs: updated decorator names
* documentation: reworked packaging guide to add build-system phases
* documentation: improvements to AutotoolsPackage autodocs
* build_systems: updated autodocs
* run-tests: added a few information on how to run tests fixes#2606 fixes#2605
* documentation: fixed items brought up by @davydden
* typos in docs
* consistent use of 'build system' (i.e. removed 'build-system' from docs)
* added a note on possible default implementations for build-time tests
* documentation: fixed items brought up by @citibeth
* added note to explain the difference between build system and language used in a package
* capitalized bullet items
* added link to API docs
* documentation: fixed multiple cross-references after rebase
* documentation: fixed minor issues raised by @tgamblin
* documentation: added entry in table for the `PythonPackage` class
* docs: fixed issues brought up by @citybeth in the second review
* Initial changes to spack create command
* Get 'spack create <url>' working again
* Simplify call to BuildSystemGuesser
* More verbose output of spack create
* Remove duplicated code from spack create and spack checksum
* Add better documentation to spack create docstrings
* Fix pluralization bug
* Flake8
* Update documentation on spack create and deprecate spack edit --force
* Make it more obvious when we are renaming a package
* Further deprecate spack edit --force
* Fix unit tests
* Rename default template to generic template
* Don't add automake/autoconf deps to Autotools packages
* Remove changes to default $EDITOR
* Completely remove all traces of spack edit --force
* Remove grammar changes to make the PR easier to review
* Fixed parser to eliminate need for escape quotes. TODO: Fix double call to shlex, fix spaces in spec __str__
* Fixed double shlex
* cleanup
* rebased on develop
* Fixed parsing for multiple specs; broken since #360
* Revoked elimination of the `-` sigil in the syntax, and added it back into tests
* flake8
* more flake8
* Cleaned up dead code and added comments to parsing code
* bugfix for spaces in arguments; new bug found in testing
* Added unit tests for kv pairs in parsing/lexing
* Even more flake8
* ... yet another flake8
* Allow multiple specs in install
* unfathomable levels of flake8
* Updated documentation to match parser fix
* Removing the nobuild, nolink, and alldeps dependency types in favor of being explicit.
* This will help with maintenance going forward, as adding more dependency types won't affect existing declared dependencies in weird ways.
* default deptype is still `('build', 'link')`
* Rename packages
* Upcasing depends_on() in packages.
* Downcased extends('r')
* Fixed erroneously changed URL that had slipped through.
* Fixed typo
* Fixed link from documentation into package source code.
* Fixed another doc problem.
* Changed underscores to dashes in package names.
* Added test to enforce lowercase, no-underscore naming convention.
* Fix r-xgboost
* Downcase more instances of 'R' in package auto-creation.
* Fix test.
* Converted unit test packages to use dashes not underscores
* Downcase `r` in the docs.
* Update module_file_support.rst
Fix r->R for class R.
* Porting: substitute nose with ytest
This huge commit substitutes nose with pytest as a testing system. Things done here:
* deleted external/nose as it is no longer used
* moved mock resources in their own directory 'test/mock/'
* ported two tests (cmd/find, build_system) to pytest native syntax as an example
* build_environment, log: used monkeypatch instead of try/catch
* moved global mocking of fetch_cache to an auto-used fixture
* moved global mocking from test/__init__.py to conftest.py
* made `spack test` a wrapper around pytest
* run-unit-tests: avoid running python 2.6 tests under coverage to speed them up
* use `pytest --cov` instead of coverage run to cut down testing time
* mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals
* concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls
* conftest.py, fixtures: added docstrings
* concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest
* directory_layout.py: uses fixtures instead of subclassing MockPackagesTest
* install.py: uses fixtures instead of subclassing MockPackagesTest
* optional_deps.py: uses fixtures instead of subclassing MockPackagesTest
optional_deps.py: uses fixtures instead of subclassing MockPackagesTest
* packages.py: uses fixtures instead of subclassing MockPackagesTest
* provider_index.py: uses fixtures instead of subclassing MockPackagesTest
* spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest
* multimethod.py: uses fixtures instead of subclassing MockPackagesTest
* install.py: now uses mock_archive_url
* git_fetch.py: uses fixtures instead of subclassing MockPackagesTest
* hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest
* svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest
repo.py: deleted
* test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest
* cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase
* database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database
* pytest: uncluttering fixture implementations
* database: changing the scope to 'module'
* config.py: uses fixtures instead of subclassing MockPackagesTest
* spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest
* stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory
* pytest: added docstrings to all the fixtures
* pytest: final cleanup
* build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp
* spec_syntax.py: added expected failure on parsing multiple specs closes#1976
* Add pytest and pytest-cov to Spack externals.
* Make `spack flake8` ignore externals.
* run-unit-tests runs spack test and not pytest.
* Remove all the special stuff for `spack test`
- Remove `conftest.py` magic and all the special case stuff in `bin/spack`
- Spack commands can optionally take unknown arguments, if they want to
handle them.
- `spack test` is now a command like the others.
- `spack test` now just delegates its arguments to `pytest`, but it does
it by receiving unknown arguments and NOT taking an explicit
help argument.
* Fix error in fixtures.
* Improve `spack test` command a bit.
- Now supports an approximation of the old simple interface
- Also supports full pytest options if you want them.
* Use external coverage instead of pytest-cov
* Make coverage use parallel-mode.
* change __init__.py docs to include pytest
* Ensure that every package has a license
Also fixes URLs with http://http:// doubled.
This is a continuation of #2656.
* Add license to every file in Spack
* Make sure Todd is the author of all packages
* Fix flake8 tests
* Don't license external Sphinx docs
* Don't display licenses in tutorial example packages
Also fixes typos and converts command-line examples
from tcsh to bash, which is more common
* Add documentation for repositories and namespaces.
* Update and extend repository documentation per review.
- Also add `-N` argument for `spack spec`
The advanced [Uninstalling Packages](spack.readthedocs.io/en/latest/tutorial_sc16_spack_basics.html#uninstalling-packages) via hash has a couple missing `.. code-block:: console` directives ;)
I have no idea what branch to direct this to though...
- Seems like older git versions won't be able to clone an LFS repo.
- Reverting to an external link for the slides to avoid storing an 8MB
file in the repo and to avoid using git LFS.
* Allow compiler wrapper to modify environment
This adds the ability to set environment variables in the compiler
wrappers. These are specified as part of the compilers.yaml config.
The user may also specify RPATHs in compilers.yaml that should be
added.
* Minor doc tweak
* Wordsmithing/minor-edits to module tutorial
A small set of wordsmithing, spell checking and minor edits to the fancy
new modules tutorial!
* Fix typo (sneaky z key...)
* Fix "S:" and "manual<" typos
* module files tutorial : first complete draft
- first complete draft for module files tutorial
- minor corrections to module file reference
* module file tutorial : first batch of corrections
- module avail spelled out fully
- typos from @adamjstewart
- rewording of a few sentences
* module file tutorial : first batch of corrections
- emphasized lines in yaml files
* module file tutorial : fixes according to @citibeth and @adamjstewart reviews
- used long format for command options
- reworded unclear sentence on tokes
- reworked table in reference manual to make it clearer
* module file tutorial : implemented corrections collected on site from @schlyfts
* module file tutorial : removed comment (@hartzell suggestion)
- Ported old run-flake8-tests qa script to `spack flake8` command.
- New command does not modify files in the source tree
- Copies files to a temp stage modifies them there, and runs tests.
- Updated docs and `run-flake8-tests` script to call `spack flake8`.
- Added a schema for config.yaml
- Moved install tree configuration to config.yaml
- Moved etc/spack/install.yaml to etc/spack/defaults/config.yaml
- renamed install_area to "store", to use a term in common with guix/nix.
- in `config.yaml` file, it's called the `install_tree` to be more
intuitive to users.
- `install_tree` might've worked in the code, but `install_tree` is
already a global function in the spack namespace, from
llnl.util.filesystem.
* module file support: major rework of docs
* module file support: fixed issues found by @adamjstewart
- list or enumeration should not be indented
- use console instead of bash or csh in things that are not scripts
- other typos
* module file support: fixed other issues found by @adamjstewart
- tables should not be indented
- substitute lines with pyobject to import an entire function
- get help output running commands
- typos
* module file support: fixes according to review comments
- @citibeth moved `spack module loads` after `spack load`
- @glennpj tried to clarify installation table + changes to language
- @tgamblin Removed top level section and moved the whole thing into the reference manual
* module file support: moved directive before spack module loads
* Fixed a bug causing config-specified compiler flags to be ignored.
Updated the compiler config so all flags are in a separate section.
* Updated the documentation for the `compilers.yaml` file spec.
* Implemented basic testing for the 'flags' section of compiler config.
* Fixed a few minor problems with the manual compiler config documentation.
