Using `sys.executable` to run Python in a sub-shell doesn't always work in a virtual environment as the `sys.executable` Python is not necessarily compatible with any loaded spack/other virtual environment.
- revert use of sys.executable to print out subshell environment (#14496)
- try instead to use an available python, then if there *is not* one, use `sys.executable`
- this addresses RHEL8 (where there is no `python` and `PYTHONHOME` issue in a simpler way
When removing packages from a view, extensions were being deactivated
in an arbitrary order. Extensions must be deactivated in preorder
traversal (dependents before dependencies), so when this order was
violated the view update would fail.
This commit ensures that views deactivate extensions based on a
preorder traversal and adds a test for it.
Despite trying very hard to keep dicts out of our hash algorithm, we seem
to still accidentally add them in ways that the tests can't catch. This
can cause errors when hashes are not computed deterministically.
This fixes an error we saw with Python 3.5, where dictionary iteration
order is random. In this instance, we saw a bug when reading Spack
environment lockfiles -- The load would fail like this:
```
...
File "/sw/spack/lib/spack/spack/environment.py", line 1249, in concretized_specs
yield (s, self.specs_by_hash[h])
KeyError: 'qcttqplkwgxzjlycbs4rfxxladnt423p'
```
This was because the hashes differed depending on whether we wrote `path`
or `module` first when recomputing the build hash as part of reading a
Spack lockfile. We can fix it by ensuring a determistic iteration order.
- [x] Fix two places (one that caused an issue, and one that did
not... yet) where our to_node_dict-like methods were using regular python
dicts.
- [x] Also add a check that statically analyzes our to_node_dict
functions and flags any that use Python dicts.
The test found the two errors fixed here, specifically:
```
E AssertionError: assert [] == ['Use syaml_dict instead of ...pack/spack/spec.py:1495:28']
E Right contains more items, first extra item: 'Use syaml_dict instead of dict at /Users/gamblin2/src/spack/lib/spack/spack/spec.py:1495:28'
E Full diff:
E - []
E + ['Use syaml_dict instead of dict at '
E + '/Users/gamblin2/src/spack/lib/spack/spack/spec.py:1495:28']
```
and
```
E AssertionError: assert [] == ['Use syaml_dict instead of ...ack/architecture.py:359:15']
E Right contains more items, first extra item: 'Use syaml_dict instead of dict at /Users/gamblin2/src/spack/lib/spack/spack/architecture.py:359:15'
E Full diff:
E - []
E + ['Use syaml_dict instead of dict at '
E + '/Users/gamblin2/src/spack/lib/spack/spack/architecture.py:359:15']
```
Before this commit we used to run the entire unit test suite
in the presence of a failure. Since we currently rely a lot
on the state of the filesystem etc. the end report was most
of the time showing spurious failures that were a consequence
of the first failing test.
This PR makes unit tests exit at the first failing test
Also, pin codecov at v4.5.4 (last one supporting Python 2.6)
`ViewDescriptor.regenerate()` calls `get_all_specs()`, which reads
`spec.yaml` files, which is slow. It's fine to do this once, but
`view.remove_specs()` *also* calls it immediately afterwards.
- [x] Pass the result of `get_all_specs()` as an optional parameter to
`view.remove_specs()` to avoid reading `spec.yaml` files twice.
`ViewDescriptor.regenerate()` was copying specs and stripping build
dependencies, which clears `_hash` and other cached fields on concrete
specs, which causes a bunch of YAML hashes to be recomputed.
- [x] Preserve the `_hash` and `_normal` fields on stripped specs, as
these will be unchanged.
`spack install` previously concretized, writes the entire environment
out, regenerated views, then wrote and regenerated views
again. Regenerating views is slow, so ensure that we only do that once.
- [x] add an option to env.write() to skip view regeneration
- [x] add a note on whether regenerate_views() shouldn't just be a
separate operation -- not clear if we want to keep it as part of write
to ensure consistency, or take it out to avoid performance issues.
Environments need to read the DB a lot when installing all specs.
- [x] Put a read transaction around `install_all()` and `install()`
to avoid repeated locking
Our `LockTransaction` class was reading overly aggressively. In cases
like this:
```
1 with spack.store.db.read_transaction():
2 with spack.store.db.write_transaction():
3 ...
```
The `ReadTransaction` on line 1 would read in the DB, but the
WriteTransaction on line 2 would read in the DB *again*, even though we
had a read lock the whole time. `WriteTransaction`s were only
considering nested writes to decide when to read, but they didn't know
when we already had a read lock.
- [x] `Lock.acquire_write()` return `False` in cases where we already had
a read lock.
If a write transaction was nested inside a read transaction, it would not
write properly on release, e.g., in a sequence like this, inside our
`LockTransaction` class:
```
1 with spack.store.db.read_transaction():
2 with spack.store.db.write_transaction():
3 ...
4 with spack.store.db.read_transaction():
...
```
The WriteTransaction on line 2 had no way of knowing that its
`__exit__()` call was the last *write* in the nesting, and it would skip
calling its write function.
The `__exit__()` call of the `ReadTransaction` on line 1 wouldn't know
how to write, and the file would never be written.
The DB would be correct in memory, but the `ReadTransaction` on line 4
would re-read the whole DB assuming that other processes may have
modified it. Since the DB was never written, we got stale data.
- [x] Make `Lock.release_write()` return `True` whenever we release the
*last write* in a nest.
Lock transactions were actually writing *after* the lock was
released. The code was looking at the result of `release_write()` before
writing, then writing based on whether the lock was released. This is
pretty obviously wrong.
- [x] Refactor `Lock` so that a release function can be passed to the
`Lock` and called *only* when a lock is really released.
- [x] Refactor `LockTransaction` classes to use the release function
instead of checking the return value of `release_read()` / `release_write()`
`ViewDescriptor.regenerate()` checks repeatedly whether packages are
installed and also does a lot of DB queries. Put a read transaction
around the whole thing to avoid repeatedly locking and unlocking the DB.
`Environment.added_specs()` has a loop around calls to
`Package.installed()`, which can result in repeated DB queries. Optimize
this with a read transaction in `Environment`.
Checks for deprecated specs were repeatedly taking out read locks on the
database, which can be very slow.
- [x] put a read transaction around the deprecation check
BundlePackages use a noop fetch strategy. The mirror logic was assuming
that the fetcher had a resource to cach after performing a fetch. This adds
a special check to skip caching if the stage is associated with a
BundleFetchStrategy. Note that this should allow caching resources
associated with BundlePackages.
When updating a mirror, Spack was re-retrieving all patches (since the
fetch logic for patches is separate). This updates the patch logic to
allow the mirror logic to avoid this.
Since cache_mirror does the fetch itself, it also needs to do the
checksum itself if it wants to verify that the source stored in the
mirror is valid. Note that this isn't strictly required because fetching
(including from mirrors) always separately verifies the checksum.
The targets for the cosmetic paths in mirrrors were being calculated
incorrectly as of fb3a3ba: the symlinks used relative paths as targets,
and the relative path was computed relative to the wrong directory.
When creating a cosmetic symlink for a resource in a mirror, remove
it if it already exists. The symlink is removed in case the logic to
create the symlink has changed.
* Some packages (e.g. mpfr at the time of this patch) can have patches
with the same name but different contents (which apply to different
versions of the package). This appends part of the patch hash to the
cache file name to avoid conflicts.
* Some exceptions which occur during fetching are not a subclass of
SpackError and therefore do not have a 'message' attribute. This
updates the logic for mirroring a single spec (add_single_spec)
to produce an appropriate error message in that case (where before
it failed with an AttributeError)
* In various circumstances, a mirror can contain the universal storage
path but not a cosmetic symlink; in this case it would not generate
a symlink. Now "spack mirror create" will create a symlink for any
package that doesn't have one.
`spack module loads` and `spack module find` previously failed if any upstream modules were missing. This prevented it from being used with upstreams (or, really, any spack instance) that blacklisted modules.
This PR makes module finding is now more lenient (especially for blacklisted modules).
- `spack module find` now does not report an error if the spec is blacklisted
- instead, it prints a single warning if any modules will be omitted from the loads file
- It comments the missing modules out of the loads file so the user can see what's missing
- Debug messages are also printed so users can check this with `spack -d...`
- also added tests for new functionality
* Add a transaction around repeated calls to `spec.prefix` in the activation process
* cache the computation of home in the python package to speed up setting deps
* ensure that module-scope variables are only set *once* per module
`mirror_archive_path` was failing to account for the case where the fetched version isn't known to Spack.
- [x] don't require the fetched version to be in `Package.versions`
- [x] add regression test for mirror paths when package does not have a version
* remove reference to `spack.store` in method definition
Referencing `spack.store` in method definition will cache the `spack.config.config` singleton variable too early, before we have a chance to add command line and environment scopes.
Add a configuration option to suppress gpg warnings during binary
package verification. This only suppresses warnings: a gpg failure
will still fail the install. This allows users who have already
explicitly trusted the gpg key they are using to avoid seeing
repeated warnings that it is self-signed.
Binaries with relative RPATHS currently do not relocate strings
hard-coded in binaries
This PR extends the best-effort relocation of strings hard-coded
in binaries to those whose RPATHs have been relativized.
If you do this in a spack environment:
spack add hdf5+hl
hdf5+hl will be the root added to the `spack.yaml` file, and you should
really expect `hdf5+hl` to display as a root in the environment.
- [x] Add decoration to roots so that you can see the details about what
is required to build.
- [x] Add a test.
This fixes a regression introduced in #10792. `spack uninstall` in an
environment would not match concrete query specs properly after the index
hash of enviroments changed.
- [x] Search by DAG hash for specs to remove instead of by build hash
* Make relative binaries relocate text files properly
* rb strings aren't valid in python 2
* move perl to new interface for setup_environment family methods
- [x] insert at beginning of list so fetch grabs local mirrors before remote resources
- [x] update the S3FetchStrategy so that it throws a SpackError if the fetch fails.
Before, it was throwing URLError, which was not being caught in stage.py.
- [x] move error handling out of S3FetchStrategy and into web_util.read_from_url()
- [x] pass string instead of URLError to SpackWebError
This changes Spack environments so that the YAML file associated with the environment is *only* written when necessary (i.e., if it is changed *by spack*). The lockfile is still written out as before.
There is a larger question here of which part of Spack should be responsible for setting defaults in config files, and how we can get rid of empty lists and data structures currently cluttering files like `compilers.yaml`. But that probably requires a rework of the default-setting validator in `spack.config`, as well as the code that uses `spack.config`. This will at least help for `spack.yaml`.
* Travis CI: Test Python 3.8
* Fix use of deprecated cgi.escape method
* Fix version comparison
* Fix flake8 F811 change in Python 3.8
* Make flake8 happy
* Use Python 3.8 for all test categories
Currently, query arguments in the Spack core are documented on the
Database._query method, where the functionality is defined.
For users of the spack python command, this makes the python builtin
method help less than ideally useful, as help(spack.store.db.query)
and help(spack.store.db.query_local) do not show relevant information.
This PR updates the doc attributes for the Database.query and
Database.query_local arguments to mirror everything after the first
line of the Database._query docstring.
* cuda: fix conflict statements for x86-64 targets
fixes#13462
This build system mixin was not updated after the support for specific
targets has been merged.
* Updated the version range of cuda that conflicts with gcc@8:
* Updated the version range of cuda that conflicts with gcc@8: for ppc64le
* Relaxed conflicts for version > 10.1
* Updated versions in conflicts
Co-Authored-By: Axel Huebl <axel.huebl@plasma.ninja>
* docs: add a spack environment for building the docs
* docs: remove tutorial and link to spack-tutorial.readthedocs.io
The tutorial now has its own standalone website, versioned by instances
of the tutorial. Link to that instead of versioning it directly with Spack.
Support mirroring all packages with `spack mirror create --all`.
In this mode there is no concretization:
* Spack pulls every version of every package into the created mirror.
* It also makes multiple attempts for each package/version combination
(if there is a temporary connection failure).
* Continues if all attempts fail. i.e., this makes its best effort to
fetch evrerything, even if all attempts to fetch one package fail.
This also changes mirroring logic to prefer storing sources by their hash
or by a unique name derived from the source. For example:
* Archives with checksums are named by the sha256 sum, i.e.,
`archive/f6/f6cf3bd233f9ea6147b21c7c02cac24e5363570ce4fd6be11dab9f499ed6a7d8.tar.gz`
vs the previous `<package-name>-package-version>.tar.gz`
* VCS repositories are stored by a path derived from their URL,
e.g. `git/google/leveldb.git/master.tar.gz`.
The new mirror layout allows different packages to refer to the same
resource or source without duplicating that download in the
mirror/cache. This change is not essential to mirroring everything but is
expected to save space when mirroring packages that all use the same
resource.
The new structure of the mirror is:
```
<base directory>/
_source-cache/ <-- the _source-cache directory is new
archive/ <-- archives/resources/patches stored by hash
00/ <-- 2-letter sha256 prefix
002748bdd0319d5ab82606cf92dc210fc1c05d0607a2e1d5538f60512b029056.tar.gz
01/
0154c25c45b5506b6d618ca8e18d0ef093dac47946ac0df464fb21e77b504118.tar.gz
0173a74a515211997a3117a47e7b9ea43594a04b865b69da5a71c0886fa829ea.tar.gz
...
git/
OpenFAST/
openfast.git/
master.tar.gz <-- repo by branch name
PHASTA/
phasta.git/
11f431f2d1a53a529dab4b0f079ab8aab7ca1109.tar.gz <-- repo by commit
...
svn/ <-- each fetch strategy has its own subdirectory
...
openmpi/ <-- the remaining package directories have the old format
openmpi-1.10.1.tar.gz <-- human-readable name is symlink to _source-cache
```
In addition to the archive names as described above, `mirror create` now
also creates symlinks with the old format to help users understand which
package each mirrored archive is associated with, and to allow mirrors to
work with old spack versions. The symlinks are relative so the mirror
directory can still itself be archived.
Other improvements:
* `spack mirror create` will not re-download resources that have already
been placed in it.
* When creating a mirror, the resources downloaded to the mirror will not
be cached (things are not stored twice).
reindexing takes a significant amount of time, and there's no reason to
do it from DB version 0.9.3 to version 5. The only difference is that v5
can contain "deprecated_for" fields.
- [x] Add a `_skip_reindex` list at the start of `database.py`
- [x] Skip the reindex for upgrades in this list. The new version will
just be written to the file the first time we actually have to write
the DB out (e.g., after an install), and reads will still work fine.
Previously, spack would error out if we tried to fetch something with no
code, but that would prevent fetching dependencies. In particular, this
would fail:
spack fetch --dependencies xsdk
- [x] Instead of raising an error, just print a message that there is nothing
to be fetched for packages like xsdk that do not have code.
- [x] Make BundleFetchStrategy a bit more quiet about doing nothing.
We've had `spack spec --yaml` for a while, and we've had methods for JSON
for a while as well. We just haven't has a `--json` argument for `spack spec`.
- [x] Add a `--json` argument to `spack spec`, just like `--yaml`
New entry for K10 microarchitecture.
Reorder Zen* microarchitectures to avoid triggering as k10.
Remove some desktop-specific flags that were preventing Opteron Bulldozer/Piledriver/Steamroller/Excavator CPUs from being recognized as such.
Remove one or two flags which weren't produced in /proc/cpuinfo on older OS (RHEL6 and friends).
* Add master branch as version to HPX package
* Remove C++98 option from HPX package
* Add option to turn off examples in HPX package
* Add MPI variant to HPX package
Rename the `spack diy` command to `spack dev-build` to make the use case clearer.
The `spack diy` command has some useful functionality for developers using Spack to build their dependencies and configure/build/install the code they are developing. Developers do not notice it, partly because of the obscure name.
The `spack dev-build` command has a `-u/--until PHASE` option to stop after a given phase of the build. This can be used to configure your project, run cmake on your project, or similarly stop after any stage of the build the user wants. These options are analogous to the existing `spack configure` and `spack build` commands, but for developer builds.
To unify the syntax, we have deprecated the `spack configure` and `spack build` commands, and added a `-u/--until PHASE` option to the `spack install` command as well.
The functionality in `spack dev-build` (specifically `spack dev-build -u cmake`) may be able to supersede the `spack setup` command, but this PR does not deprecate that command as that will require slightly more thought.
fd58c98 formats the `Stage`'s `archive_path` in `Stage.archive` (as part of `web.push_to_url`). This is not needed and if the formatted differs from the original path (for example if the archive file name contains a URL query suffix), then the copy fails.
This removes the formatting that occurs in `web.push_to_url`.
We should figure out a way to handle bad cases like this *and* to have nicer filenames for downloaded files. One option that would work in this particular case would be to also pass `-J` / `--remote-header-name` to `curl`. We'll need to do follow-up work to determine if we can use `-J` everywhere.
See also: https://github.com/spack/spack/pull/11117#discussion_r338301058
Add a new entry in `config.yaml`:
config:
shared_linking: 'rpath'
If this variable is set to `rpath` (the default) Spack will set RPATH in ELF binaries. If set to `runpath` it will set RUNPATH.
Details:
* Spack cc wrapper explicitly adds `--disable-new-dtags` when linking
* cc wrapper also strips `--enable-new-dtags` from the compile line
when disabling (and vice versa)
* We specifically do *not* add any dtags flags on macOS, which uses
Mach-O binaries, not ELF, so there's no RUNPATH)
`spack deprecate` allows for the removal of insecure packages with minimal impact to their dependents. It allows one package to be symlinked into the prefix of another to provide seamless transition for rpath'd and hard-coded applications using the old version.
Example usage:
spack deprecate /hash-of-old-openssl /hash-of-new-openssl
The spack deprecate command is designed for use only in extroardinary circumstances. The spack deprecate command makes no promises about binary compatibility. It is up to the user to ensure the replacement is suitable for the deprecated package.
* mysql: Use correct python command
python~pythoncmd does not provide a python symlink for python3, so make
sure we pick the right command.
* mysql: Adapt to build env changes
* hypre: Add new variants to expost existing features.
* hypre: Add new variants to expose existing features.
* hypre: Shorten description line.
* hypre: Add an explicit else clause to disable some features.
Previously this command only showed total counts for each regular
expression. This doesn't give you a sense of which regexes are working
well and which ones are not. We now display the number of right, wrong,
and total URL parses per regex.
It's easier to see where we might improve the URL parsing with this
change.
It seems that 3.4.2 includes a change that is supposed to fix parallel
builds (https://lists.gnu.org/archive/html/bug-bison/2019-08/msg00000.html).
Instead, it actually breaks it for me (with -j48) with errors such as:
```
mv: cannot stat 'examples/c/reccalc/scan.stamp.tmp': No such file or directory
Makefile:9323: recipe for target 'examples/c/reccalc/scan.stamp' failed
```
* Ace package. Initial commit
* Correcting header now with licence.
* Using sha256
* Making requested changes:removing import line, urls and checksums in one line, and making it an MakefilePackage.
* Removing extra line
This updates the configuration loading/dumping logic (now called
load_config/dump_config) in spack_yaml to preserve comments (by using
ruamel.yaml's RoundTripLoader). This has two effects:
* environment spack.yaml files expect to retain comments, which
load_config now supports. By using load_config, users can now use the
':' override syntax that was previously unavailable for environment
configs (but was available for other config files).
* config files now retain user comments by default (although in cases
where Spack updates/overwrites config, the comments can still be
removed).
Details:
* Subclasses `RoundTripLoader`/`RoundTripDumper` to parse yaml into
ruamel's `CommentedMap` and analogous data structures
* Applies filename info directly to ruamel objects in cases where the
updated loader returns those
* Copies management of sections in `SingleFileScope` from #10651 to allow
overrides to occur
* Updates the loader/dumper to handle the processing of overrides by
specifically checking for the `:` character
* Possibly the most controversial aspect, but without that, the parsed
objects have to be reconstructed (i.e. as was done in
`mark_overrides`). It is possible that `mark_overrides` could remain
and a deep copy will not cause problems, but IMO that's generally
worth avoiding.
* This is also possibly controversial because Spack YAML strings can
include `:`. My reckoning is that this only occurs for version
specifications, so it is safe to check for `endswith(':') and not
('@' in string)`
* As a consequence, this PR ends up reserving spack yaml functions
load_config/dump_config exclusively for the purpose of storing spack
config
`test_envoronment_status()` was printing extra output during tests.
- [x] disable output only for `env('status')` calls instead of disabling
it for the whole test.
This PR ensures that environment activation sets all environment variables set by the equivalent `module load` operations, except that the spec prefixes are "rebased" to the view associated with the environment.
Currently, Spack blindly adds paths relative to the environment view root to the user environment on activation. Issue #12731 points out ways in which this behavior is insufficient.
This PR changes that behavior to use the `setup_run_environment` logic for each package to augment the prefix inspections (as in Spack's modulefile generation logic) to ensure that all necessary variables are set to make use of the packages in the environment.
See #12731 for details on the previous problems in behavior.
This PR also updates the `ViewDescriptor` object in `spack.environment` to have a `__contains__` method. This allows for checks like `if spec in self.default_view`. The `__contains__` operator for `ViewDescriptor` objects checks whether the spec satisfies the filters of the View descriptor, not whether the spec is already linked into the underlying `FilesystemView` object.
This PR ensures that on Darwin we always append /sbin and /usr/sbin to PATH, if they are not already present, when looking for sysctl.
* Make sure we look into /sbin and /usr/sbin for sysctl
* Refactor sysctl for better readability
* Remove marker to make test pass
These changes update our gcc microarchitecture descriptions based on manuals found here https://gcc.gnu.org/onlinedocs/ and assuming that new architectures are not added during patch releases.
Boost iostream autodetects the compression libraries libzstd and
liblzma outside of the Spack environment.
This commit disables mentioned libraries. In the future if the
Spack zstd/lzma packages were added as dependencies of the Spack
Boost package, additional work could be done to build Boost with
the Spack-built versions of these libraries.
This extends Spack functionality so that it can fetch sources and binaries from-, push sources and binaries to-, and index the contents of- mirrors hosted on an S3 bucket.
High level to-do list:
- [x] Extend mirrors configuration to add support for `file://`, and `s3://` URLs.
- [x] Ensure all fetching, pushing, and indexing operations work for `file://` URLs.
- [x] Implement S3 source fetching
- [x] Implement S3 binary mirror indexing
- [x] Implement S3 binary package fetching
- [x] Implement S3 source pushing
- [x] Implement S3 binary package pushing
Important details:
* refactor URL handling to handle S3 URLs and mirror URLs more gracefully.
- updated parse() to accept already-parsed URL objects. an equivalent object
is returned with any extra s3-related attributes intact. Objects created with
urllib can also be passed, and the additional s3 handling logic will still be applied.
* update mirror schema/parsing (mirror can have separate fetch/push URLs)
* implement s3_fetch_strategy/several utility changes
* provide more feature-complete S3 fetching
* update buildcache create command to support S3
* Move the core logic for reading data from S3 out of the s3 fetch strategy and into
the s3 URL handler. The s3 fetch strategy now calls into `read_from_url()` Since
read_from_url can now handle S3 URLs, the S3 fetch strategy is redundant. It's
not clear whether the ideal design is to have S3 fetching functionality in a fetch
strategy, directly implemented in read_from_url, or both.
* expanded what can be passed to `spack buildcache` via the -d flag: In addition
to a directory on the local filesystem, the name of a configured mirror can be
passed, or a push URL can be passed directly.
For some reason, newer versions of qt fail to build because they cannot
find certain system libraries such as libatomic and libdl.
Leaving the qmake compiler set to gcc seems to fix the problem.
Fixes#13221
fixes#13073
Since #3206 was merged bootstrapping environment-modules was using the architecture of the current host or the best match supported by the default compiler. The former case is an issue since shell integration was looking for a spec targeted at the host microarchitecture.
1. Bootstrap an env modules targeted at generic architectures
2. Look for generic targets in shell integration scripts
3. Add a new entry in Travis to test shell integration
Custom string versions for compilers were raising a ValueError on
conversion to int. This commit fixes the behavior by trying to detect
the underlying compiler version when in presence of a custom string
version.
* Refactor code that deals with custom versions for better readability
* Partition version components with a regex
* Fix semantic of custom compiler versions with a suffix
* clang@x.y-apple has been special-cased
* Add unit tests
We've been doing this for quite a while now, and it does not seem to
cause issues.
- [x] Switch the noisy warning to a debug to make Spack a bit quieter
while building.
* Added architecture specific optimization flags for Clang / LLVM
* Disallow compiler optimizations for mixed toolchains
* We emit a warning when building for a mixed toolchain
* Fixed issues with suffixed versions of compilers; Apple's Clang will,
for the time being, fall back on x86-64 for every compilation.
* DataSpaces: 1.8.0
Add a newer release of DataSpaces for ADIOS2.
Also add missing `-fPIC` flags since this is a static library.
* ADIOS2: Dataspaces, libffi, libfabric
Add missing dependencies to ADIOS2. Without explicit control,
those dependencies might get picked up from the system environment
and will cause unstable builds.
* Hide transitive SST dependencies
* ADIOS2: Simplify SST CMake Flags
As proposed by Chuck
* Methods setting the environment now do it separately for build and run
Before this commit the `*_environment` methods were setting
modifications to both the build-time and run-time environment
simultaneously. This might cause issues as the two environments
inherently rely on different preconditions:
1. The build-time environment is set before building a package, thus
the package prefix doesn't exist and can't be inspected
2. The run-time environment instead is set assuming the target package
has been already installed
Here we split each of these functions into two: one setting the
build-time environment, one the run-time.
We also adopt a fallback strategy that inspects for old methods and
executes them as before, but prints a deprecation warning to tty. This
permits to port packages to use the new methods in a distributed way,
rather than having to modify all the packages at once.
* Added a test that fails if any package uses the old API
Marked the test xfail for now as we have a lot of packages in that
state.
* Added a test to check that a package modified by a PR is up to date
This test can be used any time we deprecate a method call to ensure
that during the first modification of the package we update also
the deprecated calls.
* Updated documentation
Python 3 metaclasses have a `__prepare__` method that lets us save the
class's dictionary before it is constructed. In Python 2 we had to walk
up the stack using our `caller_locals()` method to get at this. Using
`__prepare__` is much faster as it doesn't require us to use `inspect`.
This makes multimethods use the faster `__prepare__` method in Python3,
while still using `caller_locals()` in Python 2. We try to reduce the
use of caller locals using caching to speed up Python 2 a little bit.
Our importer was always parsing from source (which is considerably
slower) because the source size recorded in the .pyc file differed from
the size of the input file.
Override path_stats in the prepending importer to fool it into thinking
that the source size is the size *with* the prepended code.
Since the backup file is only created on the first invocation, it will
contain the original file without any modifications. Further invocations
will then read the backup file, effectively reverting prior invocations.
This can be reproduced easily by trying to install likwid, which will
try to install into /usr/local. Work around this by creating a temporary
file to read from.
* This updates stage names to use "spack-stage-" as a prefix.
This avoids removing non-Spack directories in "spack clean" as
c141e99 did (in this case so long as they don't contain the
prefix "spack-stage-"), and also addresses a follow-up issue
where Spack stage directories were not removed.
* Spack now does more-stringent checking of expected permissions for
staging directories. For a given stage root that includes a user
component, all directories before the user component that are
created by Spack are expected to match the permissions of their
parent; the user component and all deeper directories are expected
to be accessible to the user (read/write/execute).
This feature generates a verification manifest for each installed
package and provides a command, "spack verify", which can be used to
compare the current file checksums/permissions with those calculated
at installed time.
Verification includes
* Checksums of files
* File permissions
* Modification time
* File size
Packages installed before this PR will be skipped during verification.
To verify such a package you must reinstall it.
The spack verify command has three modes.
* With the -a,--all option it will check every installed package.
* With the -f,--files option, it will check some specific files,
determine which package they belong to, and confirm that they have
not been changed.
* With the -s,--specs option or by default, it will check some
specific packages that no files havae changed.
fixes#13005
This commit fixes an issue with the name of the root directory for
module file hierarchies. Since #3206 the root folder was named after
the microarchitecture used for the spec, which is too specific and
not backward compatible for lmod hierarchies. Here we compute the
root folder name using the target family instead of the target name
itself and we add target information in the 'whatis' portion of the
module file.
From Python docs:
--
'surrogateescape' will represent any incorrect bytes as code points in
the Unicode Private Use Area ranging from U+DC80 to U+DCFF. These
private code points will then be turned back into the same bytes when
the surrogateescape error handler is used when writing data. This is
useful for processing files in an unknown encoding.
--
This will allow us to process files with unknown encodings.
To accommodate the case of self-extracting bash scripts, filter_file
can now stop filtering text input if a certain marker is found. The
marker must be passed at call time via the "stop_at" function argument.
At that point the file will be reopened in binary mode and copied
verbatim.
* use "surrogateescape" error handling to ignore unknown chars
* permit to stop filtering if a marker is found
* add unit tests for non-ASCII and mixed text/binary files
Provide up to date specs for build options
Remove non-production versions
Format using autopep8 and flake8
Separate cmake options in adios2 2.4.0 version
Keep default as True as much as possible
Support for Python 2 and 3 for adios2 versions <= 2.4.0
Starting v2.5.0 only Python 3 is supported
Addressing review suggestions
* Add new ACTS releases
* Update ACTS dependency requirements
* We can actually be more specific here
* The material plugin is gone as of ACTS v0.10.0
* Add 0.10.4 release
* Remove material mapping plugin
It was merged into core as of 0.10.x, and given ACTS' youth it is unlikely to have actual users at this point in time.
* Minor clean-up of package information.
* Add variants, fix dependencies, and add conflicts for compilers that do not support C++17.
* Correct dependency type for CMake.
- Add a test that verifies checksums on all packages
- Also add an attribute to packages that indicates whether they need a
manual download or not, and add an exception in the tests for these
packages until we can verify them.
Some packages had incorrect checksums because they needed to be
downloaded from a new place or the package needed tweaking somehow.
These packages to sha256 and the new checksums have been verified by
fetching over https.
These packages have either been retired or are no longer retrievable from
any site we can find. They also aren't needed by other packages, so we're
removing them from Spack.
- allinea-reports: is now arm performance reports
- globus-toolkit: retired, globus is now a paid service.
- uberftp: used globus-toolkit, github site is retired.
- mpiblast: site is dead; no longer fetchable.
- py-mxnet: site and github are gone
- soap2, soapindel, soapsnp: site is gone, unfetchable
- arlecore: could not verify md5 securely
We'd like to use a consistent checksum scheme everywhere so that we can:
a) incorporate archive checksums into our specs and have a
consistent hashing algorithm across all specs.
b) index mirrors with a consistent type of checksum, and not one that
is dependent on how spack packages are written.
- [x] convert existing md5, sha224, sha512, sha1 checksums to sha256
Both floating-point and NEON are required in all standard ARMv8
implementations. Theoretically though specialized markets can support
no NEON or floating-point at all. Source:
https://developer.arm.com/docs/den0024/latest/aarch64-floating-point-and-neon
On the other hand the base procedure call standard for Aarch64
"assumes the availability of the vector registers for passing
floating-point and SIMD arguments". Further "the Arm 64-bit
architecture defines two mandatory register banks: a general-purpose
register bank which can be used for scalar integer processing and
pointer arithmetic; and a SIMD and Floating-Point register bank".
Source:
https://developer.arm.com/docs/ihi0055/latest/procedure-call-standard-for-the-arm-64-bit-architecture
This makes customization of Aarch64 with no NEON instruction set
available so unlikely that we can consider them a feature of the
generic family.
* py-line-profiler: add 2.1.2, re-run cython if needed
See rkern/line_profiler#166.
* Update var/spack/repos/builtin/packages/py-line-profiler/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Addition of 1.1.3 and 1.1.4
* Addition of conditional enum-34 installation.
* Update var/spack/repos/builtin/packages/py-maestrowf/package.py
Addition of versioning for pyyaml.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Addition of enum34 back to older versions 1.1.3 and below.
* adjust building of OpenFOAM 1806 and earlier
- spack now unpacks into spack-src, but earlier OpenFOAM versions
still expect particular directory names (eg, OpenFOAM-v1806).
Resolve by creating a symlink during patching.
* replace openfoam 1806 patch with URL version
This PR adds a 'concretize' entry to an environment's spec.yaml file
which controls how user specs are concretized. By default it is
set to 'separately' which means that each spec added by the user is
concretized separately (the behavior of environments before this PR).
If set to 'together', the environment will concretize all of the
added user specs together; this means that all specs and their
dependencies will be consistent with each other (for example, a
user could develop code linked against the set of libraries in the
environment without conflicts).
If the environment was previously concretized, this will re-concretize
all specs, in which case previously-installed specs may no longer be
used by the environment (in this sense, adding a new spec to an
environment with 'concretize: together' can be significantly more
expensive).
The 'concretize: together' setting is not compatible with Spec
matrices; this PR adds a check to look for multiple instances of the
same package added to the environment and fails early when
'concretize: together' is set (to avoid confusing messages about
conflicts later on).
While the build environment already takes share/pkgconfig into account,
the generated module files etc. only consider lib/pkgconfig and
lib64/pkgconfig.
* fujitsu-mpi: commit new package
* Add homepage.
* * Modified the MPI commands to a simple form.
* Unnecessary processing has been deleted.
* Add conflict for all compilers except the Fujitsu compiler.
* Fix typo.
* Delete versions.
* Delete conditions for specifying the Fujitsu compiler.
When removing support for dotkit in #11986 the code trying to set the
paths of the various module files was not updated to skip it. This
results in a failure because of a key error after the deprecation
warning is displayed to user.
This commit fixes the issue and adds a unit test for regression.
Note that code for Spack chains has been updated accordingly but
no unit test has been added for that case.
* fftw: grouped precisions in a single variant
* fftw: simd options are now based on target features and not on variants
* fftw: simplified computing the selected precisions
* fix ipykernel dependency, newer versions are python3-only
* Update var/spack/repos/builtin/packages/py-ipykernel/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-ipykernel/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
Dotkit is being used only at a few sites and has been deprecated on new
machines. This commit removes all the code that provide support for the
generation of dotkit module files.
A new validator named "deprecatedProperties" has been added to the
jsonschema validators. It permits to prompt a warning message or exit
with an error if a property that has been marked as deprecated is
encountered.
* Removed references to dotkit in the docs
* Removed references to dotkit in setup-env-test.sh
* Added a unit test for the 'deprecatedProperties' schema validator
* libint: switch to 2-stage build for 2.6.0+
* libint: add tune and fortran variants
* cp2k: add support for libint >= 2.0
* cp2k: use pkg-config for linking libxc
* github releases are not released versions, but from-source
releases
* they lack the mimification of the JS/CSS components and as
such require node-js and npm, this conflicts with python@3:
* this commit
* drops the from-source dependencies on node-js and npm and
* updates the checksums to the correct sha256 hashes of the
pypi releases. The pypi-tarballs are different
* ZFP: CMake Package
ZFP is primarily build via CMake, which will have all install logic
and configuration files generated.
The 0.5.0 release did not support this yet, so we toss it.
* Patch ADIOS1: ZFP 0.5.1+ support
* ZFP: Expose all CMake variants
fixes#12915closes#12916
Since Spack has support for specific targets it might happen that
software is built for targets that are not exactly the host because
it was either an explicit user request or the compiler being used is
too old to support the host.
Modules for different targets are written into different directories
and by default Spack was adding to MODULEPATH only the directory
corresponding to the current host. This PR modifies this behavior to
add all the directories that are **compatible** with the current host.
* new package: py-python-ldap
* fix dependency types
* improvements
* Update var/spack/repos/builtin/packages/py-python-ldap/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-python-ldap/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add openldap dependency
* go-md2man: disable module-mode
* Update var/spack/repos/builtin/packages/go-md2man/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add singularity 3.4.1
Signed-off-by: ArangoGutierrez <eduardo@sylabs.io>
* revert some changes
Signed-off-by: ArangoGutierrez <eduardo@sylabs.io>
* add maintainers list to pkg singularity
Add @alalazo and me to the Singularity maintainers list
* Update var/spack/repos/builtin/packages/singularity/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/singularity/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/singularity/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
7bcb306 updated setup_environment in the GCC package to examine the
bin/ directory to find alternatively-named instances of CC, but this
failed to account for when GCC was being installed and failed when
it attempted to list the not-yet-existing installation prefix. This
commit updates the logic to skip this search if the bin/ directory
doesn't exist.
* libaio: fixed a condition that checks whether we are running on OS X
* libaio: simplified package by adding an explicit conflict for darwin
* libaio: added information to the docstring
* fio: added version 3.16, added conflict when using libaio, docstring
* lvm2: conflict due to libaio dep, improved docstring, simpler install
* mariadb: conflict due to libaio dep, improved docstring, version 10.4.7
* mariadb: the dependency on libaio is now conditional, removed conflict
* mariadb: fixed a typo in previous commit
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* libaio: turned into a MakefilePackage
Sometimes when remove_file is called on a link, that link is missing
(perhaps ctrl-C happened halfway through a previous action). As
removing a non-existent file is no problem, this patch changes the
behavior so Spack continues rather than stopping with an error.
Currently you would see
ValueError: /path/to/dir is not a link tree!
and now it continues with a warning.
Unlike the compiler binary name search logic, the `setup_environment` in
GCC's package assumes the compiler names are *exactly* `gcc`, `g++`,
etc. In many external installations (Homebrew, Macports) the
installation includes only *suffixed* versions such as `gcc-9`.
This patch uses the GCC compiler search suffixes to actually locate the
correct filenames for the installed compilers, allowing the
Spack-generated module file to have useful definitions of CC, CXX, etc.
It also allows for the possibility that the user's external installation
of GCC is compiled without Fortran support, in which case the `FC`
environment variables are not defined.
bin/spack now needs to have a "-*- python -*-" line after the shebang, so
that emacs will interpret it as a python file instead of as a shell
script. Add one line to the license check limit to accommodate this.
The Python landscape is going to be changing in 2020, and Python 2 will
be end of life. Spack should *prefer* Python 3 to Python 2 by default,
but we still need to run on systems that only have Python2 available.
This is trickier than it sounds, as on some systems, the `python` command
is `python2`; on others it's `python3`, and RHEL8 doesn't even have the
`python` command. Instead, it makes you choose `python3` or
`python2`. You can thus no longer make a simple shebang to handle all the
cases.
This commit makes the `spack` script bilingual. It is still valid
Python, but its shebang is `#!/bin/sh`, and it has a tiny bit of shell
code at the beginning to pick the right python and execute itself with
what it finds.
This has a lot of advantages. I think this will help ensure that Spack
works well in Python3 -- there are cases where we've missed things
because Python2 is still the default `python` on most systems. Also,
with this change, you do not lose the ability to execute the `spack`
script directly with a python interpreter. This is useful for forcing
your own version of python, running coverage tools, and running profiling
tools. i.e., these will not break with this change:
```console
$ python2 $(which spack) <args>
$ coverage run $(which spack) <args>
$ pyinstrument $(which spack) <args>
```
These would not work if we split `spack` into a python file and a shell
script (see #11783). So, this gives us the best of both worlds. We get
to control our interpreter *and* remain a mostly pure python executable.
* The perl-uri-escape package duplicates perl-uri
The Perl URI::Escape module is in the Perl URI package so the Spack
perl-uri-escape package is a duplicate of the perl-uri package. This PR
does the following:
- replace `depends_on('perl-uri-escape')` with `depends_on('perl-uri')`
in packages that dependend on perl-uri-escape
- add the version from perl-uri-escape to perl-uri
- remove the unneeded dependency from perl-uri
- remove the perl-uri-escape package
* Reinstated perl-test-needs dependency
Put the perl-test-needs dependency for perl-uri back in but set it to
type='test'.
* For tests to succeed the type must be ('build', 'test')
Now that Spack injects microarchitecture specific optimizations for
the selected target, packages should avoid adding flags that could
step over those.
This PR adds interproscan-4.8, which has a completely different build
system than version 5. Note that this builds for running on a host as
opposed to a queue system. I am fairly certain that a queue system can
be configured later.
This PR also adds a patch for intrproscan-5 to build when the user
building the packages has a large GID by setting posix mode for tar.
* llvm: fixed issues with syntax and formatting
* use spec.architecture.target.family
* use spec.target instead of spec.architecture.target
* mesa: use == instead of __contains__
A function added to this package ran as part of importing it and
printed a warning for most Spack invocations, this removes the
warning message.
Also:
* Use compilers directly instead of Spack compiler wrappers (since
references to the compiler are embedded in text and binary files
of the installation)
* Add 'cross' variant for setting CROSS environment variable for
build
* Set UPCXX_INSTALL for generated module files
* Set UPCXX_GASNET_CONDUIT/UPCXX_NETWORK in generated module files
on Cray systems
* Also set UPCXX_NETWORK in build environment for Cray systems
The output of subprocess.check_output is a byte string in Python 3. This causes dictionary lookup to fail later on.
A try-except around this function prevented this error from being noticed. Removed this so that more errors can propagate out.
The following dependencies were added:
+ depends_on('perl-exporter-tiny', type=('build', 'run'))
+ depends_on('perl-list-moreutils-xs', type=('build', 'run'))
Preferred targets were failing because we were looking them up by
Microarchitecture object, not by string.
- [x] Add a call to `str()` to fix target lookup.
- [x] Add a test to exercise this part of concretization.
- [x] Add documentation for setting `target` in `packages.yaml`
* microarchitectures: zen starts from x86_64, not from excavator
* Unit tests: fixed a test that is wrong with the new modeling
* microarchitectures: fixed features and inheritance for 15h family
bulldozer doesn't inherit from barcelona (10h) + added xop, lwp and tbm
instruction sets to the 15h family (it distinguish the family from 17h)
Addresses #12804
This PR adds the creation of the remaining (16) templates to ensure we can create them with expected content. The goal is to facilitate catching during testing.
Spack doesn't need `requests`, and neither does `jsonschema`, but
`jsonschema` tries to import it, and it'll succeed if `requests` is on
your machine (which is likely, given how popular it is). This commit
removes the import to improve Spack's startup time a bit.
On a mac with SSD, the import of requests is ~28% of Spack's startup time
when run as `spack --print-shell-vars sh,modules` (.069 / .25 seconds),
which is what `setup-env.sh` runs.
On a Linux cluster where Python is mounted from NFS, this reduces
`setup-env.sh` source time from ~1s to .75s.
Note: This issue will be eliminated if we upgrade to a newer `jsonschema`
(we'd need to drop Python 2.6 for that). See
https://github.com/Julian/jsonschema/pull/388.
- This is needed to support Cray machines -- we need an architecture
mic_knl > x86_64
- We used Cray's naming scheme for this target to make it work seamlessly
with the module-based detection sccheme on Cray. mic_knl is pretty
much dead, so this will be the last succh target. We will need to work
wtih Cray and other vendors in the future.
Seamless translation from 'target=<generic>' to either
- target.family == <generic> (in methods)
- 'target=<generic>:' (in directives)
Also updated docs to show ranges in directives.
Spack can now:
- label ppc64, ppc64le, x86_64, etc. builds with specific
microarchitecture-specific names, like 'haswell', 'skylake' or
'icelake'.
- detect the host architecture of a machine from /proc/cpuinfo or similar
tools.
- Understand which microarchitectures are compatible with which (for
binary reuse)
- Understand which compiler flags are needed (for GCC, so far) to build
binaries for particular microarchitectures.
All of this is managed through a JSON file (microarchitectures.json) that
contains detailed auto-detection, compiler flag, and compatibility
information for specific microarchitecture targets. The `llnl.util.cpu`
module implements a library that allows detection and comparison of
microarchitectures based on the data in this file.
The `target` part of Spack specs is now essentially a Microarchitecture
object, and Specs' targets can be compared for compatibility as well.
This allows us to label optimized binary packages at a granularity that
enables them to be reused on compatible machines. Previously, we only
knew that a package was built for x86_64, NOT which x86_64 machines it
was usable on.
Currently this feature supports Intel, Power, and AMD chips. Support for
ARM is forthcoming.
Specifics:
- Add microarchitectures.json with descriptions of architectures
- Relaxed semantic of compiler's "target" attribute. Before this change
the semantic to check if a compiler could be viable for a given target
was exact match. This made sense as the finest granularity of targets
was architecture families. As now we can target micro-architectures,
this commit changes the semantic by interpreting as the architecture
family what is stored in the compiler's "target" attribute. A compiler
is then a viable choice if the target being concretized belongs to the
same family. Similarly when a new compiler is detected the architecture
family is stored in the "target" attribute.
- Make Spack's `cc` compiler wrapper inject target-specific flags on the
command line
- Architecture concretization updated to use the same algorithm as
compiler concretization
- Micro-architecture features, vendor, generation etc. are included in
the package hash. Generic architectures, such as x86_64 or ppc64, are
still dumped using the name only.
- If the compiler for a target is not supported exit with an intelligible
error message. If the compiler support is unknown don't try to use
optimization flags.
- Support and define feature aliases (e.g., sse3 -> ssse3) in
microarchitectures.json and on Microarchitecture objects. Feature
aliases are defined in targets.json and map a name (the "alias") to a
list of rules that must be met for the test to be successful. The rules
that are available can be extended later using a decorator.
- Implement subset semantics for comparing microarchitectures (treat
microarchitectures as a partial order, i.e. (a < b), (a == b) and (b <
a) can all be false.
- Implement logic to automatically demote the default target if the
compiler being used is too old to optimize for it. Updated docs to make
this behavior explicit. This avoids surprising the user if the default
compiler is older than the host architecture.
This commit adds unit tests to verify the semantics of target ranges and
target lists in constraints. The implementation to allow target ranges
and lists is minimal and doesn't add any new type. A more careful
refactor that takes into account the type system might be due later.
Co-authored-by: Gregory Becker <becker33.llnl.gov>
Add llnl.util.cpu_name, with initial support for detecting different
microarchitectures on Linux. This also adds preliminary changes for
compiler support and variants to control the optimizatoin levels by
target.
This does not yet include translations of targets to particular
compilers; that is left to another PR.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Move verbose messages to debug level
get_patchelf should return None for test platform as well because create_buildinfo invokes patchelf to get rpaths.
Update command-line (CLI) parsing to understand references to yaml
files that store Spack specs. Where a file reference is encountered,
the full Spec in the file will be read in. A file reference may
appear anywhere that a spec could appear before. For example, if you
write "spack spec -y openmpi > openmpi.yaml" you may then install the
spec using the yaml file by running "spack install ./openmpi.yaml";
you can also refer to dependencies in this way (e.g.
"spack install foo^./openmpi.yaml").
There are two requirements for file references:
* A file path entered on the CLI must include a "/" even if the file
exists in your current working directory. For example, if you
create an openmpi.yaml file as above and run
"spack install openmpi.yaml" from the same directory, it will
report an error.
* A file path entered on the CLI must end with ".yaml"
This commit adds error messages to clearly inform the user of both
violations.
* implicit_rpaths are now removed from compilers.yaml config and are always instantiated dynamically, this occurs one time in the build_environment module
* per-compiler list required libraries (e.g. libstdc++, libgfortran) and whitelist directories from rpaths including those libraries. Remove non-whitelisted implicit rpaths. Some libraries default for all compilers.
* reintroduce 'implicit_rpaths' as a config variable that can be used to disable Spack insertion of compiler RPATHs generated at build time.
Fixes#12829
This adds a variant to the util-linux package that controls whether it
builds its own libuuid. Variant defaults to True. It enables other
packages to choose to get libuuid from the libuuid package instead.
This also changes the cryptsetup package to build util-linux with
~libuuid (so it uses an explicitly-Spack-built instance of libuuid
instead).
Fixes#12732Fixes#12767c22a145 added automatic detection and RPATHing of compiler libraries
to Spack builds. However, in cases where the parsing/detection logic
fails this was terminating the build. This makes the compiler library
detection "best-effort" and reports an issue when the detection fails
rather than terminating the build.
- add missing perl dependency
- only filter Makefile if referred to dependency is in the spec
- rewrite the shebang line of scripts to use spack perl
- remove tabix dependency as tabix and bgzip are part of htslib
* cuda: New version 10.1.243
Adds CUDA 10.1.243. The installer command-line arguments have slightly
changed, hence in the install method the version needs to be inspected
in order to give the correct arguments for the new installer.
* cuda: Combine common command-line arguments
The CUDA 10.1+ installer has slightly different command-line arguments
than previous versions. Keep the common parts together and only split
where they are different.
* Make flake8 happy
* Fix CD: Packages Service First
Build the packages.spack.io service images first, so they are
guaranteed to be pushed even if further images fail to build.
Fix the query to the `spack` script executed in later builds.
* CD: Remove Spack Images
Now done on Dockerhub.
* update py-bokeh: new version & dependencies
* Revert "update py-bokeh: new version & dependencies"
This reverts commit 5e191a0b20.
* update py-bokeh: new version & dependencies
* trigger travis-ci
* adjust python dependency-type to br
* update py-bokeh: new version & dependencies
* Revert "update py-bokeh: new version & dependencies"
This reverts commit 5e191a0b20.
* new package: py-fsspec: add package.py
* trigger travis-ci
* adjust python dependency-type to br
This is similar to #10191. The Ubuntu package for clang 8.0.0 displays
a very unusual version string, and we need this new regex to detect it
as just 8.0.0
Unit test have been complemented by the output that was failing
detection.
* CGNS: Add setting of RPATH on Darwin
This should go in the CGNS CMakeLists.txt, but it isn't there yet and won't be in previous versions, so best to handle it here.
If building on darwin/MacOSX, need to set the RPATH so applications can find shared library correctly.
* Restrict MACOSX_RPATH setting
This has now been added to the CGNS distribution, so the setting of `CMAKE_MACOSX_RPATH` should be restricted to current and previous releases and not upcoming releases.
* new package: py-configojb
* add version 4.7.2
* improve dependencies
* fix url for versions > 5.0.0
* Update var/spack/repos/builtin/packages/py-configobj/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-configobj/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* use format to create url strings
* fix typo
* improve style
* Update var/spack/repos/builtin/packages/py-configobj/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* py-gpaw: Fixing missing numpy include path, adding newer version of py-gpaw and fixing a bug with libxc.c in older py-gpaw
* py-gpaw: Fix Python and ASE version dependencies
- GPAW 19.8.1 requires Python 3.x, add dependency on python@3.5:
- GPAW releases are associated with a corresponding ASE release, add
matching ASE version dependencies.
* py-gpaw: Enable the scalapack variant by default
GPAW installation instructions say scalapack is optional but "highly
recommended". Thus enable it by default.
* add new cutadapt release 2.5, and adjust dependencies conditional on version
* use https, not ssh, for git url
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* update py-xopen (add 0.8.2 checksum)
* refine version specific dependencies
add py-bz2file, which needs to be created
* create py-bz2file
* Revert "refine version specific dependencies"
This reverts commit 70649da988.
* Revert "update py-xopen (add 0.8.2 checksum)"
This reverts commit 7400f44caa.
* update py-xopen (add 0.8.2 checksum)
* refine version specific dependencies
add py-bz2file, which needs to be created
* py-bz2file needed when built with python 2.7
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* space after comma
- Fix trailing whitespace missed by the bug described in #12755.
- Fix other style issues that have crept in over time (this can happen
when flake8 adds new checks with new versions)
E501 (line too long) exemptions are probably our most common ones -- we
add them for directives, URLs, hashes, etc. in packages. But we
currently add them even when a line *doesn't* need them, which can mask
trailing whitespace errors.
This changes `spack flake8` so that it will only add E501 exemptions if
the line is *actually* too long.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
There are problems with libunwind for arm and trying to use it with mesa.
See for example:
https://github.com/grate-driver/mesa/issues/6
disabling use of libunwind for arm, allows the package to be built
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
- Use of symbol testing macro requires explicit include()
- CMake policies are not version-portable and the support has to queried
based on CMake version
* libpeas@1.22.0: new package
* libpeas: note more build dependencies
Build and link dependencies were obtained through a combination of
examining the `config.log` file from `configure` and looking at the
pkg-config files of `libpeas`.
* libpeas: add search path for GIR/typelib files
* Use spec.architecture.target insted of platform.
* remove default in simd variant value.
simd variant change to archtecture independent values.
* Add check simd values.
* rewrite check simd values.
* * flake8
* change aarch64 to arm (neon support only 32 bit arm.)
* py-ase: New version 3.18
Add ASE version 3.18. ASE 3.18 has dropped support for python 2.x,
hence add a dependency on python@3.5: for it.
* Add more ASE dependencies
Address review comments, add dependencies on py-scipy, py-matplotlib,
py-flask, and also add type=('build', 'run') dependency specifiers.
mock_archive can now take multiple extension / tar option pairs (default matches old behavior).
url_fetch.test_fetch tests more archive types.
compression.EXTS split into EXTS and NOTAR_EXTS to avoid unwanted, non-meaningful combinatoric extensions such as .tar.tbz2.
While debugging this package on our Cray machine, tmux was throwing an
error involving the ncurses headers. Tmux would look for headers in
system locations rather than where spack installed ncurses. Tested only
on a Cray.
- previously spec parsing didn't allow you to look up missing (but still
known) specs by hash
- This allows you to reference and potentially reinstall
force-uninstalled dependencies
- add testing for force uninstall and for reference by spec
- cmd/install tests now use mutable_database
* When cleaning the stage root, only remove directories that appear
to be used for staging Spack packages. Previously Spack was clearing
all directories in the stage root, which could remove content not
related to Spack if the user chose a staging root which contains
files/directories not managed by Spack.
* The documentation is updated with warnings about choosing a stage
directory that is only managed by Spack (although generally the
check added in this PR for "spack clean" should avoid removing
content that was not created by Spack)
* The default stage directory (in config.yaml) is now
$tempdir/$user/spack-stage and the logic is updated to omit the
$user portion of this path if $tempdir already contains a $user
directory.
* When creating stage root assign user read/write permissions to all
directories in the path under $user. Previously Spack was assigning
the permissions of the first existing parent directory
`spec.prefix` reads from Spack's database, and if you do this with
multiple consecutive read transactions, it can take a long time. Or, at
least, you can see the paths get written out one by one.
This uses an outer read transaction to ensure that actual disk locks are
acquired only once for the whole `spack find` operation, and that each
transaction inside `spec.prefix` is an in-memory operation. This speeds
up `spack find -p` a lot.
Refactor `spack.cmd.display_specs()` and `spack find` so that any options
can be used together with -d. This cleans up the display logic
considerably, as there are no longer multiple "modes".
This is another machine-readable version of `spack find`. Supplying the
`--json` argument causes specs to be written out as json records,
easily filered with tools like jq.
e.g.:
$ spack find --json python | jq -C ".[] | { name, version } "
[
{
"name": "python",
"version": "2.7.16"
},
{
"name": "bzip2",
"version": "1.0.8"
}
]
- spack find --format allows you to supply a format string and have specs
output in a more machine-readable way, without dedcoration
e.g.:
spack find --format "{name}-{version}-{hash}"
autoconf-2.69-icynozk7ti6h4ezzgonqe6jgw5f3ulx4
automake-1.16.1-o5v3tc77kesgonxjbmeqlwfmb5qzj7zy
bzip2-1.0.6-syohzw57v2jfag5du2x4bowziw3m5p67
...
or:
spack find --format "{hash}"
icynozk7ti6h4ezzgonqe6jgw5f3ulx4
o5v3tc77kesgonxjbmeqlwfmb5qzj7zy
syohzw57v2jfag5du2x4bowziw3m5p67
...
This is intended to make it much easier to script with `spack find`
* Install the newest version of ncbi-rmblastn
Version ncbi-rmblastn 2.2.28 coonfiguration fails for gcc compiler on ubuntu 16.04 due to some compatibility issues that requires manual intervention to keep the installation going. However you still stumble upon other issues while compiling it.
Using Intel compiler passes the configuration step but fails for the compilation process as well.
Here I'm using the latest version and patch, and the recommended configuration option as mention in http://www.repeatmasker.org/RMBlast.html
* uploading ncbi-blast patch
The mariadb-5.5 series can not be compiled with gcc-9.1.0 and above. Add
a conflicts statement to reflect that.
The error is due to -Werror=address-of-packed-member, a warning that is
new in gcc-9.
* Update R CRAN packages
This PR update R CRAN packages. It also ensures that spack built
packages that correspond to R recommended packages are specified in the
dependency list. This is needed for #12015.
* Add more updates to packages
* Updated dependencies of CRAN R packages
Also, use cloud.r-project.org rather than cran.r-project.org to use
automatic redirection to mirrors. This minimizes the load on the cran
server.
* Update to latest versions of CRAN R packages
* Fix flake8 issues
Some blank lines at the end of a coupple of package files crept in.
* Adjust r-animation dependencies
The r-magick package depends on image-magick, so r-animation should as
well. Also, add in texlive for LaTeX support.
* New package: r-magick
* Adjust line spaces in r-magick
* Add swftools dependenct to r-animation
* New package: swftools
This is needed bt r-animation
* Add 'when' clause to some package dependencies
- r-animation
- r-coin
- r-dt
- r-openssl
* New package: r-ellipsis
* New package: r-vctrs
* New package: r-zeallot
* New package: r-generics
* New package: r-cardata
* Add r-progress dependency to r-readxl
* New package: r-askpass
* New package: r-openxlsx
* New package: r-rio
* New package: r-sys
* New package: r-zip
* Add more updated packages
These are updates that have happened while working on this PR as well as
catching the ones from the cran.rstudio.com URL that were missed before.
* New package: r-gower
* New package: r-recipes
* New package: r-squarem
* New package: r-libcoin
* New package: r-rcmdcheck
* New package: r-remotes
* New package: r-sessioninfo
* New package: r-xopen
* New package: r-rgexf
* New package: r-rook
* New package: r-later
* New package: r-promises
* New package: r-xfun
* New package: r-urca
* New package: r-ggsignif
* New package: r-polynom
* New package: r-ucminf
* New package: r-ordinal
* New package: r-inum
* New package: r-debugme
* New package: r-fs
* New package: r-shinyfiles
* New package: r-gsa
* New package: r-nmof
* New package: r-dosnow
* New package: r-rann
* New package: r-reticulate
* New package: r-future
* New package: r-globals
* New package: r-listenv
* New package: r-future-apply
* New package: r-rsvd
* New package: r-sctransform
* New package: r-mitools
* Remove a stray FIXME line
* Fix typo in dependency for r-rcmdcheck
* Remove the dependency on r-splines
The r-splines package never existed in spack, but it does not exist on
CRAN either. There is an archive version from 1999 but it does not have
the proper extension format for modern R packages. The latest version
does not depend on r-splines, and apparently previous versions at least
built without it. Also, bump the required R version.
* Fix typo in r-rcpp dependency
* Fix typos in dependencies
- r-diagrammer
- r-janitor
- r-seurat
Also, bump minimum R version for r-rngtools.
* New package: r-clisymbols
* New package: r-fitdistrplus
* New package: r-gbrd
* New package: r-gh
* New package: r-ini
* New package: r-metap
* New package: r-npsurv
* New package: r-rdpack
* New package: r-snakecase
* New package: r-tinytex
* New package: r-usethis
* Make recommended changes
Made the changes that were recommended in the initial review.
- change cran.rstudio.com to cloud.r-project.org
- change R-project to r-project
- for packages with no listed homepage use the canonical link to the
package page
- for packages with a dependency constraint on R itself, specify the
dependency type
- add `list_url` where missing from package
* New package: r-hdf5r
This is needed for some older versions of r-seurat.
* Set R versions in packages
If the R version is specified in the DESCRIPTION file then set it in the
respective spack package.py file.
* Update var/spack/repos/builtin/packages/r-aer/package.py
Change dependency version of r-car.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Set constraint of mlbench dependency of r-adabag
* Set constraint on r-rcpp dependency of r-ape
* Removed "suggested" r-testthat from r-bibtex
* Update var/spack/repos/builtin/packages/r-bindrcpp/package.py
Fix dependency spec of r-bindr in r-bindrcpp.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/r-blob/package.py
Add r-vctrs version constraint.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Remove r-base64enc dependency from r-callr
* Update var/spack/repos/builtin/packages/r-car/package.py
Set constraint on r-cardata dependency.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Remove unneeded dependencies from r-clipr
* Add gmake dependency to r-construct
* Set the proper homepage for r-crayon
* Add r-jsonlite as a dependency for r-crul
The jsonlite package moved from "suggests" to "imports" with 0.8.4.
* Remove accidental colon after package name
* Add version constraint to r-fpc dependency in r-dendextend
* Additional constraints for R version with r-digest
* Add r-jsonlite dependency for latest verion of r-dt
* Add r-htmltools dependency in r-dygraphs package
* Removed unneeded dependencies from r-fnn
* Remove unneeded dependencies from r-formatr
* Add version constraint for r-mvtnorm dependency in r-fpc
* Added gmake dependency to r-fs
* Add r-globals dependency in r-future-apply
* Add perl dependency to r-gdata
* Add openssl dependency for r-ggally
* Removed unneeded dependencies from r-globaloptions
* Update r-haven
- Add version constraint on r-forcats
- Add gmake build dependency
* Fixed gmake dependency
Gmake is a build only dependency.
* Apply suggestions from code review
Apply a batch of suggestions from Adam Stewart.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Remove unneeded dependencies from r-hoardr
* Fix r-htmlwidgets homepage
* Add gmake build dependency to r-httpuv
* Remove unneeded dependency from r-log4r
* Remove unneeded dependency from r-maldiquant
* Add gmake build dependency to r-minqa
* Remove unneeded dependency from r-mlbench
* Set constraint on R6 dependenvy of r-pbdzmq
* Set constraints on r-callr dependency of r-pkgbuild
* Remove unneeded dependency from r-nloptr
* Remove unneeded dependencies from r-pkgmaker
* Set R version constraint for r-raster
* Set version constraint on rcpp for r-rcppprorgess
* Added gmake build dependency
* Remove unneeded dependencies from r-rematch
* Remove unneeded dependency from r-plogr
* Add gmake build dependency for r-rcurl
* Apply suggestions from code review
Bring in more suggestions from Adam Stewart.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Remove cnpy dependency from r-rcppcnpy
The r-rcppcnpy package has its own modified copy of the cnpy source and
will build from that internal source. The cnpy dependency is therefore
unnecessary.
* Add comment to reconcile dependencies
The CRAN site lists Perl, Getopt::Long as a dependency. The Spack Perl
will have the Getopt::Long package installed so just depend on Perl.
* Add dependency on udunits2 with newer versions of r-units
* Set additional proj constraint for r-sf
Support for PROJ6 was added in r-sf-0.7-4.
* New package: pandoc
This package installs the pandoc and pandoc-citeproc binaries in order
to satisfy dependencies on pandoc until the Haskell framework is in
Spack.
* Add pandoc dependency to r-bookdown
* Add comment about MPI in r-ergm
* Add pandoc dependency for r-reprex
* Add proj constraints to r-rgdal dependencies
Support for PROJ6 was added in version 1.4-2.
* Modifications to r-rgl
- Rearranged the dependencies to be more consistent with other packages
- put the type last on the depends_on lines to be consistent with other
packages
- Added version constraints for some dependencies
- Added pandoc build dependency
* Be more accurate with rjsonio constraint for r-rgooglemaps
* Remove unneeded dependencies from r-rhmmer
* Tweak constraints for r-rjava
- Set java version constraint
- add gmake build dependency
* Add pandoc dependency to r-markdown
* Remove unneeded dependencies from r-rrcov
* Tweak r-rstan package
- remove duplicate specifications
- remove unneeded when conditions
- add version constraints
- arrange to follow order on CRAN page more closely
- add gmake build dependency
- add pandoc build dependency
* Add pandoc build dependency for r-rstantools
* Remove unneeded dependency from r-runit
* Remove unneeded dependencies from r-selectr
* Remove example R code from description
The example would module loading to break, at least in the case of Lmod
using TCL module files. It could be during the TCL to Lua conversion but
the module fails due to interpreting ipart of the example as a command.
* Update r-seurat
New version of r-seurat with new dependencies.
* New package: r-dqrng
* New package: r-leiden
* New package: r-rcppannoy
* New package: r-rcppparallel
* New package: r-rspectra
* New package: r-sitmo
* New package: r-uwot
* Remove unneeded dependency from r-snow
* Remove unneeded dependency from r-sourcetools
* Added pandoc build dependency for r-stanheaders
* Remove unneeded dependencies from r-taxizedb
* Remove unneeded dependencies from r-tclust
* Add dependency constraints for r-threejs
The following are needed for version 0.2.2:
- r-matrix
- r-jsonlite
* Remove unneeded dependencies from r-vfs
* Remove unneeded dependencies from r-vgam
* Remove unneeded dependency from r-xlconnectjars
* Remove unneeded dependency from r-xlsxjars
* Remove unneeded dependency from r-xnomial
* Remove ~mpi from fftw dependency of swftools
* Apply suggestions from code review
Add latest round of suggestions from Adam Stewart.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Fix rjsonio spec for r-rgooglemaps
The r-rjsonio spec is a when conditional.
* Rename 3 R.* packages
- r-utils --> r-r-utils (R.utils)
- r-oo --> r-r-oo (R.oo)
- r-methodss3 --> r-r-methodss3 (R.methodsS3)
Also, renamed dependencies in packages that depend on these three.
* Loosen version constraint on mariadb for r-rmysql
Make the mariadb constraint on mariadb-5.5 rather than the more
specific, and limiting, mariadb-5.5.56. This will allow an external
mariadb, such as mariadb-5.5.60 on CentOS-7.5 to be used. It also allows
for r-rmysql to be built against newer versions of the mariadb-5.5
series that might be added to spack.
* Convert to Autotools package
* dyninst and papi are now optional (off by default)
* limit dyninst version to < 10 (see
https://github.com/bsc-performance-tools/extrae/issues/30)
* Add numactl dependency
* Enable parallel installs for 3.7 and later
* Set EXTRAE_HOME in module files and for dependent package installs
* Generalize cxx11 support flag (previously it was using a specific
syntax and this updates it to use the cxx11 support flag for the
compiler in use).
* The install for kaldi was copying library symlinks but not the
actual library files, this makes sure to copy the libraries
* All libraries are installed to 'prefix.lib' (the original
library directory structure is no longer maintained)
* The install step for executables did not account for the different
dynamic library suffix on MacOS
e891cc4 updated the cairo package to make gobject support optional
and disabled it by default. gtkplus requires gobject support so this
explicitly enables +gobject on the cairo dependency of gtkplus.
When Spack installs a package it writes the package.py file and
patches to a separate repository (which reflects the state of the
package at the time it was installed). Previously, Spack only wrote
patches that were used at installation time. This updates the
archiving step to include all patch files that are relevant to the
package (in case that repository is used in another context).
Change the depends_on type from default (build + link) to just 'link'
for binutils and a few other packages. We don't use the binutils
utilities (only libs). On cori at NERSC (Cray), this was breaking the
build by covering up the system ld.
* Only apply issue-87.patch for versions before 1.2.3
* 1.2.3 builds fine with gcc@8:, but 1.2.2 and 1.2.2_p1 both fail
(with or without the patch) so the conflict is updated to exclude
only these two versions.
This commit removes redundant calls to `libtoolize` and `aclocal`.
Some configurations, such as a Spack user using macOS with a
Homebrew-installed `libtool` added to their `packages.yaml`, have
`autoreconf` and GNU libtoolize installed as `glibtoolize`, but not
`libtoolize`. While Spack installations of `libtool` built from source
would install `glibtoolize` and symlink `libtoolize` to `glibtoolize`,
an external installation of GNU libtoolize as `glibtoolize` will not
have such a symlink, and thus the call `m.libtoolize()` will throw an
error because `libtoolize` does not exist at the path referenced by
`m.libtoolize()` (i.e.,
`self.spec['libtool'].prefix.bin.join('libtoolize')).
However, on these same systems, `autoreconf` runs correctly, and calls
`glibtoolize` instead of `libtoolize`, when appropriate. Thus,
removing the call to `libtoolize` should resolve the error mentioned
above.
The redundant call to `aclocal` is also removed in this commit because
the maintainers of GNU Automake state that "`aclocal` is expected to
disappear" and suggest that downstream users never call `aclocal`
directly -- rather, they suggest calling `autoreconf` instead.
* Make mpi, scipy, petsc4py, and nlopt support optional
* Later versions depend on python@3.5:
* Versions before 0.4.1 must build with MPI: add a conflict for
@:0.4.1~mpi
Before this commit, CMake would still attempt to detect OpenMP, even
if RAJA were being installed with `spack install raja~openmp`, because
the option `ENABLE_OPENMP` is set to "On" by default. This commit
explicitly disables OpenMP when the Spack install spec contains
'~openmp`, ensuring that CMake does not attempt to detect and link
with OpenMP.
Uses code from CMake to detect implicit link paths from compilers
System paths are filtered out of implicit link paths
Implicit link paths added to compiler config and object under `implicit_rpaths`
Implicit link paths added as rpaths to compile line through env/cc wrapper
Authored by: "Ben Boeckel <ben.boeckel@kitware.com>"
Co-authored by: "Peter Scheibel <scheibel1@llnl.gov>"
Co-authored by: "Gregory Becker <becker33@llnl.gov>"
* esmf: add patch when building with intel for 7.0.x
* esmf: set Makefile variables required to propagate CXX/F90 flags
* ncl: add ymake patch for @6.4.0:
* ncl: add pixman, bzip2, freetype dependencies
* Add version 6.3.2 (which does not require patches like 6.1.5)
* Add +python variant to build Python bindings
* Option to build shared libraries does not exist for newer
versions including 6.3.2
* Update 'libs' property (library names are different for newer
versions)
* Add support to detect and build with MKL as blas provider
SEACAS is now using versions instead of just relying on always
installing master. There are now tagged versions.
The most recent version is the first that can be used with a
non-patched develop version of the CGNS package; previously, seacas
required that the CGNS library have some internally-developed patch
applied, but this is no longer needed.
Also, added support for adios2 package with the adios2 variant.
This currently defaults to off.
c9e214f updated template creation by passing **kwargs to package
template classes but the template classes were not updated to accept
them; this adds **kwargs to package template initializers where they
are needed.
Having a non-directory invisible file causes `spack find` to die. This
fixes the logic to ignore invalid module names but only warn if they're
visible.
```
NotADirectoryError: [Errno 20] Not a directory: '/spack/var/spack/repos/builtin/packages/.DS_Store/package.py'
```
This variant (off by default) adds support for the kdreg option in
the GNI provider. It is disabled by default in order to avoid
potential runtime conflicts with the cray-mpich library. Only one
library at a time can use this feature.
This patch allows the libfabric package to work correctly out of
the box on systems like Cori at NERSC (since it's CLE7 update).
Otherwise the spack build may autodetect kdreg headers on the system
and enable this problematic feature.
* nlohmann-json: 3.7.0
The version 3.6.0 fix compile issues with GCC 9.0.
* nlohmann-json: proper test handling
tests are controlled via `spack install --test root|all ...`
instead of a variant.
This adds a special package type to Spack which is used to aggregate
a set of packages that a user might commonly install together; it
does not include any source code itself and does not require a
download URL like other Spack packages. It may include an 'install'
method to generate scripts, and Spack will run post-install hooks
(including module generation).
* Add new BundlePackage type
* Update the Xsdk package to be a BundlePackage and remove the
'install' method (previously it had a noop install method)
* "spack create --template" now takes "bundle" as an option
* Rename cmd_create_repo fixture to "mock_test_repo" and relocate it
to shared pytest fixtures
* Add unit tests for BundlePackage behavior
* new package: libglvnd
* remove virtual provides from libglvnd
libglvnd does not provide them directly, but
through implementations that depend on it.
* fix flake8 errors
* libglvnd: fix style/remove unecessary variants
* remove older libglvnd version
* explicitly enable all libglvnd stubs
* libglvnd: conflict with non-linux platform
This allows "spack spec --yaml" to generate a spec YAML file that can
be used with "spack install -f". Before, this would fail in cases
where the spec had build dependencies.
* All fetch strategies now accept the Boolean version keyword option `no_cache` in order to allow per-version control of cache-ability.
* New git-specific version keyword option `get_full_repo` (Boolean). When true, disables the default `--depth 1` and `--single-branch` optimizations that are applied if supported by the git version and (in the former case) transport protocol.
* The try / catch blog attempting `--depth 1` and retrying on failure has been removed in favor of more accurately ascertaining when the `--depth` option should work based on git version and protocol choice. Any failure is now treated as a real problem, and the clone is only attempted once.
* Test improvements:
* `mock_git_repository.checks[type_of_test].args['git']` is now specified as the URL (with leading `file://`) in order to avoid complaints when using `--depth`.
* New type_of_test `tag-branch`.
* mock_git_repository now provides `git_exe`.
* Improved the action of the `git_version` fixture, which was previously hard-wired.
* New tests of `--single-branch` and `--depth 1` behavior.
* Add documentation of new options to the packaging guide.
* Choose a specific commit as a version (before, the only version was
the master branch)
* Add CMake patch which allows for building the version added here
(this replaces the existing patches for percept in Spack)
* Update trilinos dependency
Also update dependencies to build latest py-deeptools versions and
add a patch to run tests:
* Replace pypi URL with github URL to download tests
* Update dependency versions of py-pysam, py-matplotlib
* Add new py-deeptoolsintervals package as a dependency
* Add py-plotly and py-nose dependencies
* Patch setup.py to collect and run tests
- mkdirp now takes arguments to allow it to properly set permissions on created directories.
- Two arguments (group and mode) set permissions for the leaf directory.
- Intermediate directories can inherit permissions from either the topmost existing directory (the parent) or the leaf.
On machines where $TMP is owned by a gid with no name, this avoids the
following error when the default spack stage does not exist:
(spackbook):spack$ spack clean
==> Removing all temporary build stages
==> Error: 'getgrgid(): gid not found: 57095'
Spack needs to deal with gids directly unless users pass them in.
UnifyCR has been renamed to UnifyFS. Renaming the Spack package is
the last step in this process.
This renames the package and changes any UnifyCRs to UnifyFS within
the package.
Compiler caching was using the `id()` function to refer to configuration dictionary objects. If these objects are garbage-collected, this can produce incorrect results (false positive cache hits). This change replaces `id()` with an object that keeps a reference to the config dictionary so that it is not garbage-collected.
Fixes#11163
The goal of this work is to simplify stage directory structures by eliminating use of symbolic links. This means, among other things, that` $spack/var/spack/stage` will no longer be the core staging directory. Instead, the first accessible `config:build_stage` path will be used.
Spack will no longer automatically append `spack-stage` (or the like) to configured build stage directories so the onus of distinguishing the directory from other work -- so the other work is not automatically removed with a `spack clean` operation -- falls on the user.
* Overhaul PyTorch package
* Flake8
* Distributed requires MPI, NCCL not specified correctly
* Binary was being specified incorrectly
* Run tests at install time
* Disable GLOO in addition to GLOO_IBVERBS
* Add option to disable QTDesigner plugin for qwt
Qt designer shouldn't be necessary to build downstream GUI code.
Currently `qwt^qt~tools` fails to build:
(darwin clang@10.0.1)
```
compiling qwt_designer_plugin.cpp
compiling qwt_designer_plotdialog.cpp
moc qwt_designer_plugin.h
moc qwt_designer_plotdialog.h
rcc qwt_designer_plugin.qrc
qwt_designer_plugin.h:23: Error: Undefined interface
make[1]: *** [moc/moc_qwt_designer_plugin.cpp] Error 1
make[1]: *** Waiting for unfinished jobs....
qwt_designer_plugin.cpp:17:10: fatal error: 'QDesignerFormEditorInterface' file not found
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make[1]: *** [obj/qwt_designer_plugin.o] Error 1
make: *** [sub-designer-make_default-ordered] Error 2
```
* Update VTK for newer VisIt and compiler versions
Several changes/enhancements to VTK package
---
Add version 8.1.0 (default version for VisIt 3.x)
---
Tweak VTK options for version 8 and VisIt:
- only add mpi4py version option when mpi4py is enabled
- Require object factory for downstream packages
- Remove OpenGL preference for VTK 8
---
Combine +python3 into +python
The spec for using python3 should just be `+python^python@3:`.
---
Remove unused VTK cmake argument on newer version
---
Patch for Python 3 fails on VTK 6.1 (needed for VisIt 2.x):
```
==> Patch /rnsdhpc/code/spack/var/spack/repos/builtin/packages/vtk/python3.7-const-char.patch failed.
==> Error: ProcessError: Command exited with status 1:
'/usr/bin/patch' '-s' '-p' '1' '-i' '/rnsdhpc/code/spack/var/spack/repos/builtin/packages/vtk/python3.7-const-char.patch' '-d' '.'
```
and as it happens, VisIt 2.13 is incompatible with Python 3 anyway.
---
Fix compiler error:
As spack's configuration warning says, double-quotes can cause trouble:
```
==> [2019-07-29-11:47:55.269653] Warning: Quotes in command arguments can confuse scripts like configure.
The following arguments may cause problems when executed:
-DVTK_REQUIRED_OBJCXX_FLAGS=""
Quotes aren't needed because spack doesn't use a shell.
Consider removing them
```
They were being added to the compile line, causing clang to think it was
asked to compile an empty file:
```
cd /private/var/folders/fy/x2xtwh1n7fn0_0q2kk29xkv9vvmbqb/T/s3j/spack-stage/spack-stage-3nk9bwe_/spack-build/Rendering/OpenGL && /rnsdhpc/code/spack/lib/spack/env/clang/clang++ -DVTK_IN_VTK -DvtkRenderingOpenGL_EXPORTS -F/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks -DGLX_GLXEXT_LEGACY -O2 -g -DNDEBUG -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -fPIC -fvisibility=hidden -fvisibility-inlines-hidden "" -o CMakeFiles/vtkRenderingOpenGL.dir/vtkCocoaGLView.mm.o -c /rnsdhpc/code/spack/var/spack/stage/vtk-6.1.0-b2mj276iz77fadisv2me365qe27t3gxt/spack-src/Rendering/OpenGL/vtkCocoaGLView.mm
clang: error: no input files
make[2]: *** [Rendering/OpenGL/CMakeFiles/vtkRenderingOpenGL.dir/vtkCocoaGLView.mm.o] Error 1
```
* Modernize/PEP-8 visit package statements
* Add new VisIt version and patches for building on Mac
- VisIt's cmake assumes a 'vtk(tiff|jpeg|png)' *library* target exported by
VTK, but VTK 8.1.x on my configuration seems to have these as header-only
targets with library dependencies.
- Explicitly add QT path to visit configure: Without this, somehow
`${VISIT_QT_DIR}` was being set to empty, so that the build failed with
`missing /bin/moc`.
- VisIt assumes that if it isn't being built statically, QWT has been built as
a mac Framework. QWT actually is built as a framework based on whether QT
itself is.
Error message:
```
-- Looking for QWT
CMake Error at CMake/SetUpThirdParty.cmake:131 (MESSAGE):
Include Directory for QWT (/rnsdhpc/code/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/qwt-6.1.3-bmfxirweplzhjoiil4rw632evkyteuwk/lib/qwt.framework/Versions/Current/Headers) does not exist.
Call Stack (most recent call first):
CMake/FindQwt.cmake:57 (SET_UP_THIRD_PARTY)
CMakeLists.txt:1251 (INCLUDE)
```
* ADIOS 2.4.0: Add new Deps
Version 2.4.0 of ADIOS2 added new dependencies that are not
yet listed as `depends_on` and/or disabled otherwise. Add them
now to avoid picking them up from system paths, e.g. `libpng`.
* ADIOS2: Variant Defaults Change
* Python: False by default
* ADIOS1: only used prior to 2.3.0
* UnifyCR: update dependencies and build options
This adds some specifics about dependencies and conflicts and adds
additional build options and variants. UnifyCR now also has limited
fortran support.
* Remove debug statements condition on Spack debug
Change debug print statements to now be turned on when the -g cflag
is found in the flags the user sets.
* add new package: py-python-magic
* tidy up
* Update var/spack/repos/builtin/packages/py-python-magic/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add master and dependencies
* remove gcc and make
* add variants, make doc and ch-grow dependencies optional
* capitalize description sentence
* move dependencies and variants below version block
* fix typos
* gromacs: fix simd detection for pre 2018 versions
* pre 2018 the cmake file was named differently
* fixed up post 2018 patch file which was non pretty
* Trim patch
Fixes#12062406c791 addressed "spack module load" for upstream modules but not
the "spack module loads" command. This applies the same fixes from
406c791 to "spack module loads".
* Create package.py
* applied reviewer suggestions
* Update package.py
* remove CamelCase from variant names
* add comment on why fetching package via git
* found typo in variant to cmake-argumentlist translation
* rename variants to ~nonuma ~nompi to be in sync with cmake-args, refine mpi@3-dependency definition
* Revert "rename variants to ~nonuma ~nompi to be in sync with cmake-args, refine mpi@3-dependency definition"
This reverts commit 58e8cf7d93.
* remove version-preference
* add checksum for singularity release 3.3.0
* update singularity doc URL
URL at the lbl.gov domain has message saying:
"This website is going away soon! Update your bookmarks!"
* add note about build log file for post-install step
* provide links to both 2.x and 3.x security docs
* remove whitespace. Passing flake8 confirmed locally this time :)
* add TODO comment for fakeroot support
The previous version does not properly install the required cmake
configuration files for blaze-lib, so find_package does not work
correctly.
This patch converts the blaze-lib package to use CMakePackage,
adds the required cmake dependency, and removes the install()
member function, which is not working correctly and is no longer
needed.
In a review of a previous pull request (
https://github.com/spack/spack/pull/10713 ) it was mentioned that the
proper way to figure out the target architecture is via
spec.architecture.target. This patch fixes this for the openblas
package.
Signed-off-by: Janne Blomqvist <janne.blomqvist@aalto.fi>
Not sure what happened between #5321 and now but the CC=CXX fix now
prevents py-matplotlib from building with the Intel compiler. I verified
with versions of py-matplotlib from 2.0.2 to 3.0.2 and intel-17.0.4 and
19.0.4 that py-matplotlib successfully builds with the Intel compiler
without that code.
* Add preCICE xSDK support
* Cleanup generation of cmake arguments
* Generalize numpy include
* Fix python spec
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Fix numpy spec
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* hunspell: write autoreconf method to triage error
This commit fixes the error described in GitHub issue
`AutotoolsPackage` class attempts to call `libtoolize`, which doesn't
exist on macOS systems that externally install `libtool` in Spack via
Homebrew -- Homebrew installzes GNU libtoolize as `glibtoolize`.
The fix is to call `autoreconf` only, and GitHub pull request #12344
makes this change in
AutotoolsPackage (https://github.com/spack/spack/pull/12344). However,
while waiting for that pull request to be merged, overriding the
`autoreconf` method achieves the same result for a self-contained
fix. This self-contained fix is necessary to fix a dependency bug in
the `hunspell` package.
* hunspell: add missing gettext dependency
* hunspell: add version 1.7.0
* add new package cromwell
* fix flake8 errors, add cromwell-womtool package
* change name of cromwell to cromwell-server
* renamed cromwell-server to just cromwell
* update description to what is in the project docs
* fixing format call for python2.6 compatibility
* Add comment that 1.0.1 versions are no longer supported
* Add comment that 1.0.2 versions are only supported up to
December 2019
* Add versions 1.0.2s, 1.0.2r, and 1.0.2p
* Add versions 1.1.0k, 1.1.0j, and 1.1.0i
* Add version 1.1.1c
* Add comment that 1.1.0 versions are only supported up to
September 2019
It's no longer possible to set compiler flags under as an entry under
"paths" in compilers.yaml; instead the user must list these under the
"flags" section. This updates the docs accordingly.
Spack stacks drop invalid dependencies applied to packages by a
spec_list matrix operation
Without this fix, Spack would raise an error if orthogonal dependency
constraints and non-dependency constraints were applied to the same
package by a matrix and the dependency constraint was invalid for
that package. This is an error, fixed by this PR.
An example failing configuration:
spack:
definitions:
- packages: [libelf, hdf5+mpi]
- compilers: ['%gcc']
- mpis: [^openmpi]
specs:
- matrix:
- $packages
- $compilers
- $mpis
5f74f22 enabled installing compilers for dependencies but not for the root package (and in particular not for DAGs which consist of one package)
this enables bootstrapping compilers for both types of DAGs
Resolves#11932.
Move the PIC flags from CFLAGS on the configure line to the spack
compiler wrapper for ncurses. The problem with the configure line for
autotools is that specifying CFLAGS sets the entire flags, thus
deleting the flags that configure would add itself.
By default, if CFLAGS is unspecified, then configure picks a sensible
default of `-g -O2`. But adding `-fPIC` erases these and it ends up
building unoptimized.
* Add patch to detect newer versions of MacOS and to fix a typo in
configure.ac (AM_LIBTOOLSFLAGS should be AM_LIBTOOLFLAGS)
* Remove variant declarations for unsupported languages
* Add support for quartz on MacOS
* Add optional X Window support
* Specifically build against qt@4 when building QT support
* Point to appropriate C++ standard libraries when building with
Clang on MacOS
* Disable parallel build (for all platforms)
* Increase verbosity of configure script by adding
--disable-silent-rules
* Add numpy 1.17.0
* Overhaul numpy package
* Flake8 fixes
* Undefined reference fix
* HeaderList and LibraryList need an arg
* veclibfort has no headers
* Add patch for older versions of py-numpy
* Remove py-meep hack from py-numpy package
* libflame: always add max arg hack flag
* Fix build with GCC 4.8
* Compiler flags come from self.compiler
* Only apply -std=c99 to cflags
* Try to fix libflame package
* Fix ATLAS build on macOS
* --force-clang flag added in 3.10.3
When building with GCC, c-blosc requires the -std=gnu99 flag.
Before this patch, the build would only add -std=gnu99 for GCC when
building on specific architectures; this adds -std=gnu99 in all cases
when using GCC.
When loading `openpmd-api` as a lib to build outside of
spack or to build depending spack packages, the "CMake-PUBLIC"
header-only library MPark.Variant must be available. Also,
the `*Config.cmake` package must be able to find dependent
libs if used.
* new package: py-backports-tempfile
* Update var/spack/repos/builtin/packages/py-backports-tempfile/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* remove rc version
* fix single quote
Use `spack pkg changed` and `spack maintainers` to figure out which
packages changed and who their maintainers are in a PR.
Add any maintainers to the PR as reviewers.
Using "compilers" with the "s" is an invalid config section and throws an error.
Traceback (most recent call last):
File "spack/bin/spack", line 48, in <module>
sys.exit(spack.main.main())
File "/home/omsai/src/libkmap/spack/lib/spack/spack/main.py", line 633, in main
env = ev.find_environment(args)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 263, in find_environment
return Environment(env)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 534, in __init__
self._read_manifest(f)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 561, in _read_manifest
self.yaml = _read_yaml(f)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 402, in _read_yaml
validate(data, filename)
File "/home/omsai/src/libkmap/spack/lib/spack/spack/environment.py", line 395, in validate
e, data, filename, e.instance.lc.line + 1)
spack.config.ConfigFormatError: /home/omsai/src/libkmap/spack.yaml:15: Additional properties are not allowed ('compilers' was unexpected)
* add new version, add sha256 sums
* Update var/spack/repos/builtin/packages/py-apipkg/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add python dependencies
* new package: py-usgs
* Update var/spack/repos/builtin/packages/py-usgs/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-usgs/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* new package: py-requests-futures
* Update var/spack/repos/builtin/packages/py-requests-futures/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
Environment.concretize returns newly-concretized specs rather than
printing them; as a result, the _display argument is removed from
Environment.concretize (originally only used to avoid printing specs
during unit testing). Command logic which invokes
Environment.concretize prints these explicitly.
* minimap2: remove py-mappy dependency and build on ARM
py-mappy: Change URL and new version.
* merge py-mappy to minimap2.
* 1. add type in depends_on.
2. remove unneeded which.
* Incorporate as a patch upstream commit 0c20dbf059 (diff-acb9169b4ccbac494ced5140d285a015) resolving problems with `-Wno-...` options not being understood by GCC.
* Incorporate as a patch upstream PR https://github.com/ericniebler/range-v3/pull/1202 fixing upstream issue https://github.com/ericniebler/range-v3/issues/1196.
* Compile tests only if we actually want to run them.
* Optionally compile documentation with a `+doc` variant.
* Add `+examples` and `+perf` variants to avoid unnecessary compilation of components usually not required for install.
* Add conditional build-only dependency on Boost when appropriate.
* Install docs, examples and performance tests if built (not part of upstream build instructions).
* Add @chissg as a maintainer.
- Update c-blosc to 1.17.0
- Add patch to build test_forksafe for versions 1.15.0 through 1.17.0
when compiled with the Intel compiler.
- Remove code that turned off building the test with the Intel compiler
1. xmmintrrin.h is included x86 only.
2. Add makefiles for non-x86 hosts.
3. mpicc in Makefiles is replaced spacks mpi compile command.
4. variant check is rewite with satisfies.
* Geant4 data packages: Set the install path name explicitly.
* Use the correct directory name for g4photonevaporation,g4radioactivedecay,g4realsurface
* Add versions 8.0.16, 5.7.26, and 5.6.44
* Update boost dependency for versions 8.0.16, 8.0.15, and 8.0.14
* Update dtrace script when building for versions before 8 as is done
for the glib package
* Set perl as a build dependency for versions before 8
* cxxstd=17 conflicts with version 8 when server code is built
(~client_only)
* Add -Wno-deprecated-declarations for cxxstd > 11
* Add -Wno-error=register for cxxstd > 14
This updates the Spack QT package to enable building qt version 4 on
MacOS.
This includes the following changes to the qt package:
* add version 4.8.7
* add option to build with or without shared libs
* add options to disable tools, ssl, sql, and freetype support
* add qt4-tools patch when building qt@4+tools
* add option to build as a framework (only available on MacOS)
* replace qt4-el-capitan patch with qt4-mac patch (which includes the
edits from qt4-el-capitan)
* apply qt4-pcre-include-conflict.patch only for version 4.8.6
(rather than all 4.x versions)
* apply qt4-gcc-and-webkit.patch for 4.x versions before 4.8.7 and
create a separate qt4-gcc-and-webkit-487.patch for version 4.8.7
* update patch function for qt@4 on MacOS to update configure
variables relevant to Spack (e.g. PREFIX)
* add option to build freetype with Spack, as a vendored dependency
of QT, or not at all (default is to build with Spack)
This includes the following edits outside of the qt package:
* Update MacOS version utility function to return all parts of the
Mac version (rather than just the first two)
* gettext package: implement "libs"
* python package: add gettext as a dependency
* Raise an exception and exit with a meaningful message when binary path substitution fails.
* Skip binary text replacement with padding and issue a warning when the new install path is longer than the old install path.
Fixes#12007.
The Cray cc wrappers don't handle the -s flag (strip) cleanly, It's
not essential to strip the binary, so just remove the flag on Cray.
Note: the default build on Cray is for the back end and the unzip
binary won't run on the front end. To build for FE, use something
like arch=cray-fe-x86_64.
* Add +shared variant, which builds shared library in addition to the
static library.
* Install libraries even when specifying the header-only option
(header-only is just about installing an additional folder).
* An additional make call is not required to build generator
executables (they are built by default).
* Streamlined help-line of each variant.
* Fix Mac platform check for dependency in py-ipython package: 'when'
constraints in Spack directives must be Specs (either a Spec
object or a Spec in string format)
* Fix Mac version check in py-numpy: platform.mac_ver() returns a
3-part string as its first tuple item so the check as written would
never pass; use Spack Version object to simplify check.
* Fix Mac version check in qt package (the check was incorrectly
comparing ints and strings) and use Spack version object to
simplify check.
* Add versions 4.3.0, 4.3.1, and 4.3.2
* The URL format changed after 4.1.4, so this adds a url_for_version
function to choose the URL format based on the version
With this PR, a user can designate older versions of OpenGL as an
external Spack package, and dependents can use that installation
as long as they do not have version requirements.
MacOS currently comes with OpenGL 2.1; there is currently no
'provides' directive in the OpenGL Spack package which specifies a
gl version provided for versions earlier than OpenGL 3.3, so earlier
versions of OpenGL are not considered to provide any version of gl.
This PR records that any version of OpenGL provides 'gl' (which is
sufficient for any package that does not require a specific version
of gl).
* Version 1.3.0 requires python 3.5 or later (no Python 2 support)
* Remove test of scipy.weave import (not available since 1.0)
* Add more-sensitive py-numpy constraints based on py-scipy version
* Replace py-nose dependency with py-pytest
* Add adamjstewart as maintainer
* Rename py-pytorch to py-torch
* Add versions 1.1.0 and 1.0.1
* Define modules to test import of after installation (import_modules)
* py-typing dependency is only needed for older versions of Python
(3.4 or before)
* Newer versions of py-torch depend on newer versions of CUDA
* Add adamjstewart as maintainer
* Remove libuuid dependency for python on Darwin
Python 2.7 includes Carbon, which requires a proprietary typedef defined in
the system `/usr/include/uuid/uuid.h`:
```
In file included from /rnsdhpc/code/spack/var/spack/stage/python-2.7.16-chgwvgls4ezgujsknn27vlmkhwbzpqik/Python-2.7.16/Python/mactoolboxglue.c:27:
In file included from /rnsdhpc/code/spack/var/spack/stage/python-2.7.16-chgwvgls4ezgujsknn27vlmkhwbzpqik/Python-2.7.16/Include/pymactoolbox.h:10:
In file included from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:20:
In file included from /System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:23:
In file included from /System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AE.h:20:
In file included from /System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/CarbonCore.h:208:
In file included from /System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/HFSVolumes.h:25:
/usr/include/hfs/hfs_format.h:794:2: error: unknown type name 'uuid_string_t'; did you mean 'io_string_t'?
```
Similarly, even with Python 3, having Spack's libuuid in the toolchain
causes matplotlib to fail to build downstream because its `osx` backend
links against Carbon:
```
building 'matplotlib.backends._macosx' extension
/rnsdhpc/code/spack/lib/spack/env/clang/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include -I/usr/X11/include -I/opt/X11/include -I. -I/rnsdhpc/code/spack/opt/spack/darwin-mojave-x86_64/clang-10.0.1-apple/python-3.7.2-bipx24oyxqxowqc75o5vfih7dljiaohz/include/python3.7m -c src/_macosx.m -o build/temp.macosx-10.14.5-x86_64-3.7/src/_macosx.o
In file included from src/_macosx.m:1:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:87:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSURLError.h:15:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:24:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AE.h:20:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/CarbonCore.h:208:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/HFSVolumes.h:25:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/hfs/hfs_format.h:794:2: error: unknown type name 'uuid_string_t'; did you mean 'io_string_t'?
uuid_string_t ext_jnl_uuid;
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/device/device_types.h:89:33: note: 'io_string_t' declared here
typedef char io_string_t[512];
```
MacOS version 10.14.5
Xcode 10.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
* Mark libuuid as conflicting with Darwin due to framework
If Spack's libuuid is included by any other package that tests for or
requires the Cocoa or Carbon frameworks, the build will *fail* becuse it
appears that Apple's libraries require a special version of libuuid and
uuid.h.
```
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Cocoa.framework/Headers/Cocoa.h:12:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:87:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSURLError.h:15:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:24:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AE.h:20:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/CarbonCore.h:208:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/HFSVolumes.h:25:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/hfs/hfs_format.h:794:2: error: unknown type name 'uuid_string_t'; did you mean 'io_string_t'?
uuid_string_t ext_jnl_uuid;
^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/device/device_types.h:89:33: note: 'io_string_t' declared here
typedef char io_string_t[512];
```
* Revert "Mark libuuid as conflicting with Darwin due to framework"
This reverts commit 9a68b75202.
* Document libuuid exception in python package
* update to kim-api-2.1.1 and openkim-models-2019-07-25
* openkim-models provide needed dependency constraints and settings
* add maintainers and fixup flake8 errors
* font-util: Remove duplicated dependencies.
* font-util: Fix path handling for font resources.
The "join_path" call made no sense but still managed to work somehow most of the times.
If +pthreads is specified, googletest actually gets build without
pthreads support and vice versa.
If pthreads is not available, googletest should detect it itself which
is why "enabling" pthreads in case of ~pthreads does not seem to cause
any error.
Change-Id: I29de8c4a23dc3901124b27a1ead3354949b7e65b
* xsdk: phist has a conflict with gcc version <= gcc-4.9.1 - so add a variant to disable it on older compiler builds
This is triggered by #11677
* fix typo
* Added a new package for LAME and a variant in sox for it.
* Fixed download url for LAME
* libmad: Added new package libmad for mp3 decoding. Changed sox variant from lame to mp3.
* libmad: Added m4 as a requirement
* libmad: Added comment on patch origins
* libmad: Added a list_url for alternate download location
* libmad: Fixing libmad download url
- We don't currently make enough use of the maintainers field on
packages, though we could use it to assign reviews.
- add a command that allows maintainers to be queried
- can ask who is maintaining a package or packages
- can ask what packages users are maintaining
- can list all maintained or unmaintained packages
- add tests for the command
* Add QMCPACK v. 3.8.0
* Remove QE variant conflict for serial QMCPACK installations.
* Add support for QE 6.4.1 converter.
* FindLibxml2QMC patch function only needed prior to QMCPACK 3.8.0
* Added a unit test reproducing the failure in 12085
* Fixed name clash in the 'from_environment_diff' function
The bug reported in #12085 stemmed from a name clash among variables,
introduced during the refactor in #10753 and not caught by unit tests
and reviews.
* ioapi: add version 3.2
* ioapi: unwrap version checksum; it's exempt from line length QA
* ioapi: drop overridden build(); it's the same as the default
* cmake: Move cxx11 flags for fujitsu into the flag_handler method
* cmake: sanitization and error checking in flag_handler
* cmake: Fix fujitsu flag check from 'fc' to 'fj'
- Setting specs from lockfiles was not correctly stringifying concretized
user specs.
- Fix `_set_user_specs_from_lockfile`
- Add some validation code to `SpecList` constructor
Spack has evolved to have three types of hash functions, and it's
becoming hard to tell when each one is called. Whlie we aren't yet ready
to get rid of them, we can refactor them so that the code is clearer and
easier to track.
- Add a `hash_types` module with concise descriptors for hashes.
- Consolidate hashing logic in a private `Spec._spec_hash()` function.
- `dag_hash()`, `build_hash()`, and `full_hash()` all call `_spec_hash()`
- `to_node_dict()`, `to_dict()`, `to_yaml()` and `to_json()` now take a
`hash` parameter consistent with the one that `_spec_hash()` requires.
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
- ensure that Spec._build_hash attr is defined
- add logic to compute _build_hash when it is not already computed (e.g. for specs prior to this PR)
- add test to ensure that different instance of a build dep are preserved
- test conversion of old env lockfile format to new format
- tests: avoid view creation, DAG display in tests using MockPackage
- add regression test for more-general bug also fixed by this PR
- update lockfile version since the way we are maintaining hashes has changed
- write out backup for version-1 lockfiles and test that
* openjdk
* Removing preferred from latest and adding a few lines for readability.
* Remove bad whitesapce
* removing formatting changes
* stable url for openjdk 1.8
* remove extra blank line
* actual binary
Add version for 'gpu' and variant for 'cuda'. These are currently available from the 'master-gpu' branch.
* hpctoolkit: add version gpu and variant cuda
* The gcc >= 4.8.x requirement only applies for dyninst >= 10.x.
* fix docker builds/remove extra builds/add ci builds
* preinstall vim in CI builder images
* simplify & streamline docker resources
* restore os-container-mapping.yaml file
The database and mutable_database fixtures were installing and uninstalling the same specs multiple times to ensure the database for tests has the correct state.
This commit optimizes the procedure by caching the state in an external directory, and copying it in instead of going through the installation or uninstallation again.
The database fixture is meant not to be modified by tests. This commit enforces this invariant by making the database read-only before starting the test.
* Added missing db markers to tests
* Added test for uninstall_by_spec
* `database` fixture now returns a read-only database
* Tests that modify the DB now use `mutable_database` fixture
ibm-java: fix the URL to only use a machine type that ibm-java
supports so that 'spack info' doesn't report a non-existent URL on
platforms that it doesn't support.
Summary:
- Allow multiple definitions of compiler in compilers.yaml (use first instance)
- Still print debug messages when there are duplicates, to assist users in finding this issue.
Merging configs from different scopes can result in multiple compiler being present in the same configuration list. Instead of raising when there are duplicates, take the one with highest precedence.
Print a debug message instead of raising, so that we can still diagnose this. We don't have a good way of warning the user about inconsistent configuration *in the same file* -- we'd need to dig into YAML file/line info for that.
- [x] Add shell tests to ensure that `spack env activate`, `spack env
deactivate`, and `despacktivate` continue to work.
- [x] Also ensure that activate and deactivate both work with `set -u`
* extends mkdirs with permissions for intermediate folders
Does not use os.makedirs mode parameter because its behavior is changed
with Python 3.7 (it ignores it for intermediate dirs), and moreover it
was not possible to set different modes for newly-created folders
and leaf folder.
reference:
- https://bugs.python.org/issue19930
- https://docs.python.org/3.7/library/os.html#os.makedirs
* comment mkdirp step easing code understanding
* revert mkdir to default for package metapath
since metapath is nested in package folder, there is no need
to specify permissions for intermediate folders because the prefix
already exists.
* comment create_install_directory package modes
Bug relates to the interplay between:
1. random dict orders in python 3.5
2. bugfix in initial implementation of stacks for `_concretize_dependencies`
when `self._dependencies` is empty
3. bug in coconcretization algorithm computation of split specs
Result was transient hang in coconcretization.
Fixed#3 (bug in coconcretization) to resolve.
- remove redundant code in Environment.__init__
- use socket.gethostname() instead of which('hostname')
- refactor updating SpecList references
- refactor 'specs' literals to a single variable for default list name
- use six.string_types for python 2/3 compatibility
* Fix scikit-learn build with Apple clang
* Update compiler unit tests
* Fix unit tests
* Fix OpenMP dep
* recursive=True no longer necessary
* Add myself as a maintainer
* Specify which versions require OpenMP
* Typo fixes
* LLVM-OpenMP is the correct dependency, no OMPT
* Flake8 fix
* Undo Apple Clang OpenMP flag change, split into a separate PR
Later versions of libjpeg-turbo build with CMake; to build with
user-specified cflags, the user must supply these to CMake as
-DCMAKE_C_FLAGS (Spack's typical approach of injecting these flags
into the compiler wrapper invocation is insufficient in this case).
Currently libjpeg-turbo cannot be implemented as a CMakePackage
(and thereby take advantage of the flag_handler implementation it
provides) because not all versions of libjpeg-turbo use CMake, so
this adds a custom implementation of flag_handler and
flags_to_build_system_args to libjpeg-turbo.
* cmake: change c++ spec when building with Fujitsu compiler.
* cmake: define cxxflags using a Fujitsu compiler's method.
* cmake: Add condition of whether to add cxx11 option.
* from_sourcing_file: fixed a bug + added a few ignored variables
closes#7536
Credits for this change goes to mgsternberg (original author of #7536)
The new variables being ignored are specific to Modules v4.
* Use Spack Executable in 'EnvironmentModifications.from_sourcing_file'
Using this class avoids duplicating lower level logic to decode
stdout and handle non-zero return codes
* Extracted a function that returns the environment after sourcing files
The logic in `EnvironmentModifications.from_sourcing_file` has been
simplified by extracting a function that returns a dictionary with the
environment one would have after sourcing the files passed as argument.
* Further refactoring of EnvironmentModifications.from_sourcing_file
Extracted a function that sanitizes a dictionary removing keys that are
blacklisted, but keeping those that are whitelisted. Blacklisting and
whitelisting can be done on literals or regex.
Extracted a new factory that creates an instance of
EnvironmentModifications from a diff of two environments.
* Added unit tests
* PS1 is blacklisted + more readable names for some variables
It seems that GitHub's output changed slightly for this patch (one more
letter for both short hashes on the index line), probably due to the
number of objects in the repository.
All documentation mentions that `build_jobs` is limited by the number of
cores available in the system. This is also enforced when setting it via
`--jobs`. However, when setting it via `config.yaml`, it can exceed the
number of cores available, making builds run out of memory.
This PR adds the ability to specify the auto-dispatch targets that can
be used by the Intel compilers. The `-ax` flag will be written to the
respective compiler configuration files. This ability is very handy when
wanting to build optimized builds for various architectures. This PR
does not set any optimization flags, however.
Fixes#3690Fixes#5637
Uninstalling dependents of a spec was relying on a traversal of the
parents done by inspecting spec._dependents. This is in turn a
DependencyMap that maps a package name to a single DependencySpec object
(an edge in the DAG) and cannot thus model the case where a spec has
multiple configurations of the same parent package installed (for
example if different versions of the same Python library depend on
the same Python installation).
This commit works around this issue by constructing the list of specs to
be uninstalled in an alternative way, and adds tests to verify the
behavior. The core issue with DependencyMap is not resolved here.
The default library search for a package checks the lib/ and lib64/
directories for libraries before the root prefix, in order to save
time when searching for libraries provided by externals (which e.g.
may have '/usr/' as their root).
This moves that logic into the "find_libraries" utility method so
packages implementing their own custom library search logic can
benefit from it.
This also updates packages which appear to be replicating this logic
exactly, replacing it with a single call to "find_libraries".
Fixes#11782
Spack was not properly resolving relative paths to absolute paths
when a relative path was passed to "spack compiler add [PATH]".
Now, if provided a relative path, the absolute path is written to
compilers.yaml rather than the relative path.
* Add template creation test
* Added --skip-editor option to "spack create": normally
"spack create" opens an editor for the user after generating a
package file; when the --skip-editor option is used, "spack create"
only generates the package file and does not open an editor
* Added --skip-editor option to bash completion
- `setup-env.sh` was not properly detecting a bash shell when bash was run
as /bin/sh.
- Detection routine now always reports bash when bash is run as sh, and
no longer parses the path to the executable indicated in `$BASH`.
- Namepsaces were shown without dots after the new format strings were
added.
- Add a test for `spack find` to ensure that find -N shows the right
output.
This PR has several modifications for the r package.
- The tk package is always depended on but this pulls in X11, making the
'X' variant non-functional. This PR sets a dependency of tk on '+X'.
- There is a missing dependency on libxmu when '+X' is set.
- The libraries for R wind up in a non-standard location and are thus
left out of the RPATH settings. This PR adds that directory to RPATH.
- The MKL integer interface for gfortran is not in the BLAS libs. This
PR replaces the intel interface with the gfortran interface if needed.
- Use the `LibraryList` `ld_flags` method for blas as that is more in
line with th R Installation and Administration manual.
Note that this PR depends on PR #11956. This PR closes#8642.
TK can be built with support for libXft and X Screensaver. Both of these
are turned on in the Tk configure script. That means the system
libraries will get picked up if they are on the system and nothing is
specified in the package. Since the default for both of them is 'True' I
set the default value for the variants to 'True'.
* bowtie2: Add version 2.5.3.1 and fix to build on ARM
1. Add version 2.5.3.1.
2. Add simde for ARM.
3. Disable -fopenmp-simd when gcc 4.8.X or before for ARM.
4. Add Makefile option POPCNT_CAPABILITY=0 for ARM.
* flake8.
Fixes#11781
* Rename build log to spack-build-log.txt
* Rename environment variables file to spack-build-env.txt
* The name of the log and env files is now the same during the build
and after the build completes
* Update packages which referred to the build log/env files
* For packages installed before this commit using older names for the
build and env files, search for the older names
* ferret: use libx11 from spack
* ferret: use mpi library with or without fortran support
* ferret: add version 7.2
* ferret: no need to specify ~mpi
* ferret: creating symlinks may fail for external packages; set include/lib paths instead
* use autoreconf to regenerate configure script and config.guess for
newer architectures
* Add perl build dependency for older versions. The required perl
version is constrained because the perl script uses syntax that
is deprecated in older versions
- Fix a bug introdcued by removing parse_anonymous_spec()
- Conflicts' when specs are now *actually* anonymous, and the name of the
package is implicit, so we need to remember to add it back to error
messages.
Declare a conflict for TAU versions older than 2.28.1, that they
can't depend on libelf or libdwarf. The user can still install it
by specifying tau@2.24~libelf~libdwarf.
- Add set -u to the setup-env.sh test script
- Refactor lines in setup-env.sh that tested potentially undefined
variables to use the `[ -z ${var+x} ]` construct
- tests use a shell-script harness and test all Spack commands that
require special shell support.
- tests work in bash, zsh, and dash
- run setup-env.sh tests on macos and linux builds.
- we run them on macos and linux
- replace use of [[ with [
- replace function foo { .. } with foo() { .. }
- wrap some long lines
- add lsof and /proc/fd magic so that we can find the sourced file even in dash
- only do the complicated shell checks in one place; test $_sp_shell
elsewhere.
* py-antlr4-python3-runtime: new package
* py-antlr4-python3-runtime: fix for flake8
* py-antlr4-python3-runtime: indent docstring
* py-antlr4-python3-runtime: clarify role of this package vs ANTLR in docstring
* py-antlr4-python3-runtime: remove whitespace at end of line
The py-reportlab package includes binaries for easy_install and pip.
That blocks activation if py-setuptools and/or py-pip are installed.
This PR block the binaries from py-reportlab as easy_install and pip
should be installed with their respective packages.
- There's too much information on our PRs, and the Travis test results
(which are arguably more important than coverage) get buried in
coverage information
- Remove coverage sub-categories, as you can browse codecov's file view
to see roughly the same thing
- show only project/patch on GitHub PRs.
* py-cf-units: new package
* py-cf-units: download from pypi.io
* py-cf-units: include py-pytest-runner as build dep
* py-cf-units: remove empty build_args function
* py-cf-units: fix for flake8
* py-cf-units: indent docstring
* py-cf-units: remove excess spaces
* py-cf-units: list master branch ahead of numeric versions
* py-cf-units: remove extension keyword from version
If R is provided by the system with a module (i.e. registered in
packages.yaml with a "module" entry) and has no discernible prefix,
this allows turbine to still make use of R.
This PR corrects a problem with the opa-psm2 package. It was
installing headers into <prefix>/usr/include and libraries into
<prefix>/usr/lib instead of <prefix>/include and <prefix>/lib.
The extra level of usr/ subdirectories causes dependent packages
like libfabric to fail to build.
The original packaging, with openfoam as a virtual and with
openfoam-com, openfoam-org and openfoam-extend as
providers, adds an obfuscation layer without any benefit.
- Rename `openfoam-com` to `openfoam`, and remove the
`openfoam` virtual package.
- `parse_anonymous_spec()` is a vestige of the days when Spack didn't
support nameless specs. We don't need it anymore because now we can
write Spec() for a spec that will match anything, and satisfies()
semantics work properly for anonymous specs.
- Delete `parse_anonymous_spec()` and replace its uses with simple calls
to the Spec() constructor.
- make then handling of when='...' specs in directives more consistent.
- clean up Spec.__contains__()
- refactor directives and tests slightly to accommodate the change.
- CNL OS previously used the *Cray PE* version to determine the OS
version. Cray does not synchronize PE and CLE releases; you can run
CLE7 with PrgEnv 6 (and NERSC currently does).
- Fix Spack's OS detection by using the cle-release file to detect the OS
version. This file is updated with every CLE OS release.
- Add some tests for our parsing logic
Added a wrapperrpath variant to mpich that defaults to `True`.
Users can set this to `False` (e.g., spack install mpich~wrapperrpath) to
disable the wrapper RPATHs. This makes it easier to replacing an MPICH
installation in a container with an MPICH for the host system,
e.g., Cray MPICH with Shifter on Cori, Intel MPI, MVAPICH2, etc.
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
- Trying to reduce the number of top-level files in Spack.
- CODE_OF_CONDUCT.md can be referenced from the README.
- CONTRIBUTING.md is only used to display a message on pull requests, and
it still does that when in .github.
The version of LLVM used by flang is new enough that CppBackend doesn't exist. Unfortunately, `flang-xxxxxxxx` is seen as < `3.9.0` by the version check.
* add a special case for `flang` versions.
- We've seen this a few times now where users have set up `cd` to echo
the new directory, and it screws up `setup-env.sh`
- In the past we've said this is user error.
- Here, we just fix it by sending `cd` output to /dev/null where needed.
- this works in bash, zsh, and dash
Add an example of a 'modules:' entry for an external package in
packages.yaml. The 'External Packages' section of 'Build
Customization' mentions 'paths:' and 'modules:' and gives an
example of paths, but not modules.
* OpenFOAM: rationalize architecture options
- older versions of OpenFOAM had WM_ARCH_OPTION to choose a 32-bit
build on 64-bit architectures. This is not exactly relevant anymore
and has now been largely removed from OpenFOAM.
- simultaneously the newest version of OpenFOAM has rationalized the
handling of special settings for KNL by rolling these into the
compiler options. We now do the same thing here in spack.
1. Reuse the internal variable arch_option to convey processor-specific
optimizations and add these into the C++OPT=... statement.
2. Drop spack support for configuring a 32-bit build on 64-bit
hardware.
These changes can also be applied to older OpenFOAM versions, and to
various OpenFOAM forks without issue.
* update openfoam versions. New release 1906, patched 1812 version.
ncl does not build with newer versions of gdal (or rather proj). This
also needs a small change to gdal, otherwise we end up with both proj@5
and proj@6 in the spec.
* Remove individual font-* packages and add them as resources of the
font-util package
* Update font-util package to Autotools-based package, and call
autoreconf to regenerate the old configure script
* Add font-adobe-utopia-type1
* fmt: Add cxxstd variant, plus cmake/c++ patches
Spack supported versions of fmt default to C++11 for versions
less than 5, C++14 greater than 5, with fmt implementing
fallbacks to whatever compiler supports.
To give better ABI compatibility and use of newer standards, provide
a `cxxstd` variant defaulting to 11 with 98-17 options. Use cmake_args
to set CMAKE_CXX_STANDARD as appropriate, plus use of `FMT_USE_CPP11`
option for C++98 builds. Use `conflicts` to disable use of certain
standards in versions that don't support, or fail the build, with
those standards.
Add patches to unify use of `CMAKE_CXX_STANDARD` in 3.0 versions,
remove hard-coding of compiler flags in `fmt-config.cmake`, and
prevent use of C++11 features in 4.1.0 when in supported C++98
mode.
Default to not building documents as no dependency on Doxygen is
yet present, and they are not part of the "all" build.
* Use CMake to enforce C++ standard support
Fail configure step if fmt tries to build with a cxxstd variant
not supported by the compiler (or known to CMake).
* fmt: New version 5.3.0
* The py-pytables package depends on hdf5-blosc
* Further modifications to py-pytables package
The 3.2.2 version of py-pytables should still build with internal blosc.
The issue with locking in a multithreaded environment has been
fixed/worked around in version 3.3.
Also, add bzip2 and lzo variants.
* new package/py-blessed
* make flake8 happy
* fix docstring
* tab to spaces
* Update var/spack/repos/builtin/packages/py-blessed/package.py
specify dependency version.
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-blessed/package.py
specify dependency version
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* add setuptools as depedency
Fixes#11816
Allow packages to refer to non-expanded downloads (e.g. a single
script) using Stage.archive_file. This addresses a regression from
#11688 and adds a unit test for it.
This change reverts to the previous behavior of only looking for pgcc
and friends, not pgcc-llvm and friends.
The llvm variant doesn't support all the same features as the
traditional variant of the pgi code generator; this change avoids
treating the llvm variant as a default pgi compiler.
This retains the changes in #10704 which accept the "LLVM" suffix of
the version string of the PGI compiler, which allows users to
explicitly add the llvm-pgi compiler if desired.
Starting with 1.69.0, boost added a bjam option for the default symbol
visibility. Up to 1.68.0, the value was always 'global'. 1.69.0
changed the default to 'hidden' but added an option.
Most packages will work with hidden and won't notice. But some
packages may discover that an interface that they rely on is now
hidden and inaccessible.
https://boostorg.github.io/build/manual/develop/index.html#bbv2.builtin.features.visibility
For resources, it is desirable to use the expanded archive name of
the resource as the name of the directory when adding it to the root
staging area.
#11528 established 'spack-src' as the universal directory where
source files are placed, which also affected the behavior of
resources managed with Stages.
This adds a new property ('srcdir') to Stage to remember the name of
the expanded source directory, and uses this as the default name when
placing a resource directory in the root staging area.
This also:
* Ensures that downloaded sources are archived using the expanded
archive name (otherwise Spack will not be able to determine the
original directory name when using a cached archive).
* Updates working_dir context manager to guarantee restoration of
original working directory when an exception occurs
* Adds a "temp_cwd" context manager which creates a temporary
directory and sets it as the working directory
Starting with 2.1.0, backports.functools_lru_cache replaces functools32
when using python 2.7.x until python2 support is dropped in 3.0.0.
Change-Id: I8db61eac975da0042f6f5babe0f5f0c92807200e
Fixes#11526
The xfd configure script tests for libintl but incorrectly concludes
that no additional library is needed because gettext is provided in
libc. So we add '-lintl' to ldlibs to point to the Spack-installed
gettext. Note that the xfd configure script does not have a
--with-gettext option to do this for us.
Also add version 1.1.3
The regression test for #11678 fails on at least some Mac OS systems
because they have a /usr/bin/gcc that is secretly clang.
This PR replaces the dependency on a system gcc executable with a
test-generated script that generates the expected output for the
compiler logic.
All macos tests are failing because brew cannot install ccache
without updating brew. This ensures that brew update is run
before using brew in test environment.
* xtensor-python: add version 0.23.1
* xtensor-python: update xtensor/xtl dependency version constraints
for 0.23.1
* xtensor-python: remove xtensor/xtl dependency version constraints
for the develop version to favor the latest (develop) version of
these dependencies
* xsimd: add version 7.2.3
* xtensor: add tbb variant to enable TBB parallelization
* xtensor: add version 0.20.7
* xtensor: update CMake args enable xsimd/tbb support based on
variant settings
* xtl: add version 0.6.4
* Added Cinch Spackage and updated Flecsi to Use It
Added spackage for Cinch build system and updated flecsi spackage to use
the external version of cinch
* Flake8 compliance for Cinch and Flecsi
* Fixed Cinch Issue and Requested PR Changes
Swapped cinch back to master branch rather than test branch and style
issues for PR
repeatmasker depended on HMMER but was never actually configured to
use it. The package also had many shebangs pointing to random perl
locations, now patched to use Spack's perl installation.
* Also add version 0.10.0
* Add leading '0' to 0.9.x versions (e.g. 0.09.1) and 0.8.x versions
* Don't mention acts-framework, that package isn't in Spack yet
* Updates to libMesh: added v1.3.1 checksum, added vtk variant, bugfix for enabling petsc when slepc variant is specified
* fixed indentation issues identified by Travis CI
* - updates version number
- added master as a version
- added new variant options for perflog, metaphysicl and blocked storage
-- bug-fix for metis
* fixes for spack indentation errors
* renaming develop to master following review comment of PR.
Some tests introduced in #11528 temporarily set the user's `config:build_stage`, which affected (or created) a config.yaml file in the user's `$HOME/.spack` directory that could leave entries behind if the tests fail.
This change ensures only temporary configuration files are used/affected by these tests.
- A program that generates ASCII pictures of a cow with a message.
- This is used by the ISC19 Singularity tutorial, so we should probably
have it in Spack.
* new package: mutationpp
* Update package.py
* flake8: correct package.py for flake8-python-style
* Update package.py
* modifications as requested by reviewer for #11705
The "spack location" command was previously untested. This also adds
a check to ensure that composite Stages can report whether they were
expanded (this property was previously only recorded in Stage but not
in CompositeStage).
DIYStage, used to treat a user-managed directory as a staging area,
should always be considered expanded (i.e. the source has been
decompressed if it was stored in an archive).
This also:
* Adds checks to ensure that the path used to instantiate a
DIYStage refers to an existing directory.
* Adds tests to check the behavior of DIYStage (including behavior
added here, but it was generally untested before).
Change 'destination' to 'placement' in the use of resource() for
intel-xed and hpcviewer. Recent commits (1842873f and eb584d89)
changed the directory path for a resource and thus broke the build
for these packages.
#11528 updated Stage to always store a Package's source in a fixed
directory accessible via `Stage.source_path` This left behind a
number of packages which were expecting to access the source code
via `Stage.path`. This Updates those packages to use
`Stage.source_path` instead.
This also updates the name of the fixed directory: The original name
of the fixed directory was "src", so if an expanded archive created a
"src" directory, then users inspecting the directory structure could
see paths like "src/src" (which wasn't wrong but could be confusing).
Therefore this also updates the name of the fixed directory to
"spack-src".
Fixes#11678
`spack compiler find` was not searching `PATH` when provided with no
arguments. ea7910a updated the API for the search function and the
command logic did not update how it called this function. This also
adds a test to ensure that `spack compiler find` will collect
compilers from `PATH`.
* warpx: only allow master branch, and fix build on darwin
* warpx: Added CXXFLAGS=-std=c++11 for clang
* warpx: further clean up of openmp and c++11 flags
* warpx: Get around the problem with resources
The update uses placement instead of destination
Fixed logic behind flecsi backends to not require legion for an mpi
build and to better support required combinations.
Also made graphviz support and tutorial building optional
* Paraview/VTK: Fix failure on intel builds
Paraview@5.6.0 fails to build with intel compilers on certain platforms.
Error messages have been traced to the optional MotionFX lib.
(see https://discourse.paraview.org/t/error-build-5-6-0-rc1/572)
Turning off motionfx allows for succesful builds on these platforms.
"spack module tcl find -r <spec>" (and equivalents for other module
systems) was failing when a dependency was installed in an upstream
Spack instance. This updates the module index to handle locating
module files for upstream Spack installations (encapsulating the
logic in a new class called UpstreamModuleIndex); the updated index
handles the case where a Spack installation has multiple upstream
instances.
Note that if a module is not available locally but we are using the
local package, then we shouldn't use a module (i.e. if the package is
also installed upstream, and there is a module file for it, Spack
should not use that module). Likewise, if we are instance X using
upstreams Y and Z like X->Y->Z, and if we are using a package from
instance Y, then we should only use a module from instance Y. This
commit includes tests to check that this is handled properly.
Spack currently tries to unify everything in the DAG, but this is too strict for build dependencies, where it is fine to build a dependency with a tool that conflicts with a version fo that tool for a dependent's build.
To enable a workaround for conflicts among build dependencies, so that users can install in multiple steps to avoid these conflicts, make the following changes:
* Dont apply package dependency constraints for build deps of installed packages
* Avoid applying constraints for installed packages vs. concrete packages
* Mark all dependencies of installed packages as visited in normalization method
* don't remove dependency links for concrete specs in flat_dependencies
Also add tests:
* Update test to ensure that link dependencies of installed packages have constraints applied
* Add test to check for proper handling of transitive dependencies (which is currently not the case)
ADIOS2 is the latest implementation of the ADaptable Input Output System,
ADIOS. This brand new architecture was designed to continue supporting the
performance legacy of ADIOS, and extend its current capabilities to address
current and future input/output (IO) challenges in the scientific data
lifecycle through effective research and development (R&D) activities.
ADIOS2 was recently added as a Third Party Library of Trilinos when it
was integrated to IOSS, a library inside the SEACAS package.
- spack.compilers.find_compilers now uses a multiprocess.pool.ThreadPool to execute
system commands for the detection of compiler versions.
- A few memoized functions have been introduced to avoid poking the filesystem multiple
times for the same results.
- Performance is much improved, and Spack no longer fork-bombs the system when doing a `compiler find`
- We use `spack list --foramt=html` now, as it is much faster and doesn't
make the docs build take forever.
- Remove `spack list --format=rst` as it is no longer used.
- `stage.source_path` was previously overloaded; it returned `None` if it
didn't exist and this was used by client code
- we want to be able to know the `source_path` before it's created
- make stage.source_path available before it exists.
- use a well-known stage source path name, `$stage_path/src` that is
available when `Stage` is instantiated but does not exist until it's
"expanded"
- client code can now use the variable before the stage is created.
- client code can test whether the tarball is expanded by using the new
`stage.expanded` property instead of testing whether `source_path` is
`None`
- add tests for the new source_path semantics
- make tty.msg, tty.info, etc. print the exception type and stringified
message if the message argument is an exception.
- simplify parts of the code that call tty.debug(str(e))
- add extra tty.debug statements in places where exceptions were
previously ignored
- `spack graph --static` (and `spack.graph.dot_graph`) now do the "right
thing" and print the possible dependency graph of provided packages.
- `spack graph --static` no longer concretizes specs, as it only relies
on class level metadata
- Previously the behavior was not consistent -- `spack graph --static`
would graph possible dependencies of concrete specs, but would only
include some of them. The new code properly pursues all possible
dependencies, and allows traversing by different dependency types.
- `spack dependencies` can now take a --deptype argument to only traverse
particular deptypes
- add a new "common" argument for deptype in spack.cmd.common.arguments
- Database.installed_relatives() can now also take a deptype argument
- this is used by `spack dependencies --installed`
- `PackageBase.possible_dependencies` now:
- accepts a deptype param that controls dependency types traversed
- returns a dict mapping possible depnames to their immediate possible
dependencies (this lets you build a graph easily)
- Add tests for PackageBaes
- The 'name' attribute for packages was being set in DirectiveMeta, which
wasn't consistent with other class properties (like fullname, etc.)
- Move it to be a class property of `PackageMeta`, and add the
corresponding property method wrapper on `PackageBase`
* add c99_flag, c11_flag to compiler class
* implement c99_flag, c11_flag for gcc
* implement c99_flag, c11_flag for arm
* implement c99_flag for cce
* implement c99_flag, c11_flag for clang
* implement c99_flag, c11_flag for intel
* implement c99_flag, c11_flag for xl
Previously, module files were not set with the same permissions as the package installation. For world-readable packages, this would not cause a problem. For group readable packages, it does:
```
packages:
mypackage:
permissions:
group: mygroup
read: group
write: group
```
In this case, the modulefile is unreadable by members of the group other than the one who installed it. Add logic to the modulefile writers to set the permissions based on the configuration in `packages.yaml`
Due to the large number of issues this migration has caused, this rolls back
to the older autotools build. However, it still keeps the updates wrt
variants and virtual packages.
* Apply patch to git to make it search for config files from its current location instead of the locations hard coded at install.
* Add provenance info for patch
* Pass needed flags to make
* Environment variables need because of relocation
* Build cache: relocate path to spack/bin/sbang in text files.
* Found in testing.
* update packaging test
* Make sbang replacement including #!/bin/bash. Add an additional spack prefix replacement to fix stage directory references.
* flake8
* Use buildinfo.get() so old buildcaches without buildinfo['spackprefix'] can be read.
* Updating the TAU package with 2.28 and public repo
* Cleaning up TAU package and fixing binutils to install extras
The extras variant will install the extra ELF headers needed
by the TAU package. The TAU package has been cleaned up and
extended.
* Adding some comments to the changes in binutils
* Adding correct python support, finding include and lib paths.
* Added PDT 3.25.1.
* Added TAU v2.28.1 with libdwarf and libelf.
* When +libdwarf is activated, +libelf is not automatically activated. It checks separately for libelf.
* Xrootd: set PYTHON_EXECUTABLE with +python variant
When building with python3 PYTHON_EXECUTABLE needs to be set for deprecated FindPythonInterp to work as expected.
* Add comment
* Flake8
* More flake8
* Update package.py
* typo
* Also add versions 0.9.9.104, 0.9.9.52, 0.9.9.0, and 'develop'
* Replace version 0.9.7 with 0.9.7.0
* Update Rose to be an Autotools package
* Update boost dependency to 1.60.0 or later (previously 1.47.0)
* Add EDG binary identifiers for ROSE versions installing from
archives (rather than git)
* Add variants for mvapich2 backend, c++11, debug symbols,
optimization, and ROSE-based projects (codethorn, autopar, and
polyopt)
* Remove patch add_spack_compiler_recognition
* Improve C++ standard computation in ACTS package
* Use a more clever strategy for aligning ACTS cxxstd with ROOT cxxstd
* Add new ACTS releases
* ACTS is not compatible with the new Boost CMake build system yet
* Explain why Boost 1.70 does not work currently
* Add LSF package, which cannot be installed by Spack and must be
system-installed. The package install will fail if no external
LSF is registered in packages.yaml (LSF may not be installed in a
well-known location and the external entry helps Spack locate it
for dependents).
* Add LSF dependency to OpenMPI when schedulers=lsf is chosen
Previously, the vtkm package was adding a "../" option to the
cmake command line. I suppose this was supposed to be pointing
to the source code, but did not. Rather, Spack correctly adds
the source directory as the first argument to cmake. However,
because ../ was added, it used that as the source directory
instead.
Simply remove this argument to make CMake work correctly.
* config:build_jobs now controls the number of parallel jobs to spawn during
builds, but cannot ever exceed the number of cores on the machine.
* The default is set to 16 or the number of available cores, whatever
is lowest.
* Updated docs to reflect the changes done to limit parallel builds
- `gettext_uuid=True` makes every commit update every .pot file in spack/localized-docs,
and speeds up the internationalized doc build slightly.
- Optimize for less repository churn, and use `python-levenshtein` to accelerate
the build instead.
- make all Spack paths relative to a `_spack_root` symlink, so that we
can easily relocate the docs build *outside* lib/spack/docs
- set some useful defaults for gettext translation variables in conf.py
- update `relativeinclude` and other references to the spack root in the
RST files to use _spack_root
- Add a `--update FILE` option to `spack list`
- Output is written to the file only if any package is newer than the file
- Simplify the code in docs/conf.py using this new option
The Spack documentation currently hard-codes some functionality in
`conf.py`, which makes the doc build less "pluggable" for things like
localized doc builds.
In particular, we unconditionally generate an index of commands and a
package list as part of the docs, but those should really only be done if
things are not up to date.
This commit does the following:
- Add `--header` option to `spack commands` so that it can do the work of
prepending text to its output.
- Add `--update FILE` option to `spack commands` that makes it generate a
new command index *only* if FILE is out of date w.r.t. commands in the
Spack source.
- Simplify code in `conf.py` to use these options and only update the
command index when needed.
Add variant 'pic' for including in shared libraries.
Add build dependency on perl. Compiling mbedtls runs several perl
scripts.
Add versions 2.7.10 and 2.16.1, the heads of the two main stable
branches.
This PR implements several refactors requested in #11373, specifically:
- Config scopes are used to handle builtin defaults, command line overrides
and package overrides (`parallel=False`)
- `Package.make_jobs` attribute has been removed; `make_jobs` remains
as a module-scope variable in the build environment.
- The use of the argument `-j` has been rationalized across commands
- move '-j'/'--jobs' argument into `spack.cmd.common.arguments`
- Add unit tests to check that setting parallel jobs works as expected
- add new test to ensure that build job setting is isolated to each build
- Fix packages that used `Package.make_jobs` (i.e. `bazel`)
* Add Fujitsu compiler to Spack.
* Fixes for flake8
* Chenges location of FCC to subdirectory called case-insensitive
* Add compiler tests for Fujitsu compiler
* Modify the logic of taking compiler version for new version of Fujitsu compiler
* When fabrics=auto or schedulers=auto, the intent is to defer to the
OpenMPI configure and let it determine and use what it finds
available on the system. The current behavior for 'with_or_without'
in the case of 'auto' explicitly disables all possible values.
This updates the logic to call 'with_or_without' only when the
value of fabrics/schedulers is not 'auto'.
* To allow explicitly disabling all fabrics/schedulers, each of these
variants has added support for 'none' (which is also the default
value).
* Add a conflict for the loadleveler scheduler for openmpi-3 and
above as it is no longer a valid configure option.
The regex used for finding the Cray OS version from the PrgEnv-cray
module was not exact and was at times pulling the version from other
PrgEnv modules. This updates the regular expression to be more exact.
* Update optional-lite and span-lite to use CMakePackage
These packages only have CMake installs on their most recent versions.
Therefore, we overload all of the CMake stages to fallback to the old
versions (no-op for most, copytree for install), and use CMakePackage on
the new versions.
* Add new string-view-lite version with CMake install support
* Only override phases for old versions of *-lite packages
* trilinos: add variant to disable chaco from seacas
keep it OFF by default due to the presence of the global symbol "divide"
that can lead to symbol clash with other libraries, for example see
https://github.com/dealii/dealii/issues/8170#issuecomment-492700787
* dealii: add conflict statement for adol-c and Trilinos SEACAS Chaco
* dyninst: rework the cmake args for version 10.1 and later
Add a separate when() case for the cmake args for Dyninst 10.1.0 and
later. The prereq args have changed enough that this makes things
easier going forward. (My previous commit 26b9369c was inaccurate and
temporary.)
Dyninst 10.1.0 accepts boost 1.70, but earlier dyninst are restricted
to <= 1.69 (not worth back porting every boost change).
Change all the dyninst versions to 3 digits for clarity and
consistency.
* Add version 10.1.0.
* added patch for intel-tbb@2019.1: with old compilers
* sparately specify when @:2018 and @2019
the reason we can't say @:2019 is that would include @2019.1 where the
patch doesn't work.
* should be possible to list when constraint as @:2019
According to my reading of ["Version ranges" in the Spack
docs](https://spack.readthedocs.io/en/latest/packaging_guide.html#version-ranges),
this should apply to up to `2019` but not `2019.1:`
* @:2019.0 appears to be necessary to specify 2019 but not 2019.1
* specifying 2019.1 2019.2 2019.3 2019.4 separately and explcitly
* flake8 fixes for long lines.
* add comments to explain why so many different patch directives
* remove blank like for flake8
* revert spec %gcc@4.8.0:4.99999 -> %gcc@4.8.0:
Adds executable=/bin/bash into Popen. We discovered this bug while
working in a csh/tsch environment. By executing with /bin/bash we ensure
that the module command works.
- CMake is built *very* frequently, and the number of dependencies it has
accumulated can take a very long time on some systems.
- Use +ownlibs by default to avoid the build overhead. This won't
interfere with other packages by introducing conflicting libraries, as
CMake is nearly always a build dependency.
This avoids using Boost 1.70.0, fixes library access in OpenSpeedShop
packages, adds new versions, and updates QT dependency constraints
in CBTF and OpenSpeedShop packages
- Constrain Boost dependency to 1.69.0 or earlier to avoid issues
with 1.70.0 for Open|SpeedShop and CBTF packages
- Update DYNINSTAPI_RT_LIB environment variable to use single library
from "find_libraries" (which returns a list) in OpenSpeedShop
packages
Fixes#11443
- Add version 1.9.3 for CBTF Argo Navis, CBTF Krell, CBTF LANL, and
CBTF packages
- Add version 2.4.1 for OpenSpeedShop Utils and OpenSpeedShop
packages
- Update QT dependency to 5.10.0 or greater for QtGraph and CBTF Argo
Navis packages
This makes several changes to make CDO work with gcc@9:
- Add version 1.9.7rc2
- Add a patch to make version 1.9.7rc2 build with gcc@9:
- Add a conflict with GCC 9 for earlier versions of CDO
It also adds a pkgconfig build dependency as configure checks for it.
#8612 added command extensions to Spack: a command implemented in a
separate directory. This improves the implementation by allowing
the command to import additional utility code stored within the
established directory structure for commands.
This also:
* Adds tests for command extensions
* Documents command extensions (including the expected directory
layout)
Rules N813 and N814 prevented import statements like this:
xml.etree.ElementTree as et
xml.etree.ElementTree as ET
But both of those seem pretty reasonable. We see no reason to require any camelcase import to be imported "as" a camelcase word.
* recon: new package at 1.08
* recon: add period
* recon: use proper decorators, better documentation
* recon: fix incorrect os import
* recon: add repeatmasker patch as variant
The source is managed as a set of files in an FTP URL. Since there
are no FetchStrategy's implemented for this setup, the files are
managed as a set of Resource's
Add dependencies to CMake and GnuTLS which would otherwise be
picked up from the system
* Add gnutls as explicit dependency of CMake
* Add new package libidn2 as explicit dependency for gnutls
- `svn info` prints different results depending on the system locale
- in particular, Japanese output doesn't contain "Revision:"
- Change Spack code to use XML output instead of using the human output
* LBANN: add python dependency and required python library dependencies
* LBANN: add version 0.99 as alias for develop branch
* LBANN: build Conduit library by default
* LBANN: CUB is included with Hydrogen for later versions
* New package: py-graphviz (dependency for LBANN)
* New package: py-texttable (dependency for LBANN)
* New package: py-onnx (dependency for LBANN)
* New package: py-typing-extensions (dependency for py-onnx)
* Conduit: add version 0.4.0
* NCCL: Add versions 2.4.6-1 and 2.4.2-1
* Protobuf: Add versions 3.7.1 and 3.7.0
* py-protobuf: add version 3.7.1
* py-typing: add version 3.6.4
* cuDNN: add versions. The source differs depending on the
architecture, so the versions include a suffix to encode this.
For example this adds 7.5.1-10.1-x86_64 as well as
7.5.1-10.1-ppc64le
* cuDNN: Newer versions require newer versions of CUDA
* Hydrogen: add configure options for CUB and Aluminum
The --enable-relax-coord-bound option was added in 1.8.0. NetCDF
4.6.3 enabled relax coords by default and required it from PNetCDF.
PNetCDF 1.10.0 enabled it by default but for versions between 1.8.0
and 1.10.0 it should be enabled explicitly.
* megadock: new package at 4.0.3
* megadock: remove CudaPackage redundancies
* megadock: move env vars to build targets, use cuda_arch variant
* megadock: flake8 fixes, more flexible mathlib
* Fix py-yt for Python 2 by requesting older py-ipython version.
* Add conflicts statement to py-ipython because version 7: is not
compatible with python@2.7
f242f5f removed spec.architecture.platform_os, which was used to test
an option in the 'm4' package. The attribute is now accessed as
'spec.architecture.os'.
* Add versions 3.27.{0,1,2} and 3.28.0
* Add url_for_version: the URL format includes the year so this
function maps version ranges to years
* Restore patches for versions removed in 84c547c: this commit
removed versions before 3.26.0 due to a security issue with FTS
support. Add a +fts variant (default true) that enables FTS
support along with a conflict for +fts with earlier versions.
- Codecov cannot handle as many coverage reports as we are generating
- as a result, our PR coverage pages have been broken for a while, and
it's hard to tell people where to enhance their testing in PR reviews.
- Scale back to only running coverage for 3.7 and 2.7 unit tests
- This is *probably* better. We run the build tests for good measure,
but we do not need to evaluate them for coverage. The coverage reports
are about unit tests.
Add fixes to support multiple installs and dependents using a subset
of IntelPackage functionality.
* Update IntelPackage to only return scalapack libraries if the root
spec depends on MPI: scalapack requires MPI to be mentioned as a
dependency in the DAG. Package builds using intel-mkl for its
blas/lapack implementations but not for scalapack were failing to
build.
Ideally it would be possible to ask if any of the packages in the
DAG are actually requesting the scalapack functionality provided by
the IntelPackage and only return scalapack libs in that case, but
that is not easily done at this time.
Fixes#11314Fixes#11289
* set HOME when the intel silent installer is run. This prevents the
installer from using the ~/intel directory (which can cause
conflicts for multiple installs of the same IntelPackage)
Fixes#9713
Use new `module` function instead of `get_module_cmd`
Previously, Spack relied on either examining the bash `module()` function or using the `which` command to find the underlying executable for modules. More complicated module systems do not allow for the sort of simple analysis we were doing (see #6451).
Spack now uses the `module` function directly and copies environment changes from the resulting subprocess back into Spack. This should provide a future-proof implementation for changes to the logic underlying the module system on various HPC systems.
* Starting with the upcoming 10.1 release, Dyninst changed the names of
some cmake args to camel case: LIBELF_* to LibElf_* and LIBDWARF_* to
LibDwarf_*. This change is needed now for @develop and will be used
for 10.1 when it is released.
* New versions of Dyninst now requires cmake version 3.4.0 or later.
* Restrict boost dependency version to < 1.70.0 to avoid build error
+ The additional patch enables a non-optimized random123 code path that will
work on ARM architectures. This solution is provided only for version 1.09 to
allow the most current version of Random123 to function on ARM architectures.
+ A more complete, long term solution will be the addition of a native ARM
intrinsics-based implementation of random123. This solution is being worked
on but it is several weeks or months away. Once the full implementation is
available it will be provided to the author/maintainer of Random123.
The default install for llvm should just be the common typical case, i.e.
support for local host and cpu architectures. Enablingsupport for the wide
array of auxiliary architectures should be explicit rather than implicit.
Add two functions to the EnvironmentModifications object to help
users sanitize environment variables in their package definitions:
* deprioritize_system_paths: this keeps system paths in the
environment variable but moves them to the end.
* prune_duplicate_paths: remove any duplicate paths from the
variable
This includes testing for the new functions as well as for
(previously-untested) old convenience functions for environment
variable manipulation.
This also adds special handling for bash functions so they
will be defined when the exported environment file is sourced.
* Mesa should depend_on('glproto')
The mesa package refers to `GL/glproto.h`. On systems that don't have
the OS packages installed, this leads to failures during the build
[e.g. this comment in
01482](https://github.com/spack/spack/pull/10482#issuecomment-488786745).
This fixes it. Tested on a minimally provisioned CentOS 7.
* Constrain glproto prereq to when +glx
* mesa: make glproto a build only dep
* The 'mkl' variant implementation was not complete and was
preventing the package from building, so remove it.
* The slate.hh file has moved in the 'develop' version (which is
currently the only version of SLATE listed)
* FAODEL: Add FAODEL package to spack
FAODEL (Flexible, Asynchronous, Object Data-Exchange Libraries) is a collection
of software libraries that are used to implement different data management
services on high-performance computing (HPC) platforms. This project is part of
the Advanced Technology Development and Mitigation (ATDM) effort for NNSA's ASC
program at Sandia National Laboratories.
(cherry picked from commit 6afe045d25)
* FAODEL: Update FAODEL package
Dependencies: set minimum versions for boost, cmake and libfabric
Dependencies: add conflict for boost v1.59.0
Patch: add a patch for v1.1803.1
Patch: add a when= for the v1.1811.1 specific patches
Compiler: check for C++11 support in gcc
(cherry picked from commit fb6ed2b3cf20d7e4fff647cba5a70887a78f627f)
* FAODEL: Update FAODEL package
* replace previous maintainers with @tkordenbrock and @craigulmer
* default to shared libs to meet Spack policy
* increase min boost version to 1.60.0 and remove boost 1.59.0 conflict
* replace complex version check with conflict for gcc <= 4.8.0 (C++11 support required)
* set type=build for cmake
* enable googletest for all versions with type=build
* enable tests for for all versions except when MPI is disabled
* FAODEL: Update FAODEL package
* add version v1.1811.2
* limit comment line length
Fixes#11335
Update the Spack compiler wrappers to add the headerpad_max_install_names
linker flag on MacOS. This allows the install_name_tool to rewrite
the RPATH entry of the binary to be longer if needed. This is
primarily useful for creating and distributing binary caches of
packages (i.e. using the "spack buildcache" command); binary caches
created on MacOS before this commit may not successfully relocate
(if the target root path is larger).
* Added a function that concretizes specs together
* Specs concretized together are copied instead of being referenced
This makes the specs different objects and removes any reference to the
fake root package that is needed currently for concretization.
* Factored creating a repository for concretization into its own function
* Added a test on overlapping dependencies
* arm-forge: add all available versions
* Add TODO for the version hell
* Fix linting
Looks like 'black' formatter isn't Spack friendly
* Fix versions ordering
* Remove obsolete package (allinea-forge → arm-forge)
* Set mpi compiler wrappers using CC/etc. variables as recommended
at https://spack.readthedocs.io/en/latest/packaging_guide.html#compiler-wrappers
rather than setting CMake variables like -DCMAKE_C_COMPILER
* Use convenience functionality provided by inherited CMakePackage to
export all compiler flags as CMake options
* Remove old Singularity versions, add version 3.1.1
* Recent versions of Singularity build with go rather than autotools:
update dependencies and build logic
* Move old Singularity versions to new singularity-legacy package
which uses the autotools build system
* Some binaries built by Singularity need to be run as root: include
a script that the user can run after the Spack Singularity install
to change these permissions.
* The Singularity go build expects to work with a custom stage
directory relative to GOPATH: override the stage phase to create
this expected path.
* Update Singularity install config to point to Spack-installed
makesquashfs dependency
Based on the LLVM documentation [1], Python is used to run the automated
test suite. Therefore is it always a dependency for LLVM. However, if
build without Python (~python), we limit it to a build time dependency.
Note that py-lit is not added as a spack dependency even though it is
available as a spack package. This is because it is already included
in llvm and llvm is difficult to configure using an external py-lit
(several CMake variables to set correctly). Additionally, having
py-lit as a spack dependency adds Python as a runtime dependency
for llvm even though it is not required at runtime.
[1] https://llvm.org/docs/GettingStarted.html#requirements
* extend Version class so that 2.0 > 1.develop > 1.1
* add concretization tests, with preferences and preferred version.
* add master, head, trunk as develop-like versions, develop > master > head > trunk
* update documentation on version comparison
* Version 1.0.0 was released and the master branch tracks the
latest release.
* Multiple issues with MacOS have been fixed. The static patch is
now unneeded.
* Add support for building Ginkgo with the full Block Jacobi
performance (compilation takes a long time).
* Do not build anything which is not useful for the installation.
Namely, documentation, benchmarks, examples and tests.
* mesa: re-writen to new meson build system
* xorg-server: Remove unneeded mesa dependency
* glx: provide the virtual GLX package
* Convert most "mesa" dependencies to use virtual gl and glx dependencies
* mesa: Switch to always building from the git repo instead of a tarball
* Use gl and glx version ranges instead of exact versions
* glx: provide a default package
* add auto to swr variant
* Revert "add auto to swr variant"
This reverts commit 2e7cd8a8ac.
* update mesa package to version 19.0
* set +glx by default only when running on linux
* fix style issues in geant4 package
* remove virtual package "egl"
margo package is a required dependency for unifycr but the package
was not available in spack. margo requires a recent version of argobots
that has the ABT_SCHED_BASIC_WAIT scheduler (rc1 does while previously
available versions of this package in spack did not have it).
Kahip does not build with clang (conflicts('%clang') in kahip package.
Because of this, openfoam-com %clang fails concretization. Therefore,
this changes the default openfoam-com variant for kahip to False.
* Remove -ffast-math optimization flag
* Add 'develop' version which points to git repo
* Depend on libint for all versions >= 3.0
* Build directory is different for versions >= 7 (including develop)
* Add "make test" as post install check
* Add versions 3.4.4 through 4.1.0 and 4.1.0-openvino
* Add conflict for gcc version >= 7 when building with +cuda
* Later version of protobuf is required for later versions of OpenCV
* Explicitly remove protobuf dependency with CMake option when not
building with +dnn
* root: new versions, variants and patches to bring in to line with FNAL's use.
* Support MySQL.
* Patches:
* Bigger stringbuf.
* Find MySQL even when it's called mariadb-c-client.
* Honor external provision of unuran.
* Properly handle the ROOT7 web GUI for 6.16/00.
* `memstat` variant is now default-OFF due to version-related constraints.
* Add missing dependencies.
* Propagate `python` to `libxml2`.
* Handle external FTGL dependency with X / OpenGL: depends on PR ####
* Force `SPACK_INCLUDE_DIRS` to avoid warning.
* New `gminimal` variant default ON per discussion with Javier Cervantes.
* Style fixes per flake8.
* Remove redundant `PATH_SUFFIX` per https://github.com/root-project/root/pull/1993#discussion_r276840329
* Comment patches per https://github.com/spack/spack/pull/11215#discussion_r276887199.
* Reflect the fact that MariaDB support patch is now upstream.
* Restore ncurses dependency for libtinfo.
Usage of double quotes was preventing word-splitting when parsing
module roots in setup-env.sh, which lead to an error when multiple
module roots are used (in particular when Spack is pointed to use
an upstream module root in addition to its own).
Add zlib dependency to squashfs
On my CentOS system it was getting linked in (from Spack, no less)
fortuitously.
On a minimal Ubuntu system it failed until it was added as an explicit
dependency.
* ftgl: update to 2.1.3-rc5.
This updates to the last release (from 2008) with patches required to work with Spack and Root.
* Working 2.1.2 with/without docs, 2.1.3-rc5 without.
- `spack edit` previously used `spack.util.executable` `Executable` objects,
and didn't `exec` the editor like you'd expect it to
- This meant that Spack was still running while your editor was, and
stdout/stdin were being set up in weird ways
- e.g. on macOS, if you call `spack edit` with `EDITOR` set to the
builtin `emacs` command, then type `Ctrl-g`, the whole thing dies with
a `==> Error: Keyboard interrupt`
- Fix all this by changing spack.util.editor to use `os.execv` instead of
Spack's `Executable` object
Allow specifying whether AVX2 or AVX512 should be enabled or disabled
on x86 targets.
As AVX2 hardware and toolchain support is quite ubiquitous by now,
AVX2 is enabled by default. Also AVX2 support is not disabled when
building the +virtual_machine variant.
AVX512 is not supported in older but still supported toolchains,
hardware is still expensive, and OpenBLAS AVX512 kernels still have
bugs. Thus AVX512 is disabled by default.
* Add Mercury version 1.0.1
* Remove cci and bmi dependency requirements: these are not currently
defined in Spack. If +bmi is enabled, the effect is that it may
use a system install of bmi
* Add a note that fabtests is deprecated and has been folded into the
libfabric package. Add the fabtests resource to the libfabric
package
* Add CMake option settings to Mercury based on variant settings
* Mercury: Set OFI testing protocol depending on how libfabric was
built
* Mercury: add version constraints to dependencies
Also add constructor to NoLibrariesError which can either take an
error message (like other SpackErrors) or a name and prefix (in
which case the error message is constructed).
PR #10758 made a slight change to find_versions_of_archive() which included
archive_url in the search process. While this fixed `spack create` and
`spack checksum` missing command-line arguments, it caused `spack
install` to prefer those URLs over those it found in the scrape process.
As a result, the package url was treated as a list_url causing all R
packages to stop fetching once the package was updated on CRAN.
This patch is more selective about including the archive_url in the
remote versions, explicitly overriding it with matching versions found
by the scraper.
Fixes#11226
MySQL 5.7.X and Boost 1.59.0 have a closer relationship than
MySQL >= 8.0 and their corresponding Boost versions: 5.7.X needs to
link against boost libraries which is indicated by marking boost as
a link dependency (removing the 'type' reverts to the default, which
includes link and build).
Starting with version 7.64.0, curl skips cookies specified with "-H"
(generic custom header specification) when following redirects, which
causes the "accept-license"-cookie for jdk to disappear. The result
is that jdk fails to download. This uses the "-b" option (which is
used specifically for configuring cookies) instead, which is more
permissive.
f242f5f8 changed the format strings but maintained backwards
compatibility in all cases except one: The list of valid tokens for
the module naming schemes was not updated properly to contain both
the new and old styles for compilers and package names.
This PR re-adds the old tokens into the list of valid tokens.
spec['dep'].libs.ld_flags can return system paths in some cases.
This is a targeted fix for the petsc package when retrieving
suite-sparse dependency libraries.
* added latest versions + malloc variant
* added a variant to activate different instrumentations
* added a variant to activate or not networking
* added a variant to activate or not cuda and tools
* enforce c++ standard and make it consistent with boost
#11152 added documentation for #8772 but some details were based on
an earlier implementation that had changed by the time #8772 was
merged. In particular, #11152 mentioned that upstream Spack instances
were configured in config.yaml, when in fact they should be placed in
a separate upstreams.yaml config file; this PR updates the
documentation accordingly.
fixes#11159
The 'namespace' argument to both Repo and RepoPath were used to set the
"super namespace". Currently it seems to be vestigial as the only
"super namespace" allowed for packages is 'spack.pkg' since 39c9bbf
* cflags and cxxflags from packages.yaml need to be passed into QMCPACK's CMake explictly for now.
* You need the ifcore library from the Intel compler to be manually linked in when you linked against a LAPACK provider that is not MKL.
* Make a separate CDash report for each package installed
Previously, we generated a single CDash report ("build") for the complete results
of running a `spack install` command. Now we create a separate CDash build for
each package that was installed.
This commit also changes some of the tests related to CDash reporting.
Now only one of the tests exercises the code path of uploading to a
(nonexistent) CDash server. The rest of the related tests write their reports
to disk without trying to upload them.
* Don't report errors to CDash for successful packages
Convert errors detected by our log scraper into warnings when the package
being installed reports that it was successful.
* Report a maximum of 50 errors/warnings to CDash
This is in line with what CTest does. The idea is that if you have more than
50 errors/warnings you probably aren't going to read through them all anyway.
This change reduces the amount of data that we need to transfer and store.
* Update spec format to simpler syntax, maintain backwards compatibility
* Switch to new spec.format method throughout internals
* update package files for new format strings
* documentation and minor code cleanup. removed nonsensical variant sigils
* Initial commit for v2.0 of the CEED software suite.
* Update Nek packages and gslib
* Help spack concretize the hypre version for ceed-2.0.
* Fix nekcem install error
* Add support for gfortran v8 in nek5000 and nekcem.
* Split Nek5000 into Nek5000 and Nektools
* Get Nektools to build fine in Theta
* Fix travis failure: remove unused 'import numbers' from nek5000.
* Check for gfortran if it is wrapped
* Tweak the detection of gfortran in nek5000.
* Fix Nek packages to add -std=legacy when FC=gcc
* spack install ceed~petsc works fine on Theta
* Fix flake8 errors
* Fix more flake8 tests
* Fix an import issue
* Tweak the suite-sparse package to avoid interaction with existing system
installations of suite-sparse.
* petsc: update superlu-dist dependency
* Updates in the packages: occa, libceed, and ceed.
* In the libceed package, explicitly tell nvcc which host compiler to use.
* Fix python formatting.
* Simplify the test for gfortran in nek* packages.
* ceed: 2.0 uses petsc@3.11.0
* hpgmg-0.4; use from ceed@2.0.0
* Update the hypre dependency for ceed 2.0.
* Disable the superlu-dist dependency (through hypre) when using a
+quickbuild of ceed 2.0.
* petsc-3.11.0: add xlf fix
* nekcem: has a build dependency on Python 2.7+
* hpgmg: better setting of compiler options and use python for configure
* libceed: use v0.4 tag
* libceed: fix 0.4 release oops (pkgconfig version)
* Add a patch for magma-2.5.0 that brings it up the current 'master'.
* In the mfem package, install the examples, miniapps, and data under
$prefix/share/mfem.
* In the magma package, apply a patch to v2.5.0 that disables
magma_sparse - for testing purposes.
* In the magma package, link the 'magma' library with the
'nvToolsExt' library.
* In the magma package, update the 'magma-2.5.0.patch' with the latest
commits from the magma source repository. Also, remove the library
'nvToolsExt' from the 'magma-2.5.0-cmake.patch' - now it is not
needed.
* In the magma package, disable OpenMP when using v2.5.0 with the
IBM XL compiler.
Please enter the commit message for your changes. Lines starting
* In the mfem package, add version for the 'laghos-v2.0' tag; also,
prefix the versions `laghos-v*` with their respective development
version numbers -- this way they are properly ordered within spack
relative to the official numbered versions.
* petsc: add version 3.11.1 (#11179)
(cherry picked from commit 1eab6e3c86)
* ceed-2.0: use petsc-3.11.1
* this-is-so-dumb.f -> empty.f
Fixes#11070#11010
Spack attempts to intercede on behalf of all compiler invocations for
a build. This involves adding its wrappers to PATH. Cray systems
include a "ftn" executable and Spack was only redirecting this call
when the Spec was built with cce. This updates the compiler wrappers
to add "ftn" in all cases.
The default (implied) behavior for all environments, as of ea1de6b,
is that an environment will maintain a view in a location of its
choosing. ea1de6b explicitly recorded all three possible states of
maintaining a view:
1. Maintain a view, and let the environment decide where to put it
(default)
2. Maintain a view, and let the user decide
3. Don't maintain a view
This commit updates the config writer so that for case [1], nothing
will be written to the config.yaml. This will not change any existing
behavior, it just serves to keep the config more compact.
* Add versions 1.4.0, 1.3.0, and 1.2.0
* Add dependencies and update constraints (e.g. version 1.4.0 of
precice requires a later version of boost)
* Add maintainers
Still look for BASH_SOURCE[0] first, but if it's not set,
_sp_source_file is initialized to an empty value addressing the
unset parameter error (line 217).
A Spack hack for MKL usage was needed in QMCPACK prior to version 3.5
when non-Intel compilers were used. This Spack hack could have
undesirable side-effects and led to ugly code that was extra work to
maintain. For older versions of QMCPACK, we throw a conflict if you
want to use Intel MKL with the other compilers commonly found on x86.
Note that there is no impact to version of QMCPACK since 3.5.
Compilers are treated separately from other dependencies in Spack.
#10761 added the option to automatically install compilers when a
package specifies using a compiler that is not available in Spack.
However, this did not work correctly for dependency packages (it
would only build a compiler for the root of an install DAG). This
commit enables the building of compilers for dependency packages.
Environments are nowm by default, created with views. When activated, if an environment includes a view, this view will be added to `PATH`, `CPATH`, and other shell variables to expose the Spack environment in the user's shell.
Example:
```
spack env create e1 #by default this will maintain a view in the directory Spack maintains for the env
spack env create e1 --with-view=/abs/path/to/anywhere
spack env create e1 --without-view
```
The `spack.yaml` manifest file now looks like this:
```
spack:
specs:
- python
view: true #or false, or a string
```
These commands can be used to control the view configuration for the active environment, without hand-editing the `spack.yaml` file:
```
spack env view enable
spack env view envable /abs/path/to/anywhere
spack env view disable
```
Views are automatically updated when specs are installed to an environment. A view only maintains one copy of any package. An environment may refer to a package multiple times, in particular if it appears as a dependency. This PR establishes a prioritization for which environment specs are added to views: a spec has higher priority if it was concretized first. This does not necessarily exactly match the order in which specs were added, for example, given `X->Z` and `Y->Z'`:
```
spack env activate e1
spack add X
spack install Y # immediately concretizes and installs Y and Z'
spack install # concretizes X and Z
```
In this case `Z'` will be favored over `Z`.
Specs in the environment must be concrete and installed to be added to the view, so there is another minor ordering effect: by default the view maintained for the environment ignores file conflicts between packages. If packages are not installed in order, and there are file conflicts, then the version chosen depends on the order.
Both ordering issues are avoided if `spack install`/`spack add` and `spack install <spec>` are not mixed.
* Fix bug when system perl is used.
If system perl is being used through packages.yaml there cannot be dependent modules.
The setup_dependent_packages function should not do anything in that case.
When providing a track, the cdash reporter will format the stamp
itself, as it has always done, and register the build during the
package installation process. When providing a stamp, it should
first be formatted as cdash expects, and then cdash will be sure
to report results to same build id which was registered manually
elsewhere.
* Update Spec.prefix to have special case for 'None' in database path; regression test
* Update in database reader rather than spec
* Change assertion to conditional + raise
* Added test for concrete check in Spec.prefix
The module_parsing test checks whether the module function is available
by looking for the string 'not found'. If the user has set a different
locale, the test can assume that the module function is available when
it actually is not.
* Add versions 5.2.0 and 5.1.0
* Avoid adding version 6.0.0: this version has a new API that is
not compatible with older versions (and dependents may break with
it)
* Added comment block explaining the problems with version 6.x
* Replace old datumgrid resource (1.7) with new (1.8)
* Add speculative sqlite dependency for when a 6.x version is added
* add version 7.0.1
* replace md5 with sha256 for various versions.
* v7.0.1 fixes some environment hacking to properly find MPI and
system architecture
This adds a stub script for mpirun and other standard executables
when installing OpenMPI with slurm. The purpose is to make the
removal less of a surprise to administrators/users: it explains why
they were removed and how to restore them.
* Split get_compiler_version into two functions:
get_compiler_version_output runs the compiler with the relevant
option to print the version; extract_version_from_output determines
the version by examining this output. This makes it easier to test
the customized version detection for each compiler. Users can
customize this by overriding the following:
* version_argument: this is the argument that tells the compiler to
print its version. It assumes that the compiler will report its
version if invoked with a single option (like "--version")
* version_regex: the regular expression used to extract the version
from the compiler argument. This assumes that a regular
expression is sufficient to extract the version, and that the
version can be extracted from a single capture group (Spack uses
the first capture group)
* default_version: allows you to completely override all version
detection logic
* get_compiler_version_output: if getting the compiler to report
its version is more complex than invoking it with a single arg
* extract_version_from_output: if it is difficult to define a regex
that can be used to extract the version from the output
* Added tests for version detection of most compilers
* Removed redundant code from xl_r compiler class (by inheriting
from xl compiler definition)
* adding package to install squashfs for singularity
* dont need os import
Signed-off-by: Vanessa Sochat <vsochat@stanford.edu>
* updating md5 to sha1 for squashfs package
Signed-off-by: Vanessa Sochat <vsochat@stanford.edu>
* two blank lines
Signed-off-by: Vanessa Sochat <vsochat@stanford.edu>
Replace the original implementation of the "memoized" decorator with
an implementation that exposes the docstring and arguments of the
wrapped function. This is achieved using functools.wraps.
This provides a mechanism to implement a new Spack command in a
separate directory, and with a small configuration change point Spack
to the new command.
To register the command, the directory must be added to the
"extensions" section of config.yaml. The command directory name must
have the prefix "spack-", and have the following layout:
spack-X/
pytest.ini #optional, for testing
X/
cmd/
name-of-command1.py
name-of-command2.py
...
tests/ #optional
conftest.py
test_name-of-command1.py
templates/ #optional jinja templates, if needed
And in config.yaml:
config:
extensions:
- /path/to/spack-X
If the extension includes tests, you can run them via spack by adding
the --extension option, like "spack test --extension=X"
* Newer versions of IBM XL no longer define `__xlC__`, but define `__ibmxl__`
instead. This one-line patch fixes this problem in Random123.
* This patch was also provided to the Random123 maintiners
(Random123@DEShawResearch.com). I don't expect a new release from them for a
while, so I believe that this patch is the correct path forward (for now).
* initial work to make use of an 'upstream' spack installation: this uses the DB of the upstream installation to check if a package is installed
* need to query upstream dbs when adding new record to local db
* prevent reindexing upstream DBs
* set prefix on specs read from DB based on path stored in install record
* check that Spack does not install packages that are recorded as installed in an upstream db
* externals do not add their path to install records - need to use 'external_path' to get path of upstream externals
* views need to check for upstream installations when linking metadata
* package and spec now calculate upstream installation properties on-demand themselves rather than depending on concretization to set these properties up-front. The added tests for upstream installations don't work with this new strategy so they need to be updated
* only refresh modules for local specs (not those in upstream packages); optionally generate local module files for packages installed upstream
* when a user tries to locate a module file for a package installed upstream, tell them to use the upstream spack instance to locate it
* support recursive upstream databases (allow upstream databases to use their own upstream databases)
* separate upstream config into separate file with its own schema; each entry now also includes a name
* metadata_dir is no longer customizable on a per-instance basis for YamlDirectoryLayout
* treat metadata_dir as an instance variable but dont set it from kwargs; this follows several other hardcoded variables which must be consistent between upstream and downstream DBs. Also update DirectoryLayout.metadata_path to work entirely with Spec.prefix, since Spec.prefix is set from the DB when available (so metadata_path was duplicating that logic)
* Add latest libyogrt versions
* Add url_for_version to handle differing url formats for different
versions
* Add 'scheduler' variant to configure scheduler. Currently only
slurm can be installed with Spack. A value of 'system' indicates
to use the scheduler installed on the system
Change the location of the CMake build area from the staged source
directory to the stage base directory.
This change allows CMake packages to refer to the build directory in
setup_environment (e.g. if tests need to have a directory in PATH):
Staging happens after the call to setup_environment(), and if the
stage area does not exist, then spec.stage.source_path returns None.
To accommodate this change, archived files (like config.log for
Autotools packages) are archived relative to the stage base directory
rather than the expanded source directory.
Other packages (those not using CMake) will still use the staged
source directory as the default working directory for builds (and
will still be unable to reference this directory in
setup_environment())
When multiple instances of environment-modules were installed with
different architectures, Spack was not retrieving the installation
appropriate for the current architecture when finding the module
prefix.
* py-prompt-toolkit: Add newer version for required for newer
py-ipython versions
* py-ipykernel:
* Add newer py-ipykernel version
* Depend on py-setuptools so a python egg isn't attempted to be
built
* Update dependency for newer py-ipykernel versions
* py-jupyter-console: Remove py-prompt-toolkit dependency since it is
picked up in py-ipython
* py-ipython:
* Add missing py-backcall dependency
* Adjust py-prompt-toolkit dependencies for newer versions of
ipython
* py-jupyter-notebook: Require newer version of py-ipykernel since
jupyter is broken with previous versions
* add cxxstd variant
* add CMake constraints based on platform/version
* add older versions
* update boost dependency version constraints (which are closely
tied to mysql version) and update boost cxxstd choice to be
the same as the cxxstd chosen for mysql
* add client-only support (including a patch for 5.5.x)
* record the mysql package as a provider of the mysql-client virtual
Improve management of the Fiber library and C++ standard support:
* Remove Fiber from list of libraries to build
* Improve variant management for Fiber; add variants for Context and
Coroutine libraries.
* Add known conflict with C++17 for boost < 1.63.0
* Remove C++ standard "default" option, which left the choice of
C++ standard to the compiler used to build boost
* Fixed some issues with CUDA-Intel compiler conflicts.
* Comment about expressing CUDA-compiler conflicts.
* More precise conflicts and also add support for Intel 19.0
* record conflicts with compiler versions which don't provide
required c++ standard support
* add doxygen (build) dependency
* add note that range-v3 is header-only as of 0.3.6 and update
package description
We add new variants to handle readline vs libedit, a client-only
build and install, and bindings to TCL, Python and Perl. We also add
new versions and the ability to detect remote versions not otherwise
dealt with.
This avoids using a system-installed CUDA package. In the future a
variant can be added to allow using Spack-installed CUDA, but for
now CUDA support is always disabled.
* abinit: Fix building with hdf5/netcdf.
* gromacs: Fix attempt to build with cuda support when 'cuda=False'
If for some reason there's a cuda toolkit installed by other means,
(i.e. not by spack) cmake will still try to build with cuda support,
even though 'cuda=False' is the default of the spec.
* Revert "abinit: Fix building with hdf5/netcdf."
This reverts commit e16f725e37.
This should not be here.
* Draco: add variants
+ This package has many optional build dependencies that were not registered in
older versions of this recipe. I've added (and tested) this more complete
list of optional dependencies: parmetis, superlu-dist, qt.
* fix style issues
* hpcviewer: new package
Add binary package for hpcviewer and hpctraceviewer for the Rice
hpctoolkit on Linux x86_64, ppc64 and ppc64le.
* ibm-java: add property 'home' so that spec['java'].home will work.
* Flake
* More flake.
* Test that the version, machine type pair exists before using it in the
sha dictionaries so that 'spack info' doesn't crash on unsupported
configurations.
* mariadb-c-client is a new package using the distinct, LGPL,
MySQL-compatible client library from mariadb.com. It provides the
virtual package mariadb-client
* mariadb is recorded as a provider of the mariadb-client virtual
* The mysql-client virtual package is also added, and mariadb-c-client
is recorded as a provider for it
If the user has set the environment variable VISUAL, it will be used
in preference to EDITOR for all Spack editing activities. If VISUAL
is not set or fails (perhaps due to a lack of graphical editing
capabilities),EDITOR will be used instead. We fall back to one of
several common editors if neither bears fruit.
This feature has been tailored to:
* Provide identical behavior to the previous implementation in the
case that VISUAL is not set.
* Not require any change to code utilizing the editor feature.
* Follow usual UNIX behavior concerning VISUAL and EDITOR.
* Throw InstallError if more than one GPU architecture is passed to cuda_arch. Previous cuda_arch test was not actually working because comparison with none string was on the cuda_arch list instead of the first entry of the list.
* Removing redundant cuda_arch statement.
* New package: py-mysql-connector
* Fixed docstring
* 1. Determined that py-setuptools was not needed at all, so removed.
2. Added py-protobuf. Docs seem to imply that only C protobuf library is required; however, the Python setup.py says differently, and some Python code seems to reference protobuf too. I don't know why this worked for me, but it looks like including py-protobuf is the right thing to do.
* Applied solution detailed in:
https://github.com/mysql/mysql-connector-python/pull/9
Uncommenting this patch will make `error: option --single-version-externally-managed not recognized` reappear.
* Clean up / reorder lines
* flake8
* Fix clearing EnvironmentModifications with python2
* Add EnvironmentModifications::clear unit test
Use re-assignment rather than del to clear array
* Fix flake issues
qt currently falls back to bundled versions of sqlite, harfbuzz, pcre,
double-conversion and xcb. This adds the appropriate dependencies and
configure arguments. A new variant adds multibyte support to pcre and
pcre2, which is required by qt.
Additionally, newer versions of gcc (starting with @8.3.0) cause build
failures. This adds a patch to fix the problem.
The changes have been tested with all versions of qt currently available
in Spack. 5.2 and 5.3 do not build for reasons that seem to be unrelated
to these changes, though.
* Add binary package of the IBM Java SDK for big and little-endian
powerpc (power7, 8 and 9). The jdk and openjdk packages only install
on x86_64.
* Add ibm-java as a java provider
* The jdk and openjdk packages only install on x86_64. Add conflicts
for ppc64 and ppc64le to jdk and openjdk.
shmemrun and oshrun do not exist in OpenMPI v4.0.0
(ref: https://www.open-mpi.org/doc/v4.0/)
The Spack OpenMPI package was failing the install by trying to
remove them. This guards the removal of several scripts when
using the Slurm scheduler to handle the case where they don't exist.
It seems that this is actually a glibc problem and while 2.6.4 builds
without the patch on newer versions of gcc (@8:), it still sometimes
segfaults (as observed during the doxygen build).
Fixes#10191
* Add more regular expressions to detect clang versions that were
not being picked up
* Add a test for parsing versions from the output of Clang (this
does not run Clang, but rather uses example outputs from Clang)
* Separate Clang version parsing into its own method (to make it
easier to test)
Currently, only C headers are considered, causing build failures for
packages depending on, e.g., netcdf-fortran and xerces-c. Additionally,
the regex used to look for the include path component did not consider
word boundaries, causing false matches.
* Create option to build missing compilers and add them to config before installing packages that use them
* Clean up kwarg passing for do_install, put compiler bootstrapping in separate method
* Add 'fiber' as a default library for boost
* Add autoconf/automake etc. dependencies to libseccomp package
* New package: brotli
* New package: editline
* Add brotli, editline, boost dependencies to Nix
Remove 2.6.3 as preferred version (but keep it available for
building). The latest version (currently 2.6.4) is now preferred
(according to Spack's defaults).
* Update dependencies for py-flake8 when building version 3.7.7
* Add FIXME comment for an example dependency constraint which causes
concretization to hang
* Add py-entrypoints version 0.3
* Add py-pycodestyle version 2.5.0
* Add libuv version 1.10.0
* CMake versions before 3.12.0 do not build with libuv version
1.25.0, so a constraint is added to build earlier versions of
CMake with libuv version 1.10.x
Update CPATH in setup_environment for Eigen, so that the
Spack-generated module for Eigen will help builds outside of Spack
use the appropriate include prefix for Eigen headers
(<install_prefix>/include/eigen3/ rather than <install_prefix>/include/)
Note that this only updates the run-time environment, rather than the
build-time environment, so Spack builds depending on Eigen that use
pkgconfig will not be confused by the presence of the Eigen include
directory in CPATH.
* Replace kim-api package with kim-api-v2, which has different
versions and removes the 'cmake_args' method
* Add openkim-models-v2 as an extension package
* libelf: move url to fossies.org
Libelf is no longer maintained and the original mr511.de web site no
longer exists. The final release, 0.8.13 from Nov 2009 is still
archived at fossies.org.
Fixes#10757.
* Change the homepage to the FSF entry.
New release of UnifyCR. Remove unsupported pre-1.0 versions
UnifyCR has had a few dependency changes to make the backend more
efficient. A new pre-release version has been released as well to
mark these changes. Prior versions are no longer supported.
Fixes#10769
This updates the .headers property to include header subdirectories
for Python and Eigen (as is recommended by these packages).
#10623 updated the default behavior of .headers.directories to
exclude subdirectories (since this can cause clashes with system
headers). This broke some packages which depended on the old behavior
of .headers.directories: for example if you had
<package-prefix>/include/subdir/ex1.h, .headers.directories would
include <package-prefix>/include/subdir.
* Rework of buildcache creation and install prefix checking using the functions introduced in
https://github.com/spack/spack/pull/9199
Instead of replacing rpaths with placeholder and then checking strings, make use of the functions
relocate.is_recocatable and relocate.is_file_relocatable to decide if a package needs the allow-root option.
This fixes a problem where the placeholder path was not in the first rpath entry. This was seen in c++ libraries and binaries because the compiler was outside the spack install base path and always appears first in the rpath.
Instead of checking the first rpath entry, all rpaths have the placeholder path and the old install path (if it exists) replaced with the new install path.
* flake8
* Added the `spack buildcache preview` sub-command
This is similar to `spack spec -I` but highlights which nodes in a DAG
are relocatable and which are not.
spec.tree has been generalized a little to accept a status function,
instead of always showing the install status
The current implementation works only for ELF, and needs to be
generalized to other platforms.
* Added a test to check if an executable is relocatable or not
This test requires a few commands to be present in the environment.
Currently it will run only under python 3.7 (which uses Xenial instead
of Trusty).
* Added tests for the 'buildcache preview' command.
* Fixed codebase after rebase
* Fixed the list of apt addons for Python 3.7 in travis.yaml
* Only check ELF executables and shared libraries. Skip checking virtual or external packages. (#229)
* Fixed flake8 issues
* Add handling for macOS mach binaries (#231)
The environment modules package has been updated to include
versions up to 4.0.0. The url of the package and the homepage
have been updated accordingly.
The `spack bootstrap` command now builds version 3.2.10 of
the environment-modules package, and will do until #10708
is fixed.
* Make use of new QE multi-valued HDF5 variant interface
* Remove explicit CUDA dependency (which is handled by inheriting
from CudaPackage)
* Update cuda_arch unset variant check
libbeagle compiles against CUDA by default but no there is no mention
of it in the package recipe. This PR adds explicit cuda paths and
variants, and fixes the target architecture as well (for those who
don't have compute_13)
Serial HDF5 in serial QE only works in develop version. This adds
a conflict to the QE package to prevent Spack from trying to build
specs which don't satisfy this constraint.
This restores the use of Package.headers when computing -I options
for building a package that was added in #8136 and reverted in
#10604. #8136 used utility logic that located all header files in
an installation prefix, and calculated the -I options as the
immediate roots containing those header files.
In some cases, for a package containing a directory structure like
prefix/
include/
ex1.h
subdir/
ex2.h
dependents may expect to include ex2.h relative to 'include', and
adding 'prefix/include/subdir' as a -I was causing errors,
in particular if ex2.h has the same name as a system header.
This updates header utility logic to by default return the base
"include" directory when it exists, rather than subdirectories.
It also makes it possible for package implementers to override
Package.headers to return the subdirectory when it is required
(for example with libxml2).
Update older md5 checksums to sha256. Require CMake >= 3.6.0 (updated
from 3.5.0). Remove some third-party depends_on statements (which
were commented out before).
Extract implicit xenv dependency into a new package and add it as an
explicit dependency. Update Gaudi to explicitly depend on older
versions of xenv rather than the latest develop because that
conflicts with Spack's compiler wrappers.
Spack warns users when a dependency package updates CPATH. This
warning message is generating bug reports and alarm in cases where
there is no problem. For now this downgrades the warning message to
the debug level, so it only shows up if something goes wrong for the
user and they ask for more information from Spack.
The nvptx installation support phase (added in 040b658) was always
running for gcc regardless of whether +nvptx was enabled. This
removes the "@when" syntax (which apparently is not working) and
adds a check inside of the "nvptx_install" function to bail out if
+nvptx is not enabled.
This spack command adds a new schema for a file which describes the
builder containers available, along with the compilers availabe on
each builder. The release-jobs command then generates the .gitlab-ci.yml
file by first expanding the release spec set, concretizing each spec
(in an appropriate docker container if --this-machine-only argument is
not provided on command line), and then combining and staging all the
concrete specs as jobs to be run by gitlab.
The built images are set up with fairly recent versions of gcc and
clang:
- centos_7: [ gcc@5.5.0 (built from src), clang@6.0.0 (spack-built from src) ]
- ubuntu_18.04: [ gcc@5.5.0 (system), clang@6.0.0-1ubuntu2 (system) ]
Adds four new sub-commands to the buildcache command:
1. save-yaml: Takes a root spec and a list of dependent spec names,
along with a directory in which to save yaml files, and writes out
the full spec.yaml for each of the dependent specs. This only needs
to concretize the root spec once, then indexes it with the names of
the dependent specs.
2. check: Checks a spec (via either an abstract spec or via a full
spec.yaml) against remote mirror to see if it needs to be rebuilt.
Comparies full_hash stored on remote mirror with full_hash computed
locally to determine whether spec needs to be rebuilt. Can also
generate list of specs to check against remote mirror by expanding
the set of release specs expressed in etc/spack/defaults/release.yaml.
3. get-buildcache-name: Makes it possible to attempt to read directly
the spec.yaml file on a remote or local mirror by providing the path
where the file should live based on concretizing the spec.
4. download: Downloads all buildcache files associated with a spec
on a remote mirror, including any .spack, .spec, and .cdashid files
that might exist. Puts the files into the local path provided on
the command line, and organizes them in the same hierarchy found on
the remote mirror
This commit also refactors lib/spack/spack/util/web.py to expose
functionality allowing other modules to read data from a url.
- add CombinatorialSpecSet in spack.util.spec_set module.
- class is iterable and encaspulated YAML parsing and validation.
- Adjust YAML format to be more generic
- YAML spec-set format now has a `matrix` section, which can contain
multiple lists of specs, generated different ways. Including:
- specs: a raw list of specs.
- packages: a list of package names and versions
- compilers: a list of compiler names and versions
- All of the elements of `matrix` are dimensions for the build matrix;
we take the cartesian product of these lists of specs to generate a
build matrix. This means we can add things like [^mpich, ^openmpi]
to get builds with different MPI versions. It also means we can
multiply the build matrix out with lots of different parameters.
- Add a schema format for spec-sets
On macOS, the build of libxxhash is performed by a bare `make`
launched by `MakePackage`s default `build` phase. As `prefix`
is not set at this point, the library gets compiled with a hard-coded
install name of `/usr/local/libxxhash.dylib`. Downstream clients,
in the tested case Root, will then fail to build as the install name
is incorrect.
Move setting of `prefix` from `install` to `edit` so that it
propagates to subsequent `build` and `install` phases.
* QE hdf5 is now a multi-valued variant. Update conflicts accordingly.
* Update dependencies for hdf5 multi-valued variant. Bug fix: high-level hdf5 library needs to be a hard dependecy.
* Support for HDF5 serial interface. Note that this is not supported in QE autoconf, but will be fixed in the next official release.
* Support in QMCPACK for CUDA_ARCH flag.
* No need to have explicit CUDA variant anymore.
* Internal QMCPACK CMake assumes a minimum GPU architecture if none is provided.
Fixes#10617Fixes#10624Closes: #10619#8136 dependended entirely on spec.libs to retrieve library directories
from dependencies. By default this function only retrieves libraries if
their name is something like lib<package> (e.g. "libfoo.so" for a
package called "Foo"). This unconditionally adds lib/lib64 directories
for each dependency as link/rpath directories.
This also filters system paths from link/rpaths/include directories and
removes duplicated paths that #8136 could add.
If the -f <specyamlfile> argument to install is used (rather than
providing package specs on the command line), CDash throws an exception
due to missing the installation command (the packages targeted for
install). This fixes that behavior so CDash reporting succeeds in
either case.
fixes#10601
Due to a bug this attribute is wrong for packages that use directories
as namespaces. For instance it will add "<boost-prefix>/include/boost"
instead of "<boost-prefix>/include" to the include path.
As a minor addition a few loops in the compiler wrappers have been
simplified.
* cp2k: archive the Makefile after a successful installation
* cp2k: all checksums are now sha256
* libint: all checksums are now sha256 + added versions 2.4.X
* libxc: all checksums are now sha256 + added version 4.3.2
* cp2k: os.path.join instead of join_path + simplified openmp flags
* cp2k: turned into a MakefilePackage
* cp2k: refactored edit method so that Makefile writing happens last
* icu4c: Add cxxstd variant
For versions supported in Spack, ICU4C defaults to the C++11.
Provide a variant to default to this, plus options for C++14
and 17 if dependees require for ABI compatibility.
* icu4c: cxxstd variant cannot be multi
* Updated Geant4 and dependencies clhep and vecgeom to use cxxstd= variant so that it is applied consistently.
* Make standalone geant4 data packages that are used when data is not installed with geant4. This allows installing the data files are externals that can be common to many compiler version.
* move geant4 data package dependencies to umbrella package geant4-data
Spack shell detection in setup-env.sh was originally based on
examining the executable name of $$ (from "ps"). In some cases this
does not actually give the name of the shell used, for example when
setup-env.sh was invoked from a script using "#!". To make shell
detection more robust, this adds a preliminary check for shell
variables which indicate that the shell is bash or zsh; the
executable name of $$ is retained as a fallback if those variables
are not defined.
Fixes#7855Closes#8070Closes#2645
When searching for library directories (e.g. to add "-L" arguments to
the compiler wrapper) Spack was only trying the "lib/" and "lib64/"
directories for each dependency install prefix; this missed cases
where packages would install libraries to subdirectories and also was
not customizable. This PR makes use of the ".headers" and ".libs"
properties for more-advanced location of header/library directories.
Since packages can override the default behavior of ".headers" and
".libs", it also allows package writers to customize.
The following environment variables which used to be set by Spack
for a package build have been removed:
* Remove SPACK_PREFIX and SPACK_DEPENDENCIES environment variables as
they are no-longer used
* Remove SPACK_INSTALL environment variable: it was not used before
this PR
* Added a package for the MDAnalysis toolkit.
* Updated LBANN, Hydrogen, and Aluminum to not require CUDA-aware MPI
even when using CUDA. Fixed small bugs within both LBANN and Hydrogen
packages. Added requirements for breathe and cereal to LBANN.
* Update use case for doc dependencies.
* Cleaning up some of the LBANN software stack dependencies.
* fix permission setter
Fix a typo in islink test when applied to files.
* os.walk explicitly set not to follow links
The algorithm strongly rely on not following links.
* Note that `none` is the default for lmod autoload
Save a bit of confusion by *explicitly* pointing out that `none` is
the default value for autoload in the lmod module file generator.
* Add a tip re building software externally
Add a tip about using `autoload: all` when building packages outside
of the tree that use artifacts (e.g. libraries, includes) within the
tree.
CMake supports the notion of secondary generators which provide extra
information to (e.g.) IDEs over and above that normally provided by
the primary generator. Spack only supports the 'Unix Makefiles' and
'Ninja' primary generators but was not parsing out the primary
generator when a secondary generator was also included (e.g. for
a generator attribute like 'Codeblocks - Ninja'). This adds a regex
for extracting the primary generator for validation.
Since the secondary generator is irrelevant to a Spack build, it is
passed on to CMake without further validation.
Add variant to support compiling against main C++ standards if needed
for ABI compatibility. Use 'default' value as Xerces-C autoconf system
does not enforce a minimum, or set a default, C++ standard.
* Extend current recipe for root package
* Add removed old version
* Add dependency type for Python and R packages
* Add compiler variants
* Adding new root version: 6.14.00
* Add back the minimum cmake version required
* LZ4 and xxhash support for old (using builtin options) and new versions (external packages)
* Set PYTHONPATH in setup_environment
* Address typos and issues pointed by HadrienG2
* Add newer ROOT versions
* Solve support for asimage
* Define incompatible variants
* Organize old dependencies
* Add support for minuit
* Add support for pythia6
* Remove unneeded list_url
* Update license header
* Remove block of commented old dependencies
* Contrain Qt version
* Comment pythia8 variant/dependency, not supported by spack
* Force not to build pythia8
* Adding a package for cardioid.
* Changing package to pass flake8
* Removing template leftovers per @ax3l 's advice
* Adding a homepage so tests pass.
* Removing whitespace for flake8
* Changing the name of master branch to match spack standards.
1137b18 removed the -s option from spack such that to see a stack
trace you must pass the --stacktrace option. The bug report templates
were not updated accordingly.
Add patch to fix Fortran mangling in CMake test
This is a fix for builds with GCC 6/7/8 and CMake 2/3. It includes
a link to the issue where this is reported. For now, the Spack
package applies the patch to all releases >= 2.0.2
This package depends on both python 3.5+ and node-js which depends on
python 2.7/2.8 to build. Currently it can only be installed if the
node-js package's python dependency is manually removed, so node-js
will build with python found on the system rather than provided by
spack. Upcoming commits may allow spack to reconcile this
automatically.
* xsdk@develop: switch trilinos from @master to @develop
* xsdk@develop: dtk - fix trilinos dependency on dtk.
- dkt-3.0 should be used with trilinos-12.4*
- dtk/master should be used with trilinos@develop
A version >= 0.9.4 required for py-black. Added latest, which is
0.10.0. Manually verified that the .tar.gz for v0.9.3 used to generate
the sha256 matched the previous md5.
* CUDA compiler conflicts for Linux.
* Add Volta and Turing GPUs.
* Add mandatory conflict for Volta and Turing GPUs.
* Revert "CUDA compiler conflicts for Linux."
This reverts commit 7d4ff654ac.
* Compiler conflicts introduced from previous commit into CUDA packaged moved and integrated into CUDA build system.
* More conversative with compiler conflicts for cuda 10.0.130, since I don't know what will happen with future cuda 10.x releases.
* Correct off-by-one errors in clang conflicts for x86_64 Linux.
* No restrictions on Apple Clang compiler until we are able to distinguish Xcode clang from github clang more easily. Note to fix this in the future.
* Change comment to clarify that github clang refers to LLVM clang.
* Fix and simplify index range.
* Fix overlapping conflicts for CUDA 10.0.130
* Removed extra ^cuda from conflict.
Add tribool.patch to add explicit casts from tribool to bool in two
places. Boost, starting with version 1.69, turned on explicit
conversions for tribool, which breaks the build for dyninst.
This is merged into the current dyninst develop and is expected to be
part of the next release, so the version range uses the current
release as an upper bound.
Closes#10168
* Add versions 4.1.0.0 and 4.0.12.0
* Replace md5 checksums with sha256
* Logic to generate a helper script for versions earlier than 4 was
not running. This corrects the branch which controls whether this
script is generated.
* Constrain Java dependency to version 8 (GATK does not work with
Java 11)
* Avoid adding scripts that have dependencies not handled by Spack
such as gcloud and google-chrome. These scripts are good candidates
for later extensions.
Closes#2622
This retrieves and installs the binary of Oracle's OpenJDK (an
open-source JDK implementation). The logic is largely based on the
Spack JDK package (Oracle's closed-source JDK implementation).
For now this is only supports Linux. If the download could be
customized based on OS (for the same version), binaries are also
available for MacOS and Windows.
Build system was changed to use a Makefile. Therefore older versions
(which don't support this) are removed. The new build system also
supports testing by default.
Fixes#7372
Added patch method which renames config.h in the fftw subdir of the
source tree. fftw 2.1.5 appears to ship with a copy of this file
with all defines commented out. This gets read by the #include
directives instead of the version in the build directory with the
correct defines. As a result, many C preprocessor macros left
undefined, including F77_FUNC_ which causes the bulk of fttwf77.c
to be skipped due to an #ifdef, so fftw_reverse_int_array et al
not included in library. Fixes#7372
Also fixed some inconsistencies with the handling of quad and
long_double in specs between the configure method and the build,
check, and install methods.
The Mercury package previously depended on libfabric@develop on
Cray platforms in order to pick up a fix for the GNI provider that
was only available in git. This is no longer needed now that the
libfabric package is at 1.7.0.
Add new versions for py-keras. Versions of py-keras at or after
2.2.0 require keras-processing and keras-applications packages.
Add these packages to Spack and add dependencies to the py-keras
package.
* Add version 11.2.68
* Add variant to disable AVX2 for older hardware
* Change the makefile so libraries and includes are not placed
under $PREFIX/usr
* Updated URL and homepage
Added versions 1.0.0-rc1, 0.6.1, 0.6.0, 0.5.1
hwloc dependency was removed after 0.5.1, so the related configure
option is not specified after 0.5.1
Require py-matplotlib dependency version 2.2.3 (version was not
specified before)
Fixes#8343#4154#7370Closes#4155
All but one of the backports packages were using "pkgutil-style"
namespacing, but configparser was using "pkg_resource-style".
A namespace needs to be one style or another. This adds a new
configparser version which switches to using the pkgutil-style
namespace approach.
With this update, one can now install flake8 with Spack and use
it by loading modules.
Debug output now includes the output of modulecmd executions. Only
output module content when a failure occurs; always report when a
module is loaded/unloaded.
The PREFIX variable in the makefiles is directly used to construct the
install name on the library metadata on darwin. If not passed, the
default /usr/local is used and the resulting librhash.dylib contains
usage informaiton of /usr/local/lib/librhash.0.dylib. This adds the
PREFIX= variable directly to the make command so the resulting usage
metadata contains the correct valid library path.
* flux-core: add v0.11.0 and updated dependencies
libuuid hasn't been a direct dependency of flux-core since 2015 (well
before any of the versions listed in the package).
The yaml-cpp dependency moved to flux-sched when we moved the jobpsec
library from core to sched.
Some python3 incompatibilities were found in 0.11.0, so bump the python3
support back a version.
* flux-sched: add v0.7.0 and make boost dependency more precise
* llvm: Bump version to 7.0.1
* llvm: Added perl-data-dumper build dependency for openmp
* llvm: Enable exception handling and RTTI
Useful to have turned on in general with RTTI but also necessary
to workaround some lldb stability issues with some versions of
libstdc++.
* Update the PMIx package
- Add latest releases to the list.
- Add dependency on hwloc for versions >= 3.0
- Flag preferred versions for libevent and hwloc
- Improve the description
- Point the project URL to the correct site
Signed-off-by: Ralph Castain <rhc@pmix.org>
* Only install the hwloc dependency for versions >= 3.0.0
Thanks to @ax3l for the suggestion
Signed-off-by: Ralph Castain <rhc@pmix.org>
* Add myself as a listed maintainer
Signed-off-by: Ralph Castain <rhc@pmix.org>
* Add warnings for existing conflicts. New versions of QMCPACK require a compiler with C++14 support.
* QMCPACK now requires a newer release of CMake.
* Switch QMCPACK to using FFTW-api provider.
* Relax dependencies on QMCPACK +da variant.
* Add QMCPACK 3.6.0 release.
* QMCPACK compiler-CUDA conflicts.
* QMCPACK fix typo in expressing conflict.
* Conditional needed since these variables are only used when FFTW is a dependency to QMCPACK.
* QMCPACK QE variant dependency was underspecified. Must include MPI as a hard dependency on QE.
* Update QMCPACK dependencies for HDF5, CMake and Boost.
* QMCPACK clean-up, more consistent use of spec vs. self.spec.
* Add support for collective I/O variant of QMCPACK.
* Update QMCPACK conflicts for QE and PHDF5 variants.
* Use built-in install method for QMCPACK starting with 3.6.0
* QMCPACK compiler C++14 conflicts were off by one version.
* QMCPACK package needs to setup_environment method.
* In support of PR#10252, get rid of slashes in string.
* Migrate these to the CUDA package and make a seperate PR.
* In support of PR#10252, get rid of join_path for nexus.
* In support of PR#10252, work around concretizer limitation.
* In support of PR#10252, convert to conditionals to if-else clauses for compactness.
* In support of PR#10252, get rid of extra space and new line in warning messages.
"spack install" will install all packages added to the current
environment. When this included external packages, the environment
update would fail because it would attempt to copy log files that
were only generated if Spack handled the install itself. This skips
that step for external packages.
* Allow overwrite nonexistent and multiple packages
initial implementation
give one prompt to users instead of a prompt per spec
testing
* flake
* bugfix: install overwrite check each spec against installed
* python3 compliance for filter/map
* Updated MPI and TAU versions in the TAU package
* Updated PDT versions
* Added support for Python (+python), default to `False`
* Cleaned up binutils, python, mpi usage.
* Updated binutils install to add additional header files to the include/extra directory.
* Resolving review problems with TAU and binutils packages.
The binutils package was updated to include support for installing
the ELF headers as a +headers variant. The TAU package was updated
to use all known Spack dependencies.
* r-gamlss-data: new package at 5.1-0
* r-gamlss-dist: new package at 5.1-1
* r-gamlss: new package at 5.1-2
* r-gamlss: add mass, survival, nlme imports
Fixes#10361Fixes#6634
When mkl isnt in the spec, still write an explicit mkl entry to
prevent numpy from looking in system dirs and using a system install
of mkl. Do the same for openblas and atlas (although mkl was the only
system install reported to interfere with Spack's choice).
* Updating nalu-wind, nalu, openfast, and tioga packages. Includes
improved handling of +shared variant.
* Add fftw and catalyst support to nalu-wind.
* Restrict ParaView to use matplotlib versions before 3 (which
require Python 3)
* py-modred: New package
Parallelized python module for finding modal decompositions and
reduced order models.
* fix license
I think?
* Add dependency on py-numpy
and remove commented out code
* vigra: new package
* Better constraints and types on some depends_on. Use site_packages_dir variable for Python install dir. Explicit switch-offs where available. Change some local variables to suit style checks.
* Take care of lines too long
* Explicit WITH_HDF5=ON
* docbook processing works correctly for gdk-pixbuf
1. The various bits of documentation in gdk-pixbuf include hardcoded
references to dtd's and stuff at their canonical, Internet,
locations. BUT, gdk-pixbuf runs xslt-proc with the `--nonet`
option, which forbids it from using the network. Sadness ensues.
Traditionally folks use XML Catalogs to map these to local copies.
Our docbook-xsl package wasn't setting the appropriate env var in
its dependents environments to use our catalog.
Now it does. Less sadness ensues.
2. If we're going to use these things, we should depend on them.
* Add gdk-pixbuf's bin to build environment
The "post-install.sh" script uses gdk-pixbuf-query-loaders, which was
installed earlier.
If py-psyclone can set its own bin on its PATH, so can we...
* Make gdk-pixbuf use recognizable docbook location
Our docbook-xsl package assumes that the canonical location is
`http://cdn.docbook.org/release/xsl/current/manpages/docbook.xsl`, but
the gdk-pixbuf's meson build script uses
`http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl`.
This means that our XML Catalog doesn't fix the reference and sadness
happens.
Just patch the build that we see what we want to see, then we can make
it all go away.
* Add commentary re docbook patch
* Make catalog helper a property
Thanks @adamjstewart!
* Run tests if/when run_tests is true
Thanks @adamjstewart!
* Tune up dependencies
Thanks @adamjstewart!
* Wordsmit commentary
This adds a lot of fixes and updates for alpine ascent and its dependencies:
* add support for static (via ~shared) and use vtk-m 1.2
* update vtkh package to output cmake configure file and pinning it to vtkm 1.2
* make conduit respect ~python
* fix ascent python logic
* consistant cmake usage
* conditionally add tbb in ascent if vtkh
* enable openmp
* more robust handling of variants
* update ascent and conduit for static builds
* add optional support for mfem in ascent
* enable mfem conduit support for ascent
* add optional adios dep to conduit
* remove ver req from conduit
* ascent: remove confusing comment
* tweaks to conduit and ascent pkg recipes
* fix typo in conduit package
* pref conduit master
* changing mfem to depend on conduit@master to get updated relay
* restore use of conduit 0.3.1 or greater for mfem
* set master as preferred conduit version
* allow mfem to use conduit master
* adding rover package and editing ascent
* updating vtkm cmake dep
* guard ascent python support on +shared
* removing rover to simply ascent package
* add fortran variant to conduit, to allow us to turn off conduit support even when a fortran compiler is specified
* fix fortran compiler check so it can work on cray systems
* working towards cuda fix for vtkm lagrange filter
* update ascent package with more variants, and patch to avoid nvcc issue
* hdf5 flags fix for BGQ
* add post install test
* add testing to ascent
* add testing of the using-with-make example
* add ctest output on error for run_tests
* Remove Cray CC compilers causing problems on case-insensitive filesystems
* cray -> cce
* Ensure that compiler-specific directory comes first in build-env
* Point to compiler-specific symlinks
Binary caches of packages with absolute symlinks had broken symlinks.
As a stopgap measure, #9747 addressed this by replacing symlinks with
copies of files when creating binary cached packages.
This reverts #9747 and instead, either relative-izes the symlink or
rewrites the target. If the binary cache is created using '--rel' (as
in "spack buildcache create --rel...") then absolute symlinks will be
replaced with relative symlinks (in addition to making RPATHs relative
as before); otherwise they are rewritten (when the binary cache is
unpacked and installed).
The current output of buildcache list is very verbose and I feel like
some details are getting lost. By making the output similar to find, I
think users will be able to get a better overview of what is stored in
the cache.
* dealii: fix concretization of xsdk package
* tests: add concretization tests for deal.II and xSDK, which are often broken due to limitations in the concretizer
* use pytest.mark.parametrize
Updated LBANN, Hydrogen, and Aluminum to not require CUDA-aware MPI
even when using CUDA. Fixed small bugs within both LBANN and Hydrogen
packages. Added requirements for breathe and cereal to LBANN.
Fixes issues #9613 and #3209.
Remove the spack wrapper directories from PATH for the bootstrap step.
This was breaking the build for Cray (and other cross-compile) because
bjam was built for the BE and died on SIGILL on the FE.
This only affects building bjam. The boost libraries are still built
the normal spack way with the spack wrappers.
Add variants 'cray' and 'bgq' to build for Cray and Blue Gene compute
nodes. These systems need separate options to find the MPI compiler
for hpcprof-mpi.
The 4.0.1 release is the culmination of the SUNDIALS 4.0.0 preview releases,
i.e. the 4.0.0-dev versions. As such, the 4.0.0-dev releases are now
being removed.
* Explicitly listing blas and lapack sections in site.cfg for numpy when using netlib-lapack so that scipy can find them as well.
* Reducing code to use a function call instead of copying and pasting.
* Fixing flake8 errors.
* Fixing python 3.6 error when site.cfg lists library_dirs multiple times.
* Formatting.
* Verifying blas and lapack are enabled before writing to site.cfg.
* Fixing flake8 errors.
* Fixing conflicts since this package file has been updated.
* Fixing flake8 stuff.
* Handling blas and lapack variants for netlib-lapack in site.cfg for py-numpy.
* Don't write netlib-lapack site.cfg file if no blas or lapack variants enabled in py-numpy.
* Removing redundant if statement.
* Fixing mistake in py-numpy site.cfg generation.
* Separating blas and lapack further in site.cfg for netlib-lapack in py-numpy.
Allow customizing views with Spec-formatted directory structure
Allow views to specify projections that are more complicated than
merging every package into a single shared prefix. This will allow
sites to configure a view for the way they want to present packages
to their users; for example this can be used to create a prefix for
each package but omit the DAG hash from the path.
This includes a new YAML format file for specifying the simplified
prefix for a spec in a view. This configuration allows the use of
different prefix formats for different specs (i.e. specs depending
on MPI can include the MPI implementation in the prefix).
Documentation on usage of the view projection configuration is
included.
Depending on the projection configuration, paths are not guaranteed
to be unique and it may not be possible to add multiple installs of
a package to a view.
* elfutils: option to install elf.h header file
Add variant 'elfh' (default False) to install the elf.h header file to
the include directory. This provides access to a newer version of
elf.h and allows an elf application to build with the same elf.h that
libelf.so was built with.
Add version 0.175.
* Drop the variant 'elfh' and always install elf.h instead.
* new package: git-fat
* backport of rsync -> older versions no longer available
Change-Id: I3a41f17df0f88ca11e08ea6fd2f2d381435e0f09
* fix legal stuff and use new git attribute
Change-Id: Ice156dfe8ad4137a861f7b2938ee92f4024142be
* use date as version number rather than hash
Change-Id: Ib537659e368f69124895306d6a6ac0c5e36061a7
* only use the date once
Change-Id: I2592fa28aaa53fed949ddc82959d712ff3f6f752
Fixes#10284#10152 replaced shutil.move with llnl's copy and copy_tree for
resources. This did not copy permissions so led to later failures
if an executable was copied (e.g. a configure script). This uses
install/install_tree instead, which preserve permissions.
* Initial compiler support
* added arm.py
* Changed licence to Arm suggested header
* Changed licence to the same as clang.py
Main author of file is Nick Forrington <Nick.Forrington@arm.com>
Minor changes by Srinath Vadlamani <srinath.vadlamani@arm.com>
* compilers: add arm compiler detection to Spack
- added arm.py with support for detecting `armclang` and `armflang`
Co-authored-by: Srinath Vadlamani <srinath.vadlamani@arm.com>
* Changed to using get get_compiler_version
* linking to general cc for arm compiler
* For arm compiler add CFLAGS to use compiler-rt rtlib.
* Escape for special characters in rexep
* Cleaned up for Flake8 to pass.
* libcompiler-rt should be part of the LDFLAGS not CFLAGS
* fixed m4 when using clang to used LDFLAGS. Fixed comments for arm.py to display compiler --version output with # NOAQ for flakes pass.
* added arm compilers
* proper linked names
* add the tfel package
* fix the tfel package
* fix the tfel package
* fix the tfel package
* Taking Adam J. Steward' remarks into account
* fixes trailing white spaces
* Update description
* Update dependencies following @adamjstewart adices
* Style fixes
* Style fixes
* Add java optional support
* add the maintainers attribute (following @alalazo advice), disable interface not selected (following @adamjstewart advice)
* flake8 fixes
* Fix Cast3M and python-bindings support. Python detection is made compatible with cmake'FindPythonLibs module (at least how it is used in TFEL)
* Style fixes
* Style fixes
* Fix test on python version
* Follow @adamjstewart advices: code is much cleaner and readable
* Small fix
* Small fix
* Add comment
* Small fix in cmake option
* try again (trying to overcome Travis CI unstable build process)
* Add support for the MFrontGenericInterfaceSupport project (MGIS)
* Style fixes
* Package documentation update
* Package documentation update
* Fix a typo thanks to Andreas Baumbach review
* Follow Adam J. Stewart advices
* Fix type
* bowtie: new versions and %gcc@8.0.0: fix
Bowtie development shifted from Sourceforge to GitHub.
This commit adds several versions from GitHub, using the archive
tarballs. Note that the URL for 1.2.2 tarball is actually an '_p1'
tag....
It leaves the old 1.2 release download info in place.
Versions 1.2.0 and newer come from GitHub (I'm unsure if the 1.2 and
1.2.0 are equivalent).
Include a fix that enables %gcc@8.0.0: from:
https://github.com/BenLangmead/bowtie/issues/87
But, v1.2.2 has trouble with "newer" gcc's, so it only adds v1.2.2 for
%gcc@6.0.0:.
Feel free to tighten this. I know that:
- 1.2 -> 1.2.2 work with %gcc@5.5.0;
- 1.2 -> 1.2.1.1 work with %gcc@8.2.0; and
- 1.2.2 fails with %gcc@8.2.0
* Tighten to `conflicts('%gcc@8:', when='@1.2.2')`
* Point 1.2.2 and 1.2.2_p1 at the 1.2.2_p1 tarball
* Update package.py
Doxygen has migrated from a private SVN to GitHub. This PR updates the URLs and adds versioned commit hashes from GitHub. It also includes version 1.8.15 as the latest option.
* Update package.py
This enforces conventions that allow for correct handling of
multi-valued variants where specifying no value is an option,
and adds convenience functionality for specifying multi-valued
variants with conflicting sets of values. This also adds a notion
of "feature values" for variants, which are those that are understood
by the build system (e.g. those that would appear as configure
options). In more detail:
* Add documentation on variants to the packaging guide
* Forbid usage of '' or None as a possible variant value, in
particular as a default. To indicate choosing no value, the user
must explicitly define an option like 'none'. Without this,
multi-valued variants with default set to None were not parsable
from the command line (Fixes#6314)
* Add "disjoint_sets" function to support the declaration of
multi-valued variants with conflicting sets of options. For example
a variant "foo" with possible values "a", "b", and "c" where "c"
is exclusive of the other values ("foo=a,b" and "foo=c" are
valid but "foo=a,c" is not).
* Add "any_combination_of" function to support the declaration of
multi-valued variants where it is valid to choose none of the
values. This automatically defines "none" as an option (exclusive
with all other choices); this value does not appear when iterating
over the variant's values, for example in "with_or_without" (which
constructs autotools option strings from variant values).
* The "disjoint_sets" and "any_combination_of" methods return an
object which tracks the possible values. It is also possible to
indicate that some of these values do not correspond to options
understood by the package's build system, such that methods like
"with_or_without" will not define options for those values (this
occurs automatically for "none")
* Add documentation for usage of new functions for specifying
multi-valued variants
Non-expanded resources were being deleted from the cache on account
of two behaviors:
* ResourceStage was moving files rather than copying them, and uses
"os.path.realpath" to resolve symlinks
* CacheFetchStrategy creates a symlink to a cached resource rather
than copying it
This alters the first behavior: ResourceStage now copies the file
rather than moving it.
"mirror create" was invoking a package's do_patch method in order to
retrieve and archive URL patches. If a package implements a "patch"
method, this is also called as part of do_patch; this failed when the
package-specific implementation referred to environment variables
that are only available at the time the package is built
(e.g. "spack_cc").
This change introduces fetch and clean methods for patches. They are
no-ops for FilePatch but perform the appropriate actions for
UrlPatch. This allows "mirror create" to invoke do_fetch, which does
not call the package's patch method.
* PARALLEL-NETCDF: Update new version and location
PnetCDF-1.11.0 is released.
Also, the canonical download area has been changed and they are now using git, so can also provide a develop and master checkout.
One issue is that they changed the name of the tar files, so 1.11.0 needs special handling (and future versions will also).
All checksums at new location match the checksums from the old location.
* Address concerns in review
Added a `url_for_version` function to handle the name change in the tar files at version 1.11.0 and later. Updated description to match current shown on website. Updated `homepage` setting since it has recently moved.
* Address issues in second review
- in many files, regular strings were used in places where raw strings
should've been used.
- convert these to raw strings and get rid of new flake8 errors
This PR improves the validation of `modules.yaml` by introducing a custom validator that checks if an attribute listed in `properties` or `patternProperties` is a valid spec. This new check applied to the test case in #9857 gives:
```console
$ spack install szip
==> Error: /home/mculpo/.spack/linux/modules.yaml:5: "^python@2.7@" is an invalid spec [Invalid version specifier]
```
Details:
* Moved the set-up of a custom validator class to spack.schema
* In Spack we use `jsonschema` to validate configuration files
against a schema. We also need custom validators to enforce
writing default values within "properties" or "patternProperties"
attributes.
* Currently, validators were customized at the place of use and with the
recent introduction of environments that meant we were setting-up and
using 2 different validator classes in two different modules.
* This commit moves the set-up of a custom validator class in the
`spack.schema` module and refactors the code in `spack.config` and
`spack.environments` to use it.
* Added a custom validator to check if an attribute is a valid spec
* Added a custom validator that can be used on objects, which yields an
error if the attribute is not a valid spec.
* Updated the schema for modules.yaml
* Updated modules.yaml to fix a few inconsistencies:
- a few attributes were not tested properly using 'anyOf'
- suffixes has been updated to also check that the attribute is a spec
- hierarchical_scheme has been updated to hierarchy
* Removed $ref from every schema
* $ref is not composable or particularly legible
* Use python dicts and regular old variables instead.
* express: new version, use tags and fix gcc@6.0.0:
Express fails to build with gcc@6.0.0:.
The debian project [has a
fix](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=811859) but
they don't seem to have pushed it upstream.
I've opened an issue and a PR in eXpress repo, but eXpress isn't
actively developed, so I'm fixing it here too.
Since the Spack package was created, the eXpress team tagged their
releases. I've updated the package to use the tags.
Version 1.5.1 used to be known as 2015-11-29 (same commit). 1.5.2 is
new(er).
* Make flake8 happ{y,ier}
Apply fix from aspell issue (519) for a pointer dereference bug that
newer versions of gcc won't let slip past.
There hasn't been a release that includes the fix, this applies the
change to the latest release.
There's a missing break in a switch statement that newer gcc's
dislike.
Our #4696 simply disallowed newer gcc's.
This fixes the problem instead.
It's been [PR'ed upstream](https://github.com/agordon/fastx_toolkit/pull/22).
Tested with gcc@5.5.0 and gcc@8.2.0 on CentOS.
The most recent release of bamutil that we support uses an embedded
copy of libStatGen that has several issues that keep it from building
with newer releases of gcc.
They've all been fixed upstream and the latest release of bamutil
would pick them up if/when we support it. The build process has
changed though, plus my team needs *this* version.
This commit backports those fixes.
- The nested directive implementation was broken for python 3
- directive results were not properly removed from the directive list
when it was processed in the DirectiveMeta metaclass.
- the issue was that remove_directives only descended into a list or
tuple, but in Python3, the initial value passed to the function is a
view of dictionary values.
- make it a list to fix things, and add a regression test.
- currently just looks at patches
- allows you to find out which package applied a patch to a spec
- intended to work with tarballs and resources in the future.
- add tab completion for `spack resource` and subcommands
- previously, if a concrete sub-DAG with patched specs was written out
and read back in, its patches would not be found because the dependent
that patched it was no longer in the DAG.
- Add a test to ensure that the PatchCache handles this case.
- Also add tests to ensure that patch objects are properly created from
Specs -- previously we only checked that the patches were on the Spec.
- this fixes a bug where if we save a concretized sug-DAG where a package
had been patched by a dependent, and the dependent was not in the DAG,
we would not read in all patches correctly.
- Rather than looking up patches in the DAG, we look them up globally
from an index created from the entire repository.
- The patch cache is a bit tricky for several reasons:
- we have to cache information from packages, specifically, the patch
level and working directory.
- FilePatches need to know which package owns them, so that they can
figure out where the patch lives. The repo can change locations from
run to run, so we have to store relative paths and restore them when
the cache is reloaded.
- Patch files can change underneath the cache, because repo indexes
only update on package changes. We currently punt on this -- there
are stub methods for needs_update() that will need to check patch
files when packages are loaded. There isn't an easy way to do this
at global indexing time without making the FastPackageChecker a lot
slower. This is TBD for a future commit.
- Currently, the same patch can only be used one way in a package. That
is, if it appears twice with different level/working_dir settings,
bad things will happen. There's no package that current uses the
same patch two different ways, so we've punted on this as well, but
we may need to fix this in the future by moving a lot of the metdata
(level, working dir) to the spec, and *only* caching sha256sums in
the PatchCache. That would require some much more complicated tweaks
to the Spec, so we're holding off on that til later.
- This required patches to be refactored somewhat -- the difference
between a UrlPatch and a FilePatch is still not particularly clean.
- indexes should use json, not YAML, to optimize for speed
- only use YAML in human-editable files
- this makes ProviderIndex consistent with other indexes
- virtual provider cache and tags were previously generated by nearly
identical but separate methods.
- factor out an Indexer interface for updating repository caches, and
provide implementations for each type of index (TagIndex,
ProviderIndex) so that more can be added if needed.
- Among other things, this allows all indexes to be updated at once.
This is an advantage because loading package files is the real
overhead, and building the indexes once the packages are loaded is
trivial. We avoid extra bulk read-ins by generating all package indexes
at once.
- This can be extended for dependents (reverse dependencies) and patches
later.
- cleanup patch.py:
- make patch.py constructors more understandable
- loosen coupling of patch.py with package
- in Package: make package_dir, module, and namespace class properties
- These were previously instance properties and couldn't be called from
directives, e.g. in patch.create()
- make them class properties so that they can be used in class definition
- also add some instance properties to delegate to class properties so
that prior usage on Package objects still works
* Always build glib with iconv
My early PR, #10165, which added a variant to configure glib to use
libiconv and defaulted to false, seems to be causing more trouble than
the knob is worth.
This changes the glib package to always depend on and use libiconv.
* libiconv depends_on is no longer conditional
* First draft for installing Stata 15
I've tested this works on my systems and gives a successful install of STATA 15.
* Flake8 cleanup
* Another Flake8
* Update license header
* Various improvements
A lot of improvements as suggested by @adamjstewart
* flake8 check
- When returning string output, use text_type and decode utf-8 in Python
2 instead of using `str`
- This properly handles unicode, whereas before we would pass bad strings
to colify in `spack blame` when reading git output
- add a test that round-trips some unicode through an Executable object
* paraview: adding variants to use external packages as internal do not compile
* paraview: add latest paraview version
* catalyst: fixed libvtkexpat undefined reference linking error in Catalyst 5.5
* catalyst: add latest catalyst version
* catalyst: added ParaView_DIR env variable to catalyst module
* add paraview, catalyst patches
- https://gitlab.kitware.com/vtk/vtk-m/merge_requests/1166
- https://gitlab.kitware.com/paraview/paraview/merge_requests/2433
- https://gitlab.kitware.com/paraview/paraview/merge_requests/2436
* - Handle updated library paths for catalyst.
Versions 5.4 and old places libraries under a paraview subdirectory.
Eg, "lib/paraview-5.4", they are now placed directly under "lib"
- Minor code style cleanup
* Handle update library and python paths for ParaView-5.5
* catalyst: added ParaView_DIR path to spack_env
* BUG: applied the patch to the extracted catalyst source files
* paraview: added missing self to a member variable
* paraview: added Paraview_DIR to env
* catalyst: added osmesa variant
* of-catalyst: added new package
* add (FOAM,WM)_PROJECT_DIR also to spack_env environment
* depends on first openfoam release supporting catalyst
* openfoam-com: added missing env variables to module generation
* openfoam: fixed flake8 errors
* of-catalyst: added full variant and openfoam version dependency
* paraview: adding variants to use external packages as internal do not compile
* catalyst: fixed libvtkexpat undefined reference linking error in Catalyst 5.5
* catalyst: added ParaView_DIR env variable to catalyst module
* add paraview, catalyst patches
- https://gitlab.kitware.com/vtk/vtk-m/merge_requests/1166
- https://gitlab.kitware.com/paraview/paraview/merge_requests/2433
- https://gitlab.kitware.com/paraview/paraview/merge_requests/2436
* - Handle updated library paths for catalyst.
Versions 5.4 and old places libraries under a paraview subdirectory.
Eg, "lib/paraview-5.4", they are now placed directly under "lib"
- Minor code style cleanup
* Handle update library and python paths for ParaView-5.5
* catalyst: added ParaView_DIR path to spack_env
* BUG: applied the patch to the extracted catalyst source files
* of-catalyst: added new package
* add (FOAM,WM)_PROJECT_DIR also to spack_env environment
* depends on first openfoam release supporting catalyst
* paraview: added missing self to a member variable
* openfoam-com: added missing env variables to module generation
* openfoam: fixed flake8 errors
* paraview: added Paraview_DIR to env
* catalyst: added osmesa variant
* of-catalyst: added full variant and openfoam version dependency
* paraview-catalyst: use always external expat and netcdf
* of-catalyst: reformatted package description
* paraview-catalyst: removed duplicated function
* catalyst: fixed flake8 error
* of-catalyst: fixed license header
* of-catalyst: minor changes
* of-catalyst: renamed gitrepo with git
* of-catalyst: removed useless gitrepo parameter
* Added a package for the MDAnalysis toolkit.
* Added python packages for breathe and m2r. Updated versions and URL
for the sphinx package.
* Fixed flake8 error
* Updated package requirements based on feedback.
* Updated with requirements from the setup.py
* Switched it back to pypi.
* py-pynio: new package
* Major re-working to add some of the optional dependencies, and make it conform to the approach described in the pynio setup.py comments. Much of it turned out to be too complicated, and I have added some notes in comments to explain this to future adventurers.
* Added variants for the hdf5 and gdal dependencies, which do seem to be genuinely optional. Also added absolute dependencies for jpeg and zlib, which setup.py says it wants, but were being found as the os-installed versions.
* Added descriptions for the new variants, and moved them to above the dependencies
Enable explicitly building glib with or without libiconv.
Heads up: if you build with ~iconv but there is something else in the
spec that includes libiconv then glib will *still* automagically
discover it and the build will fail.
* Remove /nfs/tmp2 from default configuration
* /nfs/tmp2 is going away from LC... and doesn’t exist for the rest of the world.
* update documentation to remove /nfs/tmp2 as well
* add the tfel package
* fix the tfel package
* fix the tfel package
* fix the tfel package
* Taking Adam J. Steward' remarks into account
* fixes trailing white spaces
* Update description
* Update dependencies following @adamjstewart adices
* Style fixes
* Style fixes
* Add java optional support
* add the maintainers attribute (following @alalazo advice), disable interface not selected (following @adamjstewart advice)
* flake8 fixes
* Fix Cast3M and python-bindings support. Python detection is made compatible with cmake'FindPythonLibs module (at least how it is used in TFEL)
* Style fixes
* Style fixes
* Fix test on python version
* Follow @adamjstewart advices: code is much cleaner and readable
* Small fix
* Small fix
* Add comment
* Small fix in cmake option
* try again (trying to overcome Travis CI unstable build process)
- fixed an issue where some undesirable parts of
the spack source tree were being copied into
the image context.
- added a workaround for a tty ioctl warning on
ubuntu
- adjusted how the main images are built so that
`RUN spack ...` works automatically for child
images that base themselves on them.
* Record build output as an array of lines rather than concatenating to a
single large string.
* Use string.find to avoid running re.search on every line of output.
* CGNS: Allow use of git branch versions
Add ability to use git `master` and `develop` branches of the CGNS library. The `seacas` package sometimes needs to use the `master` or `develop` branch due to some new changes to the parallel I/O implementation in CGNS which has not yet been released.
* CGNS: Reorder to put newest version on top
Based on review comments, place newest version (develop) first in list.
- some commands were missed in the rollout of spack environments
- this makes all commands that need to disambiguate specs restrict the
disambiguation to installed packages in the active environment, as
users would expect
* dyninst: patch to build dyninst with older gcc
Add 'v9.3.2-auto.patch'. This patch changes some 'auto t: type' usage
to the older but equivalent 'type.begin()'. This allows building
dyninst 9.3.2 with gcc 4.4 which doesn't support the newer syntax.
This patch is harmless with newer gcc.
* Fix typo in patch. (My bad for sloppy cut-and-paste.)
* Restrict the patch to gcc 4.7 and earlier. gcc 4.8 supports the newer
usage and thus doesn't need the patch.
* proper include & linking of libarchive
* due to linux headers, mfu can't build on mac
* hard code deps in configure line
rather than relying on pkg-config
* +xattr during config
* must also link the lib
* find mfu.h
* long lines
* Add thread count variant
Atlas automatically configures itself and optimizes to run with a fixed number of threads. This can be overridden during compile by setting a flag and atlas will tune itself to use the specified number, rather than all system CPUs.
Default value is the existing logic, autoconfigure.
* Fix Linting
* Another lint for longline
* Added rocksdb package
* Fixes based on @tristan0x and @pramodk comments
* Cleanup
* A couple minor fixes based on @adamjstewart 's advice
* Fixed flake8 errors
- previous removal of FOAM_EXT_LIBBIN (ThirdParty) from the environment
during the build could lead to the system scotch library being found
instead of the spack installation.
Avoid this largely cosmetic change to the setting in favour of
improved build robustness. These modifications will be largely
superfluous in the 1812 release anyhow.
- Split the patching algorithm into 'regular' and '@:1806' since the
next versions of openfoam will again need fewer patches.
- Add hook into foamCreateManpage to be used where available
* SEACAS: Add preliminary support for seacas package
* Eliminate leftover from package I copied from.
* And remove another stray line
Sorry, thought I had checked this better...
* SEACAS: Shorten long lines; fixup indentation
* SEACAS: Remove unused os import
* Add PLASMA version 18.11.0 and Lua variant (#9689)
* Add PLASMA 18.11.1 with optional Lua and test for headers
* New versions for a few python modules (#9969)
* py-jupyter-console: Add version 5.2.0
* py-jupyter-core: add version 4.4.0
* py-lxml: add version 4.2.5
* py-zmq: add version 17.1.2
* py-terminado: add version 0.8.1
* py-pexpect: add version 4.6.0
* py-pytables: add version 3.4.4
* SEACAS: Default to mpi on
* SEACAS: Modify to reduce variants
Instead of using an `allpkgs` or explicitly specifying each individual subset,
changed to minimize variants:
* By default, the exodus and IOSS libraries are built and a few
executables directly related to the IOSS library.
* variant `common` defaults to True and builds the "common" seacas
applications. This is based on my definition, so there may be
could be some discussion on what belongs here.
* variant `legacy` defaults to True and builds the remainder of the
seacas applications. These are still used, but mainly in more
esoteric areas. Might be better to default to False, but I wanted
default to just build everything which is what other SEACAS builds
always do...
Removed the `kokkos` variants for now as I need to do some more
thinking on how best to support this. Not often used currently,
so shouldn't be an issue.
Cleaned up other issues raised by Adam and found by me.
* SEACAS: Address flake8 issues
* SEACAS: Clean up variant package logic
Fixed package variant logic.
Consolidated some other variant handling logic
Other minor refactors for hopefully better readability
* SEACAS: Fix flake8 error
Missing whitespace around operator fixed.
* GDL: python integration fixes
* renamed python-related variants to follow the convention
* building the Python module requires patches currently targetting 0.9.8 othwerwise asking for the Python module *only* builds the Pyhton module
* building the python module also requires patching the vendored (with the GDL) antlr to be built as a shared library
* Typo
Co-Authored-By: rmsds <ricardo.d.silva@gmail.com>
* Rename embed-python variant to embed_python
* zoltan: only add gussed MPI libs if no MPI wrappers are used
- if MPI-wrappers are used for compilation, we can assume that
linking works without manually specifying MPI libs
(guessing may result in wrong libs, cf. #8979)
- thus, only guess the NPI libs and add them explicitly if no
MPI-wrappers are used
- use llnl.util.filesystem.find_libraries instead of a locally
defined routine to guess the MPI libs if needed
(cf. #8979)
* zoltan: rely on MPI-wrappers to know the required MPI libs
* This fixes a number of bugs:
* Patches were not properly downloaded and added to mirrors.
* Mirror create didn't respect `list_url` in packages
* Update the `spack mirror` command to add all packages in the
concretized DAG (where originally it only added the package specified
by the user). This is required in order to collect patches that are specified
by dependents. Example:
* if X->Y and X requires a patch on Y called Pxy, then Pxy will only
be discovered if you create a mirror with X.
* replace confusing --one-version-per-spec option for `spack mirror create`
with --versions-per-spec; support retrieving multiple versions for
concrete specs
* Implementation details:
* `spack mirror create` now uses regular staging logic to download files
into a mirror, instead of reimplementing it in `add_single_spec`.
* use a separate resource caching object to keep track of new
resources and already-existing resources; also accepts storing
resources retrieved from a cache (unlike the local cache)
* mirror cache object now stores resources that are considered
non-cachable, like (e.g. the tip of a branch);
* the 'create' function of the mirror module no longer traverses
dependencies since this was already handled by the 'mirror' command;
* Change handling of `--no-checksum`:
* now that 'mirror create' uses stages, the mirror tests disable
checksums when creating the mirror
* remove `no_checksum` argument from library functions - this is now
handled at the Spack-command-level (like for 'spack install')
* Store ccache directory explicitly in Travis.
Despite we started using ccache on `develop`, it seems the cache itself
is not stored from one CI build to the next. This might be du to the
fact that our language on Travis is Python and not C nor C++.
Hence here we store the ccache directory explicitly.
Lately many CI runs for PRs are failing due to the `mpich` build that
times out on Travis (10 mins. without output). As the timeout seems to
happen consistently during the build phase, increasing the verbosity of
that test can help working around the issue.
The sys_ustat.h.patch to file sanitizer_platform_limits_posix.cc from
PR #10046 does not apply cleanly to gcc 4.8 or 4.9 (or earlier).
GCC up to 4.8.x either don't have libsanitizer or else don't include
ustat.h in sanitizer_platform_limits_posix.cc.
GCC 4.9.x includes ustat.h, but needs a slightly different patch.
The patch applies to GCC 5.x up to 6.4, and 7.x up to 7.3 and also
8.1.0.
The patch is already included in the tar files for gcc 6.5.0, 7.4.0
and 8.2.0.
* flux-sched: fix compilation errors
configure was being run twice becuase the staging symlinks were
confusing autotools
mirror flux-core in building in a subdirectory to avoid other common
compilation problems, like the one in PR #3543
* flux-core: update dependencies
munge is no longer a dependency
provide hints for lua version so that `spack install flux-core` works
"out-of-the-box". Currently, when no explicity version is provided, the
concretizer attempts to use lua@5.3, which is invalid. Closes#10000
* Added a package for the MDAnalysis toolkit.
* Added Ninja as the generator for LBANN, Hydrogen, and Aluminum. Also
fixed a bug in how the Conduit variant is included in LBANN. Added
new versions for Hydrogen and LBANN.
* Updated lbann package so that the CMAKE parameters are set when the
variant is on and off.
Fix the install of the single header amalgate in catch2.
The public API for catch < 2.3.0 was "#include <catch.hpp>".
For 2.3.0+ it is "#include <catch2/catch.hpp>"
* Spades: Add version 3.13.0
In version 3.13.0 the CmakeLists.txt dir is at a different location than
in other versions. Versions up to 3.12.0 are located in "src" whereas
3.13.0 is a level deeper at "assemblers/src".
* Remove git and add new version hash
- all multimethod tests are now run for both `multimethod` and
`multimethod-inheritor`
- do this with a parameterized fixture (pkg_name) that runs the same
tests on both
- Since early Spack versions, the SpecParser has (weirdly) been
responsible for initializing Spec fields.
- This refactors initialization to take place in Spec.__init__, as it
probably should have originally.
- This makes the code easier to read, the parser easier to understand,
and removes the use of __new__ in the parser to initialize the Spec.
- This also makes it possible to make a completely empty Spec with
`Spec()` -- this is an abstract Spec that will match anything.
* Add mising QE releases.
* Update QE dependencies and conflicts.
* QE 6.3 install method requires a patch to work properly.
* include QE git develop branch
* QE HDF5 needs Fortran support. HDF5 versions prior to 1.8.16 leads to QE runtime errors.
* Number of MKL related conflicts for QE.
* Flake8 fixes for QE for hdf5 lines.
* Pass in all compilers properly to QE configure
* Fixed external BLAS/LAPACK linkage in QE.
* Library search was problematic in QE.
* Add section headings to QE package.
* External FFTW compatible libraries properly detected in QE.
* Flake8 fixes for QE
* QE configure defaults to False for ELPA, thus Spack package should as well.
* QE depends on FFTW3 provider instead of FFTW explictly.
* Primary QE repo is gitlab.com, not github.com. Github mirror lags behind gitlab site.
* Support both 6.3-backports branch and develop branch in QE.
* QE patch needed for proper MKL detection. MKL_ROOT is detected and preferred, still falls back to finding first MKL installation in /opt/intel.
* Use FFTW-API provider versioned interface for specify QE FFT dependency.
* QE FFTW conflicts no longer need thanks to FFTW-API provider versioned interface.
* Flake8 3.6.0 fix in support of https://github.com/spack/spack/pull/9853
* Fix QE conflict with ilp64 variant in support of https://github.com/spack/spack/pull/9853
* Remove custom url_for_version for QE package and use builtin url instead. In support of https://github.com/spack/spack/pull/9853
* Use FFTW builtin flags for QE instead of custom Python code. In support of https://github.com/spack/spack/pull/9853
* Fix typo in QE package comments. In support of https://github.com/spack/spack/pull/9853
* Future proof QE package patches in anticipation of QE >= 6.3.1. In support of https://github.com/spack/spack/pull/9853
* Adding version 2.0.12 of numactl.
* Adding new versions of hwloc.
* Adding NVML variant to hwloc.
* Adding gl variant to hwloc.
* Updating hwloc numactl dependency and gl dependency.
* Using gl virtual package in hwloc.
Clang 8.0 now supports openmp 4.5 and many commerical compilers also support
openmp >= 4.0. This commit removes the openmp version guard and rely on CMake
to throw errors out when openmp version requirement is not satisified.
It also add options for precision control.
* CGNS: Add option for 64-bit integers
Added the `int64` variant which will build the library using 64-bit integers for certain values. This gives the capability to have models with more than 2 billion cells and/or nodes.
Beginning with CGNS-3.1.0, two new typedef variables have been introduced to support 64-bit mode. The `cglong_t` typedef is always a 64-bit integer, and `cgsize_t` will be either a 32-bit or 64-bit integer depending on how the library was built. Many of the C functions in the MLL have been changed to to use `cgsize_t` instead of `int` in the arguments. These functions include any that may exceed the 2Gb limit of an` int`, e.g. zone dimensions, element data, boundary conditions, and connectivity. In Fortran, all integer data is taken to be `integer*4` for 32-bit and `integer*8` for 64-bit builds.
* EXODUS: Update version and dependencies
Allow pulling master version from git. Note that I hope to have better versioning "soon"
Add requirement to use Netcdf-4.6.1 or later as those versions no longer require changing the `maxdims` or `maxvars` defines in the netcdf.h include file. With these versions, Exodus can use an unmodified NetCDF. These versions of NetCDF are also better than previous versions (code quality, performance, robustness).
It is still possible to run exodusII with previous NetCDF versions, but I don't know how to specify that the dependency where maxdims/maxvars variants are only needed for versions prior to 4.6.0.
I also removed the explicit dependency on HDF5 as that dependency should be applied through the NetCDF dependency.
* Remove white-space on blank line
* Provide a new spackage for py-pint.
Pint is a Python package to define, operate and manipulate physical quantities:
the product of a numerical value and a unit of measurement. It allows arithmetic
operations between them and conversions from and to different
units. https://pint.readthedocs.io/en/latest/
* Switch hash from md5 to sha256.
Add several new versions so that current functionality can be accessed. Current version is now 1.5.13.
Update `url`, `list_url`, and `list_depth` so spack can find available versions
* PNETCDF: Update with current versions
Added latest versions. (1.10.0, 1.9.0, 1.8.1)
* PNETCDF: Revert version change in url
Reverted the change in the url variable version since not required.
* NETCDF: Add new versions and update url
Added the 4.6.2 version which was recently released.
Changed the url to point to the official Unidata site and added option to retrieve master version from git.
* NETCDF: checksum and version updates
Reverted back to use of previous URL. The `https://github.com/Unidata/netcdf-c/archive/v4.6.2.tar.gz` location is newer, but spack does not seem to be able to access all of the versions at that location -- only goes back to 4.5.0. Since several installations are still providing 4.4.0, thought it best to go back to previous url. However, it looks like the naming convention has changed since 4.6.2, so need to provide explicit url for 4.6.2. Probably best to go to a `url_for_version()` scheme if that naming convention stays in place?
This commit allows users to enable openmp, rendering, logging and mpi
in vtkm. Meanwhile, it adds support for cuda architecture selection.
[VTK-m 1.3 Release notes](https://gitlab.kitware.com/vtk/vtk-m/tags/v1.3.0)
Overhaul hpctoolkit to AutotoolsPackage and to build its dependencies
directly from spack instead of using hpctoolkit-externals.
Remove hpctoolkit-externals since it's no longer needed.
* py-jupyter-console: Add version 5.2.0
* py-jupyter-core: add version 4.4.0
* py-lxml: add version 4.2.5
* py-zmq: add version 17.1.2
* py-terminado: add version 0.8.1
* py-pexpect: add version 4.6.0
* py-pytables: add version 3.4.4
Fixes#994473c978d replaced a move with a copy, which cased rmdir to fail (as
it expects an empty directory). This switches to use `shutil.rmtree`
instead.
The latest OpenMPI release, v4.0.0, does not build with many GCC
variants. Since this is our default, a lot of users get hit.
Let's wait for some point releases.
* py-geopandas: New package, including packages for its dependencies
* Trailing whitespace removed, etc
* One more trailing whitespace
* Removed some extraneous dependencies. No idea why I thought I needed them, but it now builds fine without them.
* Added a variant +plotting, to include the dependencies required for plotting, as these are large-ish if not being used.
* Added explicit dependency on py-matplotlib
* r-geor: new package, and a couple of its dependencies
* Update to new copyright header
* Include the list_url for the R package archive dir
* Added explicit version ranges on dependencies, as per the relevant CRAN pages
* Correct dependency types in r-splancs.
* r-compositions: new package, including its dependencies
* Include the list_url for the R package archive dir
* Added explicit version ranges on dependencies, as per the relevant CRAN pages
Drop versions 8.1.x. These were > 5 years old, no other spack package
uses them, and they used autotools. Packages 'stat' and 'mitos' still
use 8.2.1, so we keep that.
Switch class Dyninst to CMakePackage.
Clean up how the versions depend on various packages. For example,
Dyninst used libelf up to 9.2, and then elfutils starting with 9.3.
This makes things easier going forward.
Add variant 'openmp' (default True) to enable openmp support for
ParseAPI. This was added in 10.0.0.
Add variant 'static' (default False) to also build static libraries.
This goes back to 9.1.0.
Disable cotire for the 9.3.x versions. Cotire has no real use in a
one-time build and can break parallel builds with both static and
shared libs.
* package: add asciidoctor gem
Faster, easier to use, less impossible to install correctly
implementation of asciidoc.
* ritual sacrifice of EOF whitespace to flake8
* openblas: enable parallel builds
* cp2k: enable parallel builds
* cp2k: fix building on multilib/Suse distros
use the actual directory path where files where installed to instead of
the default prefix+'/lib'
* cp2k: ensure we have a non-header-only libxsmm
* openblas: disable max num CPU detection on virtualized build
* cp2k: install data and set compiled-in DATA_DIR
* cp2k: make libxc an optional dependency (enabled by default)
* cp2k: link libint statically
* cp2k: declare statically linked library deps as type=build
* cp2k: add support for PGI compiler
* cp2k: rename smm=none to smm=blas for clarification
* cp2k: blacklist unsupported compilers
* cp2k: mark wannier90 a build-time dep since statically linked
* cp2k: make pexsi and elpa optional
* cp2k: add support for v6.1
* libxc: add version 4.2.3
* cp2k: use pkg-config to link properly to libxsmm
* cp2k: fix OpenMP support by making it explicit
Previously, CP2K accepted threaded ELPA or BLAS, leading to #(CPU) processes
being spawned even though no explicit OpenMP was requested. Now the
`popt` variant should truly be thread free while the `psmp` variant uses
threads also internally.
* cp2k: source tarballs moved to GitHub
* Add initial packages for citcoms and dependencies
* Ignore Pythia imports
* Add py-merlin package
* Add GMT package
* Fixes to get GMT working
* Get hc package building
* Get CitcomS 3.2.0 working
* Add fstrack package
* Fix hash
* added develop version in argobots package and added valgrind variant
* made autotools required only for develop version
* corrected style
* Added --disable-valgrind
Added --disable-valgrind to explicitly disable it when +valgrind isn't provided.
Pass cflags to configure so that configure gets the values from the
spack install line.
Disable -Werror so that we don't fail the build over a stray warning.
Libunwind releases are few and far between, so we need access to
recent snapshots from the git repository.
Pass cflags to configure so that make will respect the value from the
spack install line.
glib should not use the globally installed gtk-doc. Otherwise, gtk-doc
can fail with Python errors such as `ImportError: No module named site`.
This is due to the fact that Spack sets `PYTHONHOME`, which can confuse
the global Python installation that is used by gtk-doc.
- Add version 2.1.0
- Now that #9264 has been merged, gplates seems to build fine with newer
versions of gcc (tested with gcc@8)
- When using the ninja generator, gplates can be built in parallel
@@ -30,7 +30,7 @@ Project maintainers have the right and responsibility to remove, edit, or reject
## Scope
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the Spack project or its community. Examples of representing the project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of the project may be further defined and clarified by Spack maintainers.
about: Report a bug in the core of Spack (command not working as expected, etc.)
labels: bug
---
@@ -24,10 +24,10 @@ If Spack reported an error, provide the error message. If it did not report an e
but the output appears incorrect, provide the incorrect output. If there was no error
message and no output but the result is incorrect, describe how it does not match
what you expect. To provide more information you might re-run the commands with
the additional -sd flags:
the additional -d/--stacktrace flags:
```console
$ spack -sd <command1> <spec>
$ spack -sd <command2> <spec>
$ spack -d --stacktrace <command1> <spec>
$ spack -d --stacktrace <command2> <spec>
...
```
that activate the full debug output.
@@ -46,4 +46,4 @@ We encourage you to try, as much as possible, to reduce your problem to the mini
If you want to ask a question about the tool (how to use it, what it can currently do, etc.), try the `#general` channel on our Slack first. We have a welcoming community and chances are you'll get your reply faster and without opening an issue.
Other than that, thanks for taking the time to contribute to Spack!
Other than that, thanks for taking the time to contribute to Spack!
- Simplified package syntax for specifying build/run environment modifications
- Numerous improvements to support for environments across Spack commands
- Concretization improvements
## Documentation
- Multi-lingual documentation (Started a Japanese translation)
- Tutorial now has its own site at spack-tutorial.readthedocs.io
- This enables us to keep multiple versions of the tutorial around
## Deprecations
- Spack no longer supports dotkit (LLNL's homegrown, now deprecated module tool)
-`spack build`, `spack configure`, `spack diy` deprecated in favor of
`spack dev-build` and `spack install`
## Important package changes
- 3,563 total packages (718 added since 0.12.1)
- Spack now defaults to Python 3 (previously preferred 2.7 by default)
- Much improved ARM support thanks to Fugaku (RIKEN) and SNL teams
- Support new special versions: master, trunk, and head (in addition to develop)
- Better finding logic for libraries and headers
# v0.12.1 (2018-11-13)
This is a minor bugfix release, with a minor fix in the tutorial and a `flake8` fix.
Bugfixes
* Add `r` back to regex strings in binary distribution
* Fix gcc install version in the tutorial
# v0.12.0 (2018-11-13)
## Major new features
- Spack environments
-`spack.yaml` and `spack.lock` files for tracking dependencies
- Custom configurations via command line
- Better support for linking Python packages into view directories
- Packages have more control over compiler flags via flag handlers
- Better support for module file generation
- Better support for Intel compilers, Intel MPI, etc.
- Many performance improvements, improved startup time
## License
- As of this release, all of Spack is permissively licensed under Apache-2.0 or MIT, at the user's option.
- Consents from over 300 contributors were obtained to make this relicense possible.
- Previous versions were distributed under the LGPL license, version 2.1.
## New packages
Over 2,900 packages (800 added since last year)
Spack would not be possible without our community. Thanks to all of our
[contributors](https://github.com/spack/spack/graphs/contributors) for the
new features and packages in this release!
# v0.11.2 (2018-02-07)
This release contains the following fixes:
* Fixes for `gfortran` 7 compiler detection (#7017)
* Fixes for exceptions thrown during module generation (#7173)
# v0.11.1 (2018-01-19)
This release contains bugfixes for compiler flag handling. There were issues in `v0.11.0` that caused some packages to be built without proper optimization.
Fixes:
* Issue #6999: FFTW installed with Spack 0.11.0 gets built without optimisations
Includes:
* PR #6415: Fixes for flag handling behavior
* PR #6960: Fix type issues with setting flag handlers
* 880e319: Upstream fixes to `list_url` in various R packages
# v0.11.0 (2018-01-17)
Spack v0.11.0 contains many improvements since v0.10.0.
Below is a summary of the major features, broken down by category.
## New packages
- Spack now has 2,178 packages (from 1,114 in v0.10.0)
- Many more Python packages (356) and R packages (471)
- 48 Exascale Proxy Apps (try `spack list -t proxy-app`)
to learn how to use spack, write packages, or deploy packages for users
at your site.
Tutorial
----------------
See also:
* [Technical paper](http://www.computer.org/csdl/proceedings/sc/2015/3723/00/2807623.pdf) and
[slides](https://tgamblin.github.io/files/Gamblin-Spack-SC15-Talk.pdf) on Spack's design and implementation.
* [Short presentation](https://tgamblin.github.io/files/Gamblin-Spack-Lightning-Talk-BOF-SC15.pdf) from the *Getting Scientific Software Installed* BOF session at Supercomputing 2015.
==> pkg-config is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/pkg-config-0.29.2-ae2hwm7q57byfbxtymts55xppqwk7ecj
...
==> openmpi is already installed in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f
==> Installing arpack-ng
==> Using cached archive: /usr/local/var/spack/cache/arpack-ng/arpack-ng-3.5.0.tar.gz
==> Already staged arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un in /usr/local/var/spack/stage/arpack-ng-3.5.0-bloz7cqirpdxj33pg7uj32zs5likz2un
==> No patches needed for arpack-ng
==> Building arpack-ng [Package]
==> Executing phase: 'install'
==> Error: RuntimeError: Unable to recursively locate netlib-lapack libraries in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-lapack-3.6.1-jjfe23wgt7nkjnp2adeklhseg3ftpx6z
RuntimeError: RuntimeError: Unable to recursively locate netlib-lapack libraries in /usr/local/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/netlib-lapack-3.6.1-jjfe23wgt7nkjnp2adeklhseg3ftpx6z
/usr/local/var/spack/repos/builtin/packages/arpack-ng/package.py:105, in install:
==> Warning: Spack was unable to fetch url list due to a certificate verification problem. You can try running spack -k, which will not check SSL certificates. Use this at your own risk.
checking dependency style of /home/spack1/spack/lib/spack/env/gcc/g++... gcc3
checking for /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc... /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc
Checking whether /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc responds to '-showme:compile'... yes
configure: error: unable to locate adept-utils installation
This gives us the output from the build, and mpileaks isn't
finding its ``adept-utils`` package. Spack has
automatically added the include and library directories of
``adept-utils`` to the compiler's search path, but some packages like
mpileaks can sometimes be picky and still want things spelled out on
their command line. But let's continue to pretend we're not brilliant
developers, and explore some other debugging paths:
We can also enter the build area and try to manually run the build:
..code-block::console
$ spack build-env mpileaks bash
$ spack cd mpileaks
The ``spack env`` command spawned a new shell that contains the same
environment that Spack used to build the mpileaks package (you can
substitute bash for your favorite shell). The ``spack cd`` command
changed our working dirctory to the last attempted build for mpileaks.
From here we can manually re-run the build:
..code-block::console
$ ./configure
checking metadata... no
checking installation directory variables... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gcc... /home/spack1/spack/lib/spack/env/gcc/gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking dependency style of /home/spack1/spack/lib/spack/env/gcc/g++... gcc3
checking for /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc... /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc
Checking whether /home/spack1/spack/opt/spack/linux-ubuntu16.04-x86_64/gcc-5.4.0/openmpi-3.0.0-yo5qkfvumpmgmvlbalqcadu46j5bd52f/bin/mpicc responds to '-showme:compile'... yes
configure: error: unable to locate adept-utils installation
We're seeing the same error, but now we're in a shell where we can run
the command ourselves and debug as needed. We could, for example, run
``./configure --help`` to see what options we can use to specify
dependencies.
We can use the ``exit`` command to leave the shell spawned by ``spack
env``.
------------------------------
Specifying Configure Arguments
------------------------------
Let's add the configure arguments to the mpileaks' ``package.py``. This
There are some special circumstances in package that are worth highlighting.
Normally spack would have automatically detected that mpileaks was an
Autotools-based package when we ran ``spack create`` and made it an ``AutoToolsPackage`` class (except we added the ``-t generic`` option to skip this). Instead of
a full install routine we would have just written:
The entries in the projections configuration file must all be either
specs or the keyword ``all``. For each spec, the projection used will
be the first non-``all`` entry that the spec satisfies, or ``all`` if
there is an entry for ``all`` and no other entry is satisfied by the
spec. Where the keyword ``all`` appears in the file does not
matter. Given the example above, any spec satisfying ``zlib@1.2.8``
will be linked into ``/my/view/zlib-1.2.8/``, any spec satisfying
``hdf5@1.8.10+mpi %gcc@4.9.3 ^mvapich2@2.2`` will be linked into
``/my/view/hdf5-1.8.10/mvapich2-2.2-gcc-4.9.3``, and any spec
satisfying ``hdf5@1.8.10~mpi %gcc@4.9.3`` will be linked into
``/my/view/hdf5-1.8.10/gcc-4.9.3``.
If the keyword ``all`` does not appear in the projections
configuration file, any spec that does not satisfy any entry in the
file will be linked into the root of the view as in a single-prefix
view. Any entries that appear below the keyword ``all`` in the
projections configuration file will not be used, as all specs will use
the projection under ``all`` before reaching those entries.
""""""""""""""""""
Fine-Grain Control
""""""""""""""""""
@@ -1437,4 +1494,3 @@ Disadvantages:
2. Although patches of a few lines work OK, large patch files can be
hard to create and maintain.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
