* Bump the build cache layout version from 1 to 2
* Version to lists parent directories of the prefix in the tarball too, which is required from some container runtimes
* Move in vs. satisfies to a note and mention special cases of in
* Address feedback: oveoverlap -> intersect
* Re-word the satisfies versus in note.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This fixes an issue where pkg.stage throws because a patch cannot be found,
but the patch is redundant because the spec is reused from a build cache and
will be installed from existing binaries.
Part 4 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ provided_spec: {when_spec, ...} }
to this:
{ when_spec: {provided_spec, ...} }
`make_when_spec()` was being used in the solver, but it has semantics that are specific
to parsing when specs from `package.py`. In particular, it returns `None` when the
`when` spec is `False`, and directives are responsible for ignoring that case and not
adding requirements, deps, etc. when there's an actual `False` passed in from
`package.py`.
In `asp.py`, we know that there won't ever be a raw boolean when spec or constraint, so
we know we can parse them without any of the special boolean handling. However, we
should report where in the file the error happened on error, so this adds some parsing
logic to extract the `mark` from YAML and alert the user where the bad parse is.
- [x] refactor `config.py` so that basic `spack_yaml` mark info is in its own method
- [x] refactor `asp.py` so that it uses the smarter YAML parsing routine
- [x] refactor `asp.py` so that YAML input validation for requirements is done up front
Part 3 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ (requirement_spec, ...): [(when_spec, policy, msg)] }
to this:
{ when_spec: [((requirement_spec, ...), policy, msg), ...] }
Part 2 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ conflict_spec: [(when_spec, msg), ...] }
to this:
{ when_spec: [(conflict_spec, msg), ...] }
Also attempts to consistently name the variables used to iterate over conflict
dictionaries.
Part 1 of making all package metadata indexed by `when` condition. This
will allow us to handle all the dictionaries on `PackageBase` consistently.
Convert the current dependency dictionary structure from this:
{ name: { when_spec: [Dependency ...] } }
to this:
{ when_spec: { name: [Dependency ...] } }
On an M1 mac, this actually shaves 5% off the time it takes to load all
packages, I think because we're able to trade off lookups by spec key
for more lookups by name.
* Boost: add version 1.84.0
* Conflict with 98/03
* Set C++11 as default
Starting with 1.84.0, the minimum required is c++11. It has been a very
long time since 98/03 has been required. It's time to bump the minimum.
* Add OpenMPI 5.0.0/5.0.1 release
* Fix a problem with dlopen syms with 5.0.0
* Crank up lex buffer to 1MB so that Open MPI's compiler wrapper can parse the enormously long lines present in, for example, mpicc-wrapper-data.txt when the spack install is utilizing Spack's path padding feature.
* Disable romio by default for 5.0.0 and beyond owing to problems compiling the romio package when using the Intel OneAPI compiler.
* Patch for addiing cuda lib location in case of non-standard location of libcuda.so
* build accel components as DSOs. It appears from looking at some of the spack CI that it implicitly assumes that Open MPI is built with components as DSOs. The default behavior for Open MPI was changed between the 4.1.x release stream and the 5.0.x release stream changed and this premise is now incorrect.
Turns out that starting with Open MPI 5.0.0 building static
does not work when using a now very important variant, namely cuda.
In older versions of Open MPI the libcuda.so was dlopened at
run time when needed, but now libcuda is linked in to the cuda
components of openmpi directly. This works when using Open MPI's
dynamically loadable component option, but doesn't work now for
a lot of the Spack CI pipelines because they don't include libcuda.so
in LD_LIBRARY_PATH of packages that dont think they are using
cuda themselves.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Co-authored-by: Jack Morrison <jack.morrison@cornelisnetworks.com>
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Needed for #40326, which can changes the iteration order over package dependencies during concretization.
While clingo doesn't have this problem, the original concretizer (which we still use for bootstrapping) can be sensitive to iteration order when evaluating dependency constraints in `when` conditions. This can cause it to ignore conditional dependencies unless the dependencies in the condition are listed first in the package.
The issue was in the way the original concretizer would disconnect specs *every* time `normalize()` ran. When specs were disconnected, `^dependency` constraints wouldn't see the dependency in the dependency condition loop.
We now only only disconnect *all* dependencies at the start of `concretize()` and `normalize()`, and we disconnect any leftover dependents from replaced externals at the *end* of `normalize()`. This trims stale connections while keeping the ones that are needed to trigger dependency conditions.
- [x] refactor `flat_dependencies()` to not disconnect the spec by default.
- [x] `flat_dependencies()` is never called with `copy=True` -- remove the `copy` kwarg.
- [x] disconnect only once at the beginning of `normalize()` or `concretize()`.
- [x] add a test that perturbs dependency iteration order to ensure this doesn't regress.
- [x] disconnect unused dependents at end of `normalize()`
* setup-env: Fix back and forth between two instances
* setup-env.csh: Fix SPACK_ROOT when switch to a different instance
i.e. Always look for the current SPACK_ROOT
* setup-env: Update comments
This adds options to `spack list` that allow you to list only packages from specific
repositories/namespaces, e.g.:
```console
spack list -r builtin
```
only lists packages from the `builtin` repo, while:
```console
spack list -r myrepo -r myrepo2
```
would list packages from `myrepo` and `myrepo2`, but not from `builtin`. Note that you
can use the same argument multiple times.
You can use either `-r` / `--repo` or `-N` / `--namespace`. `-N` is there to match the
corresponding option on `spack find`.
- [x] add `-r` / `--repo` / `-N` / `--namespace` argument
- [x] add test
* Add rust-analyzer as variant to rust build
* Expose cargo module only when +cargo
* rust: add v1.74.0 and v1.75.0 and remove variants in favor of +dev
* [@spackbot] updating style on behalf of alecbcs
* Fix variant typo
---------
Co-authored-by: alecbcs <alecbcs@users.noreply.github.com>
This method is vestigial; the only arg we ever used was `ignore=`, and that was
eliminated in #29317 and #35588.
The `kwargs` field of the extensions dictionary is actually completely unused now. Add a
note for future removal.
Literal compiler config in `test_requires_directive` specifically lists `target:
x86_64`, but it doesn't need to, and the unnecessary target makes the test fail on
non-`x86_64` machines.
- [x] Remove target from config yaml in `test_requires_directive`
* py-torch: set env OpenBLAS_HOME
Because [`FindOpenBLAS.cmake`](https://github.com/pytorch/pytorch/blob/main/cmake/Modules/FindOpenBLAS.cmake) uses a hardcoded list of search paths for includes and libraries, we have to pass the `OpenBLAS_HOME` environment variable.
* py-torch: patch for ${OpenBLAS_HOME}/include/openblas
The context of this patch is unchanged since v0.4.0.
* py-torch: move patch before def patch
* py-torch: also set Atlas_ROOT_DIR and BLIS_HOME
* py-torch: fix openblas patch range to @:2.1
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Xorg pkgs: updated version to current latest
This updates the versions of multiple Xorg packages to their current
latest version. Verified the requirements and version dependencies, and
updated where needed. Modified one homepage (xkbcomp) in the interest of
conformity with the other packages.
Summary of dependency changes:
- libsm:
- depends_on("libice@1.1.0:", when="@1.2.4:")
- libx11:
- depends_on("libxcb@1.11.1:", when="@1.6.4:")
- libxcomposite:
- depends_on("xproto@7.0.22:", when="@0.4.6")
- libxfixes:
- depends_on("fixesproto@5.0:", when="@5")
- depends_on("fixesproto@6.0:", when="@6")
- libxi:
- depends_on("inputproto@2.2.99.1:", when="@1.7:")
- depends_on("inputproto@2.3.99.1:", when="@1.8:")
* xcb-proto, libxcb: new version 1.15
* xorg libs: additional new versions
New minor version upgrades:
- libXcursor (no changed needed)
- libXres
- depends_on("resourceproto@1.0:", when="@1.0")
- depends_on("resourceproto@1.2:", when="@1.2")
* libxpm: ... depends_on ncompress only when 3.5.15
* Xorg libs: add maintainer
* xtrans: new version 1.5.0
* xcb-proto: new version 1.16.0
* libxt: new version 1.3.0
* libxrandr: new version 1.5.4
* libxpm: new versions 3.5.16, 3.5.17
* libxi: new version 1.8.1
* libxft: new version 2.3.8
* libxfixes: new version 6.0.1
* libxcb: new version 1.16
* libx11: new version 1.8.5, 1.8.6, 1.8.7
* libxfixes: comment out problematic fixesproto versions
* libxi: comment out problematic inputproto versions
* libxfixes, libxi: add reference to issue that blocks updates
* patch on variant fftw3; fix for #41577
* a line of doubtful blas/fftw +openmp. Are they needed with nwchem+openmp?
* Update var/spack/repos/builtin/packages/nwchem/package.py
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
---------
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
* gaudi: add a patch for catch2
* Fix indentation
* Add a diff at the end of the path
* gaudi: canonicalize patch url
* gaudi: canonicalize patch url for gitlab diff
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
PAPI CI checks a `spack install` of `papi@master`, and the open range here breaks their
CI with the fix because the patch is no longer needed (see #26784, #27625 for why it's
difficult to avoid this).
The patch issue is going to be fixed in PAPI upstream with whatever release is after
`7.1.0`, so we can restrict the patch to `7.1.0` and avoid this issue.
* shell: fix zsh color formatting for PS1 in environments
The `colorize` function in `llnl.util.tty.color` only applies proper formatting for Bash
ANSI and for console output, but this is not what zsh expects for environment variables.
In particular, when using `zsh`, `spack env activate -p` produces a `PS1` prompt that
looks like this:
```
\[\033[0;92m\][ENVIRONMENT]\[\033[0m\]
```
For zsh the formatting should be:
```
\e[0;92m[ENVIRONMENT]\e0;m
```
- [x] Add a `zsh` option to `colorize()` to enable zsh color formatting
- [x] Add conditional to choose the right `PS1` for `zsh`, `bash`, and `sh`
- [x] Don't use color escapes for `sh`, as they don't print properly
* convert lots of += lines to triple quotes
No online release notes or repository, but the new tarball has the following in `NEWS`:
* ThePEG-2.3.0 release: 2023-12-11
** gcc-12/c++17/c++20 compatibility added
** hepmc3 compatibility added
** rivet interface improved
** inforstructure for dark interaction added
PAPI 7.1.0 unconditionally adds `FFLAGS = -ffree-form` in the sysdetect tests,
regardless of the compiler.
This was added in https://github.com/icl-utk-edu/papi/pull/108 to make a build with
`armflang` work, but it breaks CCE (and our `develop` pipeline).
- [x] Add a patch that fixes both problems
- [x] Patch PAPI when at 7.1.0 or higher
Add a "checked_by" field to the `license()` directive so that we can track who verified
the license for a project. also check the license of 18 or so projects and mark them
checked.
This patch adds license information for about 5,300 packages from automated sources.
The license information was obtained from Alpine Linux and PyPI and processed
using tooling available in https://github.com/boomanaiden154/spack-license-utils.
The license field was added in after all other directives in an automated fashion.
Note that while this license information is probably fairly accurate, it is not
guaranteed to be accurate. In addition some of the license strings from Alpine Linux
might not be valid SPDX license strings. Invalid SPDX identifiers can be picked up
and fixed once we have validation/parsing infrastructure in place for the solver,
and issues can be fixed as they come up.
This adds a few options to `spack gc`.
One to give you a little more control over dependencies:
* `-b` / `--keep-build-dependencies`: By default, `spack gc` considers build dependencies to be "no longer needed" once their dependents are installed. With this option, we'll keep build dependencies of needed installations as well.
And two more to make working with environments easier:
* `-E` / `--except-any-environment`: Garbage collect anything NOT needed by an environment. `spack gc -E` and `spack gc -bE` are now easy ways to get rid of everytihng not used by some environment.
* `-e` / `--except-environment` `ENV`: Instead of considering all environments, garbage collect everything not needed by a *specific* environment. Note that you can use this with `-E` to add directory environments to the list of considered envs, e.g.:
spack gc -E -e /path/to/direnv1 -e /path/to/direnv2 #...
- [x] rework `unused_specs()` method on DB to add options for roots and deptypes
- [x] add `all_hashes()` method on DB
- [x] rework `spack gc` command to add 3 more options
- [x] tests
* Add all versions back to 0.20, add more depends_on (flex, bison, libffi and ccache), add the ability to enable or disable both abc and ccache, abc is enabled by default, ccache is disabled by default
* Fixed style with black
* Removed unused f-string setups
* Fixed style with black (again)
* Add enum34, numdifftools, and updated pyomo packages
* Syntax error
* Apply black style
* Trying to get around Python spec issue
* All SHAs were somehow wrong
* Change enum version
* Change optional dependencies to be on run, not build
* Add Pyomo 6.7.0
* Update SHA and version mismatch
* Remove py-enum34
* Add three new packages to address comments
* Fix linting errors; move casadi to py-casadi
* Update license; add in dependency
* Update setuptools version
* Update class name to python class
* Remove other boielerplate stuff
* Update homepage addresses; update py-casadi
message: "If you are referencing Spack in a publication, please cite the paper below."
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
abstract: >-
Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools.
Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible.
However, managing many configurations is difficult because the configuration space is combinatorial in size.
We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages and dependencies.
It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment.
We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.
Large HPC centers spend considerable time supporting software for thousands of users, but the
complexity of HPC software is quickly outpacing the capabilities of existing software management
tools. Scientific applications require specific versions of compilers, MPI, and other dependency
libraries, so using a single, standard software stack is infeasible. However, managing many
configurations is difficult because the configuration space is combinatorial in size. We
introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages
and dependencies. It allows any number of builds to coexist on the same system, and it ensures
that installed packages can find their dependencies, regardless of the environment. We show
through real-world use cases that Spack supports diverse and demanding applications, bringing
order to HPC software chaos.
preferred-citation:
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
type: conference-paper
@@ -71,7 +75,7 @@ preferred-citation:
type: doi
value: 10.1145/2807591.2807623
- description: "The DOE Document Release Number of the work"
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
