Fixes#49403.
When one scope included another, we were appending to a list stored on the scope to
track what was included, and we would clear the list when the scope was removed.
This assumes that the scopes are always strictly pushed then popped, but the order can
be violated when serializing config scopes across processes (and then activating
environments in subprocesses), or if, e.g., instead of removing each scope we simply
cleared the list of config scopes. Removal can be skipped, which can cause the list of
includes on a cached scope (like the one we use for environments) to grow every time it
is pushed, and this triggers an assertion error.
There isn't actually a need to construct and destroy the include list. We can just
compute it once and cache it -- it's the same every time.
- [x] Cache included scope list on scope objects
- [x] Do not dynamically append/clear the included scope list
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Right now the Spack %msvc compiler is inherently a hybrid compiler
that uses Intel's oneAPI fortran compiler.
This was addressed in Spacks MSVC compiler class, but detection has
since stopped using the compiler class, so this PR moves the logic
into the `msvc` compiler package (does not delete the original code
because that is handled in #45189).
This includes a change to the general detection logic to deprioritize
paths that include a symlink anywhere in the path, in order to prefer
"2025.0/bin" over "latest/bin" for the oneAPI compiler.
* style.py: add spack style --spec-strings for compat with v1.0
* add --fix also, and avoid infinite recursion and too large files
* tests: check identify and check edit files
* samurai: new package
- Add samurai : an HPC library of mesh and physics
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Remove Whitespace
- Remove whitespace for spack style check
* Update var/spack/repos/builtin/packages/samurai/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Add tags
- Add tags for the last versions of samurai
- All tags are tested and worked properly
- Add maintainers ("gouarin" - the samurai project lead and "sbstndb" - me, working on samurai)
- Add licence
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Trilinos launch blocking + maintainers
Cuda launch blocking is not needed and slowing modern apps down.
More maintainers to spot issues like this.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Elpa's custom preprocessor createst temporary files for which it
assembles long filenames and then uses the last 250 characters. This
results in compilation errors when the first character happens to be a
dash.
* Slurm: extend spack external find support
On Debian srun/salloc --version returns 'slurm-wlm VERSION'. Check for both strings and return the first match.
* non-capturing group for slurm determine_version
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* slurm: add detection test
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
This commit removes the +fortran variant when building HDF5 for WRF.
This seems unnecessary, and prevents building WRF with some versions of
Intel MPI, as HDF5 doesn't appear to build with Fortran support and
Intel MPI.
* add new versions up to 1.5 and new variants
variant vtk: make vtk optional
variant shared: build shared libs
added patch to fix parmmg cmake so that it can be used by other software with find_package
* use +private for mmg@5.8: and parmmg@1.5:
* fix style and constraint mmg version
* add a condition on patch, use private_headers from mmg PR feelpp/spack#14
* add MET v12.0.0 and METplus v6.0.0
* Set correct dependencies for metplus@6 in var/spack/repos/builtin/packages/metplus/package.py
* Add missing dependency on proj for met@12
* Add met@12.0.1
* Change @6.0.0 to @6: for requirements in var/spack/repos/builtin/packages/metplus/package.py
* Address reviewer comments for met and metplus
---------
Co-authored-by: Rick Grubin <Richard.Grubin@noaa.gov>
This should help resolve the "No binary found when cache-only was specified"
errors we've recently seen in our GitLab CI pipelines.
example failing job here:
https://gitlab.spack.io/spack/spack/-/jobs/15570931#L370
This error is caused when a generate job finds a spec in the local root
binary mirror, and that spec does not yet exist in the stack-specific mirror.
The fix here is to instead locally cache the stack-specific mirrors and only
use the root-level mirror for public use.
Windows paths with drives were being interpreted as network protocols
in canonicalize_path (which was expanded to handle more general URLs
in #48784).
This fixes that and adds some tests for it.
In Spack v1.0 we plan to parse caret ^ and percent % the same. Their meaning is direct and transitive dependency respectively. It means that variants, versions, arch, platform, os, target and dag hash should go before the %, so that they apply to dependent not the %dependency.
When requiring a constraint on a virtual package, it makes little
sense to use anonymous specs, and our documentation shows no example
of requirements on virtual packages starting with `^`.
Right now, due to how `^` is implemented in the solver, writing:
```yaml
mpi:
require: "^openmpi"
```
is equivalent to the more correct form:
```yaml
mpi:
require: "openmpi"
```
but the situation will change when `%` will shift its meaning to be a
direct dependency.
To avoid later errors that are both unclear, and quite slow to get to the user,
this commit makes anonymous specs under virtual requirements an error,
and shows a clear error message pointing to the file and line where the
spec needs to be changed.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Without it, the build fails with errors like this:
```
Can't locate File/Compare.pm in @INC (you may need to install the File::Compare module) (@INC contains: ...) at ../../../src/backend/catalog/Catalog.pm line 19.
```
Having variants all conditional leaves a lot more degree of freedom to clingo,
and slows down the search.
If variants have inconsistent defaults, we might end up with multiple, equally
sub-optimal solutions. Sometimes this creates a "plateau" in the search space.
Remove conditional boolean variants that can't be activated, since this just increases
the complexity of the model.
If 4 variants have to be all active / inactive together, it's better to use a single requires,
than to explode it into multiple statements dealing with a single variant at a time.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
ICU4C's NMAKE seems to over-quote to the degree
that it passes paths like ""<path>"" which
confuses the Python command line in subprocesses
the build starts
If you use `spack config change` to modify a `require:` section that did
not exist before, Spack was inserting the merged configuration into the
highest modification scope (which for example would clutter the
environment's `spack.yaml` with a bunch of configuration details
from the defaults).
Supersedes #46792.
Closes#40018.
Closes#31026.
Closes#2700.
There were a number of feature requests for os-specific config. This enables os-specific
config without adding a lot of special sub-scopes.
Support `include:` as an independent configuration schema, allowing users to include
configuration scopes from files or directories. Includes can be:
* conditional (similar to definitions in environments), and/or
* optional (i.e., the include will be skipped if it does not exist).
Includes can be paths or URLs (`ftp`, `https`, `http` or `file`). Paths can be absolute or
relative . Environments can include configuration files using the same schema. Remote includes
must be checked by `sha256`.
Includes can also be recursive, and this modifies the config system accordingly so that
we push included configuration scopes on the stack *before* their including scopes, and
we remove configuration scopes from the stack when their including scopes are removed.
For example, you could have an `include.yaml` file (e.g., under `$HOME/.spack`) to specify
global includes:
```
include:
- ./enable_debug.yaml
- path: https://github.com/spack/spack-configs/blob/main/NREL/configs/mac/config.yaml
sha256: 37f982915b03de18cc4e722c42c5267bf04e46b6a6d6e0ef3a67871fcb1d258b
```
Or an environment `spack.yaml`:
```
spack:
include:
- path: "/path/to/a/config-dir-or-file"
when: os == "ventura"
- ./path/relative/to/containing/file/that/is/required
- path: "/path/with/spack/variables/$os/$target"
optional: true
- path: https://raw.githubusercontent.com/spack/spack-configs/refs/heads/main/path/to/required/raw/config.yaml
sha256: 26e871804a92cd07bb3d611b31b4156ae93d35b6a6d6e0ef3a67871fcb1d258b
```
Updated TODO:
- [x] Get existing unit tests to pass with Todd's changes
- [x] Resolve new (or old) circular imports
- [x] Ensure remote includes (global) work
- [x] Ensure remote includes for environments work (note: caches remote
files under user cache root)
- [x] add sha256 field to include paths, validate, and require for remote includes
- [x] add sha256 remote file unit tests
- [x] revisit how diamond includes should work
- [x] support recursive includes
- [x] add recursive include unit tests
- [x] update docs and unit test to indicate ordering of recursive includes with
conflicting options is deferred to follow-on work
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Peter Scheibel <scheibel1@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
The SPACK_USER_CACHE_PATH was being overwritten in the windows CI
before_script. This should set the path for all systems unless
explicitly overridden.
* gasnet: deprecate old versions
GASNet versions more than 2 years old are not supported.
Update description text.
* gasnet: add 2025.2.0-snapshot version
Defines `spack.package_api_version` and `spack.min_package_api_version`
as tuples (major, minor).
This defines resp. the current Package API version implemented by this version
of Spack and the minimal Package API version it is backwards compatible with.
Repositories can optionally define:
```yaml
repo:
namespace: my_repo
api: v1.2
```
which indicates they are compatible with versions of Spack that implement
Package API `>= 1.2` and `< 2.0`. When the `api` key is omitted, the default
`v1.0` is assumed.
This package has not been maintained since 2016.
We maintain an active fork in the hydrogen
package, so remove this one.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* py-networkx: add new versions up to 3.4.2
* py-networkx: add more requirements
* py-networkx: fix typo
* py-networkx: fix python and py-setuptools dependencies
---------
Co-authored-by: Joseph C Wang <joequant@gmail.com>
* Update rpy2 to newest version and clean up package
* Add me as maintainer
* Update depends section as per review. Add ipython variant. Fix some ranges and add support for python 3.9. Deprecated outdated versions
* refine depends_on and remove redundant version info
* style
* Adding ability for repo paths from a manifest file to be expanded when creating an environment.
A unit test was added to check that an environment variable will be expanded.
Also, a bug was fixed in the expansion of develop paths where if an environment variable
was in the path that then produced an absolute path the path would not be extended.
* Fixing new unit test for env repo var substitution
* Adding ability for repo paths from a manifest file to be expanded when creating an environment.
A unit test was added to check that an environment variable will be expanded.
Also, a bug was fixed in the expansion of develop paths where if an environment variable
was in the path that then produced an absolute path the path would not be extended.
* Messed up resolving last rebase
* qwt: support building against Qt6
* qwt: fix style
* qwt: depends_on qt-base+opengl+widgets when +opengl
* visit: patch for missing cmath include
---------
Co-authored-by: Bernhard Kaindl <contact@bernhard.kaindl.dev>
* pull in new changes from axom project
* add new versions
* convert more conditionals to spec.satisfies
-------------
Co-authored-by: white238 <white238@users.noreply.github.com>
* rct: update packages (RE, RG, RP, RS, RU) with new version 1.90
* radical: added `url_for_version` for older versions
* radical: set latest versions for `radical.pilot` and `radical.utils`
* radical: fixed `url_for_version` setup
* radical: set the latest version for `radical.entk`
* radical: fixed style for `url_for_version`
* Apply suggestions from code review (python version dependency)
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
On Windows, libraries search their directory for dependencies, and
we help libraries in Spack-built packages locate their dependencies
by symlinking them into the dependent's directory (we refer to this
as simulated RPATHing).
We extend the convenience functionality here to support base library
directories outside of the package prefix: this is primarily for
running tests in the build directory (which is not located inside
of the final install prefix chosen by spack).
This release contains modifications to most of the SEACAS applications to support ChangeSets to some degree.
See https://github.com/SandiaLabs/seacas/wiki/Dynamic_Topology for information about Change Sets and
See https://github.com/SandiaLabs/seacas/wiki/Supporting-Change-Sets for information about how the various seacas applications are supporting the use or creation of change sets.
The release also includes various other small changes including formatting, portability, intallation, TPL version updates, and spelling.
* add sendme package
* style fix
* add docstring for test function
* changed maintainer string, run test after install
* removed redundant test
* Follow the common package license header format
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Split requirements to get better error messages in case of unsat solves.
* use list requirements instead of string
* activate static_analysis in a few pipelines
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* ci: darwin stacks: update tags following system updates
* disable SPACK_CI_DISABLE_STACKS; only enable *darwin* stacks for testing
* manually chmod u+w tmp/ before cleanup due to issue#49147
* comment out failing specs for now
* re-enable logic for disabling stacks
* add explanatory comment for darwin after_script additions
* remove more darwin-only targetting
* restore build_stage to default location
* move build-job-remove out of individual darwin stacks into darwin top level config
* keep build_stage in $spack/tmp for now
Python was removed from being a build tool in #46980, due to issues
when reusing specs. This PR adds a new rule to match the interpreter
among different Python packages, in clingo.
It also adds a bunch of new "build-tools", so that specs like:
```
py-matplotlib backend=tkagg
```
can be concretized in one go.
Modifications:
- [x] Make `py-matplotlib backend=tkagg` concretizable
- [x] Add unit-tests to ensure situations like in #46980 do not happen
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Currently, the custom config scopes are pushed at the top when constructing
configuration, and are demoted whenever a context manager activating an
environment is used - see #48414 for details. Workflows that rely on the order
in the [docs](https://spack.readthedocs.io/en/latest/configuration.html#custom-scopes)
are thus fragile, and may break
This PR allows to assign priorities to scopes, and ensures that scopes of lower priorities
are always "below" scopes of higher priorities. When scopes have the same priority,
what matters is the insertion order.
Modifications:
- [x] Add a mapping that iterates over keys according to priorities set when
adding the key/value pair
- [x] Use that mapping to allow assigning priorities to configuration scopes
- [x] Assign different priorities for different kind of scopes, to fix a bug, and
add a regression test
- [x] Simplify `Configuration` constructor
- [x] Remove `Configuration.pop_scope`
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
All the build jobs in pipelines are apparently relying on the bug that was fixed.
The issue was not caught in the PR because generation jobs were fine, and
there was nothing to rebuild.
Reverting to fix pipelines in a new PR.
This reverts commit 3ad99d75f9.
VariantMap.concrete is unused, and would be incorrect if it were used
due to conditional variants.
Just let the Spec dictate what is concrete and what is not.
Currently, environments can end up with higher priority than `-C` custom
config scopes and `-c` command line arguments sometimes. This shouldn't
happen -- those explicit CLI scopes should override active environments.
Up to now configuration behaved like a stack, where scopes could be only be
pushed at the top. This PR allows to assign priorities to scopes, and ensures
that scopes of lower priorities are always "below" scopes of higher priorities.
When scopes have the same priority, what matters is the insertion order.
Modifications:
- [x] Add a mapping that iterates over keys according to priorities set when
adding the key/value pair
- [x] Use that mapping to allow assigning priorities to configuration scopes
- [x] Assign different priorities for different kind of scopes, to fix a bug, and
add a regression test
- [x] Simplify `Configuration` constructor
- [x] Remove `Configuration.pop_scope`
- [x] Remove `unify:false` from custom `-C` scope in pipelines
On the last modification: on `develop`, pipelines are relying on the environment
being able to override `-C` scopes, which is a bug. After this fix, we need to be
explicit about the unification strategy in each stack, and remove the blanket
`unify:false` from the highest priority scope
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Update survey package file with latest releases and python path settings for building with autoload none.
* Submitting reformatted file.
* update survey package file with libmonitor dependency changes, take out py-gpustat, and minor comment change.
* Trigger build.
Currently, we have `config:shared_linking:missing_library_policy` to error
or warn when shared libraries cannot be resolved upon install.
The new `spack verify libraries` command allows users to run this post
install hook at any point in time to check whether their current
installations can resolve shared libs in rpaths.
* remove preferred to allow seamless python@3.12 usage
* glib: remove deprecated versions
* glib: use extends because python-venv is pulled in from build deps and put into path
* dont patch patch versions, use new patch releases containing the fix instead
* restrict patch of shebangs, group relevant bits together
* simplify lowerbound
* fix pinned glib version
---------
Co-authored-by: Chris Marsh <chrismarsh.c2@gmail.com>
* Bug fix for compiling node-js@21: with gcc@11.2 (var/spack/repos/builtin/packages/node-js/package.py var/spack/repos/builtin/packages/node-js/wasm-compiler-gcc11p2.patch)
Since this bug fix is not sufficient, add a conflict for node-js@21: with gcc@11.2
* In var/spack/repos/builtin/packages/node-js/package.py, restrict patch wasm-compiler-gcc11p2.patch to versions 21:22 for gcc@11.2
* Revert "REVERTME: move celeritas changes to another branch"
This reverts commit a063e43aaf.
* Use predicted g4vg version
* Use
* fixup! Use predicted g4vg version
* Use spec for versions and improve dependency specification
* add new versions of dependencies
* modify pypi url for newest version
* add option for url depending on version number
* add version ranges of dependencies
* [@spackbot] updating style on behalf of kchilleri
* remove unnecessary py-cache-control version number
* spack: add version 0.13
* typst: fix version order
Co-authored-by: Alec Scott <hi@alecbcs.com>
* typst: more precise version requirements
* typst: use build_directory
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Installing `rust@nightly` fails because the package file declares a conflict of rust versions older than `:1.64` with `gcc>=13`. However, because `nightly` is alphanumerically smaller than any actual version number, `nightly` is incorrectly detected to have a conflict with `gcc>=13` as well. Marking `nightly` as an infinity version instead solves this.
* Reproducer should decude artifact root from concrete environment
* Add documentation on the layout of the artifacts directory
* Use dag hash in the container name
* Add reproducer options to improve local testing
* --use-local-head allows running reproducer with
the current Spack HEAD commit rather than computing
a commit for the reproducer
* Add test to verify commits and recreating reproduction environment
* Add test for non-merge commit case
* ci reproduce-build: Drop overwrite option
in favor of throwing an error if the working dir is non-empty
* py-tensorflow: patch for v2.15 build errors with new compilers
* py-tensorflow: patch for v2.15 build errors with new compilers
* py-tensorflow: fix clang build and add clang version constraints
* py-tensorflow: use compiler wrapper
* py-tensorflow: relax clang conflict
* nim: fix build with new compilers
* narrow condition for disabling warnings
* move flags into offending module
disables warnings also for compiling projects other than the Nim compiler when necessary
* specify different versions pthread modules
* instead patch SysThread type
* adapt patch for old Nim versions
* Specify hypothetical `:@0.19.6` for patch version constraint
* In var/spack/repos/builtin/packages/ectrans/package.py, always set cmake argument ECTRANS_HAVE_CONTIGUOUS_ISSUE to turn off problematic use of Fortran 'contiguous' keyword
* In var/spack/repos/builtin/packages/ectrans/package.py, always set cmake argument ENABLE_FINAL=OFF to turn off problematic finalization of derived data types
* Update links to issues in fckit and ectrans
* Fix wrong cmake argument for ECTRANS_HAVE_CONTIGUOUS_ISSUE in var/spack/repos/builtin/packages/ectrans/package.py
This commit adds an additional requirement to the vecmem package,
requiring the OneAPI compiler iff the `sycl` variant is turned on. This
allows us to correctly set the non-standard `SYCLCXX` environment
variable.
* likwid: Fix the perms script
The script loops over the path (which includes the prefix), but
additionally adds the prefix up front which results in duplicate paths
and a double "/" in the command like in the following one:
chown root:root /opt/csg/spack/opt/spack/linux-debian12-zen2/gcc-12.2.0/likwid-5.4.1-xfc6quebnf2kosydl3ospaeoskxnxwhn//opt/csg/spack/opt/spack/linux-debian12-zen2/gcc-12.2.0/likwid-5.4.1-xfc6quebnf2kosydl3ospaeoskxnxwhn/sbin/likwid-accessD
Additionally the path is currently not quoted which can potentially
result in word splitting for weird paths.
Signed-off-by: Christian Heusel <christian@heusel.eu>
* likwid: Make the perm scripts' name unique
Also move it into the proper binary folder as per the Filesystem
Hierarchy Standard.
Signed-off-by: Christian Heusel <christian@heusel.eu>
---------
Signed-off-by: Christian Heusel <christian@heusel.eu>
* Initial checkin of compile file for the package libgpiod, this covers versions, 1.6.3 through 2.2
* Update var/spack/repos/builtin/packages/libgpiod/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/libgpiod/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update how version to url is manage in the x.y.0 cases based on fix by @wdconinc
* Remove redundant url at the top since it is now inside url_for_version function
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* xios: update to v2.6
- xios: update to v2.6
- xios: patch remap earcut.hpp for missing include file
- xios: address C++11 requirement
- xios: instruct to use external boost and blitz libraries
* xios: bump revision to 2714
- Bump revision to changeset 2714
- Deprecate older versions that I can't manage to compile
- Confirm clang workaround is still needed
A directory and a symlink to it under the same relative path in a
different prefix
```
/prefix1/dir/
/prefix1/dir/file
/prefix2/dir -> /prefix1/dir/
```
are not a blocker to create a view. The view structure simply looks like
this:
```
/view/dir/
/view/dir/file
```
This should be the case independently of the order in which we visit
prefixes, so we could in principle create views order independently.
* damaris: add v1.12.0, update maintainers (#48674)
Co-authored-by: Etienne Ndamlabin <jean-etienne.ndamlabin-mboula@inria.fr>
* Bump up the version for rocm-6.3.1 release (#48440)
This PR updates the versions for the rocm recipes for rocm-6.3.1 release.
* py-flash-attn: add missing triton dependency (#48645)
* hep stack: additional event generator packages (#48565)
* hep stack: additional event generator packages
* hep: adidtional packages
* hep: collier doesn't have +pic +shared
* py-awkward-cpp: fix scikit-build-core range of applicability
* hep: disable agile
* hep: disable garfieldpp and genie
* py-wxpython: depends_on pkgconfig even if using external wxwidgets
* hep: disable professor
* papi: fix error finding gmake during post-install testing (#48592)
* JAX: add v0.4.32+ (#46346)
* JAX: add v0.4.34
* Disable search for clang
* Update CUDA flags
* Add py-jax 0.4.33, comment out until py-jaxlib 0.4.33 is also released
* Fix GCC build
* Try TF_NVCC_CLANG
* py-jax: add v0.4.34
* jax no longer has separate tags for jaxlib
* Install compiled wheel
* Join path before glob
* Wheel is in spack stage, not tmp path
* Add 0.4.35
* Add newer versions
* Build system has been refactored yet again
* Drop clang
* Fix build with source tarball, rocm support
* Support GCC
* Remove clang-specific compiler flags
* enable_cuda flag was removed
* Fix logic
* py-jax: add v0.4.38
* Add patch to fix GCC support
* Patch no longer needed
* Skip patching, directly pass flags
* New flags
* Remove unused import
* Patch changed
* Use older version of patch
* Newer patch
* Add CUDA symlink
* Symlink more directories
* Recursive symlink
* Import function
* Recursive search
* Undo cuda changes
* Add v0.5.0
* I quit
* py-geemap: add new package (#48602)
* easi: add v1.5.1; relax yaml-cpp and lua requirements (#48675)
* thepeg: extend the rivet@:3 dependency up to version 2.3 (#48691)
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
* py-ipyrad: adding version 0.9.102 (#48686)
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* autodock-vina: adding version 1.2.6 (#48684)
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* toybox: add v0.8.12 (#48657)
* Changes for NVIDIA HPC SDK 25.1 (#48696)
* update hypre version and add new memalign for petsc (#47831)
* petsc+rocm: add dependency on hipblas-common (#48644)
* spec.py: fix ArchSpec.intersects (#48741)
fixes a bug where `x86_64:` and `ppc64le:` intersected, and x86_64: and :haswell did not.
* ucx: adding 1.18.0 (#48742)
* Adding UCX 1.18.0
* Verified and correct hash.
* MAGMA: add v2.9.0 (#48750)
* Deprecate frontend/backend os/target (#47756)
* package api: drop wildcard re-export (#48760)
* package api: drop wildcard re-export
To ensure package repos are forward/backward compatibility with Spack,
we should explicitly export all symbols we want to expose in the public
package API, and drop `from spack.something import *` because
removal/addition to the public API will go unnoticed.
Also `llnl.util.filesystem` has some methods that shouldn't be exposed
in the package API, so better to enumerate a subset explicitly.
* remove flatten_dependencies / install_dependency_symlinks
* py-cmake: remove. remove deprecated cmake versions (#48763)
* Remove pipelines and images based on ppc64le (#48767)
* petsc: only conflict with kokkos@4.5: if it is enabled (#48698)
* hpctoolkit: Add `+docs` variant and manpages (#48566)
* py-mdit-py-plugins: Add new versions 0.3.5, 0.4.2
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* py-myst-parser: Add new versions 0.19.0 to 4.0.0
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* hpctoolkit: Add +docs variant and manpages
This commit unconditionally enables manpages for the HPCToolkit tools.
The new `+docs` variant enables additional documentation, specifically
the user's manual. Both require new build-time dependencies.
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
---------
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* fmt: simplify +pic (#48766)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
* Docs/bugfix: correct return for Adding flags to configure (#48434)
* binutils: conflict on configuration with build issues (#42949)
* Create SALT package.py (#48758)
* Create SALT package.py
Added a package for the SALT Source AnaLysis Toolkit
@zbeekman
* [@spackbot] updating style on behalf of wspear
* Update package.py
Line wrap
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* smee-client: add v2.0.4 (#48384)
* CMake: add v3.31.5, v3.30.7 (#48759)
* builtin: remove redundant imports (#48765)
* builtin: remove redundant llnl.util.filesystem import
* remove redundant import spack.version
* unsorted fixes
* more spack.version
* abinit: pass flag correctly (#48788)
* Add py-zarr 3, which includes a new required package py-donfig, and a bug fix to the patch range with numcodecs (#48786)
* libxc: add CMake builder (#48772)
* libsmeagol
* libxc cmake
* cmake support
* revert changes
* make spackbot happy
* fix
* Update package.py
* hipblaslt: update cmake dependency (#48637)
* hipblaslt: update cmake dependency
1 error found in build log:
>> 3 CMake Error at CMakeLists.txt:24 (cmake_minimum_required):
4 CMake 3.25.2 or higher is required. You are running version 3.22.1
5
6
7 -- Configuring incomplete, errors occurred!
See build log for details:
/scratch/svcpetsc/spack-rocm/spack-stage/spack-stage-hipblaslt-6.3.0-pabb7t4rheqkz74lfzbsnqi6vnpiqwlq/spack-build-out.txt
* Update var/spack/repos/builtin/packages/hipblaslt/package.py
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
---------
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
* Move from python2 compliant IOError and EnvironmentError to python3-only OSError (#48764)
* IOError -> OSError
* also do EnvironmentError
* py-sphinx: mark Python compatibility (#48796)
* spack.package: wrap llnl.util.tty (#48793)
avoid import of llnl.util.tty in packages
* spack.package: re-export EnvironmentModifications / Prefix (#48792)
* Remove unused values (#48795)
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* petsc, py-petsc4py: add v3.22.3 (#48785)
* libsmeagol (#48776)
* libsmeagol
* add support for intel and add conflicts
* cp2k
* Bug Fix: Better incremental check for CMake (#48775)
* Bug Fix: Better incremental check for CMake
* Fix syntax error
* Ensure match of config artifact with generator
* add cdo@2.5.0 (#48801)
* Added salt variant to tau (#48782)
* Added salt variant to tau
* Update package.py
* [@spackbot] updating style on behalf of wspear
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* Remove patch on main (#48798)
Patch got merged: https://github.com/natefoo/slurm-drmaa/pull/62
* kokkos-nvcc-wrapper: add version 4.5.00 and 4.5.01 (#48802)
* env create: create copies of relative include files in envs created from manifest (#48689)
Currently, environments created from manifest files with relative includes result in broken
references to config files.
This PR modifies `spack env create` to create local copies in the new environment of any local
config files from relative paths in the environment manifest passed as an init file.
This PR does not change the behavior if the include is an absolute path or if the include is from
a relative path outside the environment directory, but it does warn about missing relative includes if
they are inside the environment directory.
Includes regression test and short blurb in docs.
* py-sphinx-rtd-theme: add v2.0.0, v3.0.0 (#48756)
* Add versions 2 and 3 of py-sphinx-rtd-theme.
Allow for versions of py-sphinx greater than 6.
Fix the Python version for older versions that depend on distutils.
Get the py-docutils dependency from the py-sphinx recipe.
* Depend purely on the py-docutils dependency in py-sphinx.
* More refined dependency versioning.
* Fixed versioning for py-sphinx and py-docutils.
* CP2K: add 2025.1 version and DFTD4 support (#48489)
* cp2k: add dftd4 variant
* better conflict and make support
* typo
* Update var/spack/repos/builtin/packages/cp2k/package.py
* Update var/spack/repos/builtin/packages/cp2k/package.py
* oci/opener.py: respect system proxy settings (#48783)
* vtk-m: CMAKE_CXX_COMPILER is not a BOOL (#48813)
* gcc: remove --with-ld=ld-classic (#48826)
* nanotron: add new package (#48582)
* nanotron: add new package
Also, update some dependencies and add missing ones.
* Add variant +examples needed to execute example scripts
* fix: add missing branch attribute
* Remove master version
* fix: use Github hash
* embree: fix tests by building tutorial's embree_viewer for tests (#48392)
* import-check: improve how problematic imports are displayed (#48825)
The import-check action now presents problematic import statements
introduced by the PR better.
The idea is roughly:
* Let (V₁, E₁) be the graph of modules as vertices and import statements
as edges before the change
* Let (V₂, E₂) be the graph after the code change, which is typically a small
perturbation of (V₁, E₁).
* X₁ = FAS(V₁, E₁) is the feedback arc set before (a minimal set of edges to
delete to make it acyclic)
* X₂ = FAS(V₂, E₂ ∖ X₁) is the feedback arc set after deletion of the minimal
set of edges that made the old graph acyclic.
* X₃ = FAS(V₂, E₂) is the feedback arc set after
Previously I displayed X₁ and X₃ and users had to diff themselves.
Now, I'm showing X₂, which is a small set, typically directly related to
code changes.
However, it can be that a small code change adding say 2 problematic imports
creates a completely different solution X₃ that only requires deletion of just 1
different import. In that case the user is informed that they can potentially do
less work.
So for PR #48784 the output is now:
> The overall number of problematic import statements increased by 1 from 31 to 32.
> This is likely a direct consequence of the following import statements:
>
> ```
> spack/config imports: spack.spec, spack.util.path, spack.util.remote_file_cache
> ```
>
> However, instead of removing 3 import statements, it is sufficient to remove only 1
> import statement from the following list:
>
> ```
> spack/concretize imports: spack.bootstrap, spack.solver.asp
> spack/environment imports: spack.bootstrap, spack.environment
> spack/fetch_strategy imports: spack.version.git_ref_lookup
> spack/install_test imports: spack.build_environment, spack.package_base
> spack/modules imports: spack.modules
> spack/platforms imports: spack.config
> spack/relocate imports: spack.bootstrap
> spack/repo imports: spack.package_base, spack.patch, spack.tag
> spack/spec imports: spack.binary_distribution, spack.compiler, spack.compilers, spack.concretize, spack.environment, spack.hash_types, spack.provider_index, spack.repo, spack.spec_parser, spack.store, spack.traverse, spack.variant, spack.version.git_ref_lookup
> spack/subprocess_context imports: spack.environment
> spack/util/gpg imports: spack.bootstrap
> spack/util/package_hash imports: spack.package_base
> spack/util/path imports: spack.config, spack.environment
> spack/util/remote_file_cache imports: spack.util.web
> ```
from which the user can figure out that
`spack/util/remote_file_cache imports: spack.util.web` is the "bottleneck" now.
* spack_yaml: use unambiguous variable name (#48832)
* style: fix `not in` and `is not` (#48831)
These are some changes that `ruff check --fix` would make that the current
`spack style` also agrees with. Make the changes now so that the `ruff`
change is less disruptive.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Set version to v1.0.0.dev0 (#48791)
* import-check: enable color output (#48842)
* berkeleygw: add -o flag to tar extraction (#48816)
when extracting as root user, avoid that tar attempts to change file ownership
* llvm: deprecate old patch releases (#48762)
* cuda: add v12.8 (#48708)
* sirius: patch pugixml (#48841)
* ziatest: add new package (#48809)
* gitlab: remove isc stacks (#48811)
* gcc: deprecate old patch releases (#48761)
* CP2K: use libxc@7 for master/next release (#48808)
* packge_base.py: remove _patches_by_hash (#48768)
* cdash: avoid build_opener (#48846)
* g4vg: new package (#48844)
* g4vg: new package
* [@spackbot] updating style on behalf of sethrj
* r-dmrcate: add v2.16.0, v3.0.0, v3.2.0 (#48158)
* r-dmrcate: add new versions
* r-dmrcate: require `r@4.3.0` for v2.99.0+
* r-dmrcate: update dependencies
* py-tensorflow: add 2.18.0-rocm-enhanced (#48711)
* py-tensorflow: add 2.18.0-rocm-enhanced
* fix style
* fix style
* fix style
* review changes
* review changes
* remove hipblaslt dependency
* remove ci changes and force ROCm 6.3.1 for newest TF
* remove rocm 6.3.1 dependency
* simplify configure fix
* rocm-examples and rocjpeg: new packages (#47695)
* new package: rocm-examples
* add new package rocjpeg and update rocm-examples for 6.3.0
* fix licenses
* add versions 6.3.1
* change homepage and git
* add f-string
* Tesseract v5.5.0 (#48866)
* leptonica: adding v1.85.0
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* tesseract: adding v5.5.0
Signed-off-by: Shane Nehring <snehring@iastate.edu>
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* py-geojson: Add new package (#48847)
* Add new package py-geojson
* fix when
* isa-l: add v2.31.1 (#48859)
* Add new package func (#48849)
* icu4c: add v75.1, v76.1 (#48858)
* Add new package py-metis (#48848)
* cppgsl: add v4.1.0 (#48864)
* libdrm: add v2.4.124 (#48860)
* amrex: add v25.02 (#48853)
* sherpa: support cxxstd=20 when=@3: (#48829)
* sherpa: support cxxstd=20 when=@3:
* hep: sherpa cxxstd=20
* netlib-scalapack: Update version (#48667)
* Update scalapack version
Signed-off-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
* use url_for_version
Signed-off-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
* use spec.satisfies instead of version()
---------
Signed-off-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
Co-authored-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
* fcgi: add v2.4.3, v2.4.4 (#48856)
* update pyproject.toml for `ruff format` (#48823)
Add ruff configuration to `pyproject.toml`.
This allows `ruff format` in the Spack repository to format all the files we care about,
with our line length of 99, the exceptions we already put in place, and excluding things
we don't auto-format, like vendored dependencies.
Right now it'll reformat 175 or so files, but only slightly, in places where `ruff` differs from
`black`. For the most part I like the ruff format decisions better than `black`, but none of
the changes seem too severe.
This does not change `spack style` -- I figure that can come later but this at least will
let people start playing with `ruff`.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* dd4hep: add v1.31 (#48850)
* libsm: add v1.2.5 (#48862)
* Apply workaround for oneAPI compiler for upcxx problem with a template argument list (#48843)
* Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword
* Revert "Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword"
This reverts commit faf9b8ce85.
* Apply workaround for oneAPI compiler
* style problem resolved
* use spec.satisfies syntax
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
* Remove ISC stacks environment files (#48851)
Follow-up to #48811
* Remove variable from cmake.py (#48824)
* Remove variable from cmake.py
#48775 left a dangling variable that was not caught in CI but by the eyes of @haampie. Restructure variable to local method.
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
* Update lib/spack/spack/build_systems/cmake.py
* Update lib/spack/spack/build_systems/cmake.py
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* sirius: add v7.6.2 (#48797)
Co-authored-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
* gha: standalone import-check (#48873)
* ci: add codecov token secret to coverage upload job (#48880)
Codecov needs to see the token secret when uploading, so we have to
add this line to the workflow YAML:
```yaml
with:
token: ${{ secrets.CODECOV_TOKEN }}
```
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* ci: bump import-check (#48883)
* pfind: new package (#48685)
* py-elevation: new package (#48836)
* spec.py: fix hash change due to None vs {} (#48854)
* Fix hash change due to None vs {}
* Enforce null for empty list of external_modules
* nnn: new package (#46174)
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* Fix esmf usage, add new version (#48835)
* bash: add autotools dependencies (#48874)
* Scotch: add v7.0.6, add testing option (#48781)
* code-server: update to v4.96.4 (#48828)
* acts: add v39.0.0 (#48839)
This commit adds version 39.0.0 of the ACTS package which, as far as I
can tell, doesn't require any dependency updates.
* spec.py: ensure spec.extra_attributes is {} if is null in json (#48896)
* harfbuzz: add v10.2.0 (#48857)
* libice: add v1.1.2 (#48861)
* relocate.py: don't warn about symlinks (#48904)
`relocate_links` warns when the target is absolute and not matched by
any prefix from the prefix to prefix map.
This can lead to false positives, cause the prefix to prefix map does
not contain trivial/identity entries whenever a package is installed to
its original location.
Since relocate_links is the odd one out there (we don't warn about
similar issues with rpaths, etc), just remove the warning.
* hip-tests: new package (#47273)
* hip-tests: add new package
* remove hip-tests from hip recipe
* remove old versions
* fix style
* add missing import
* bump hip-tests to 6.3.1
* fix style
* flecsi: new version 2.3.1 (#48867)
* flecsi: add new version 2.3.1, remove develop
* flecsi: remove kokkos and openmp variants moving forward
* flecsi: propagate cuda and rocm settings from kokkos
* Update var/spack/repos/builtin/packages/flecsi/package.py
Co-authored-by: Davis Herring <herring@lanl.gov>
* flecsi: remove redundant depends_on lines
* flecsi: correct legion dependency
* flecsi: deprecate v2.0.0 and v2.1.0
* flecsi: force +openmp if ^kokkos+openmp
---------
Co-authored-by: Davis Herring <herring@lanl.gov>
* Update py-arch, py-statsmodels (add 0.14.1), py-patsy (add 0.5.4) to be able to use py-cython@3 (#48769)
* Add py-patsy@0.5.4
* Correct py-numpy dependency in py-arch
* Add py-statsmodels@0.14.1 and update dependencies
* Add climbfuji as maintainer for py-patsy
* Add climbfuji as maintainer for py-statsmodels
* Update var/spack/repos/builtin/packages/py-statsmodels/package.py
* enzyme: add v0.0.172 (#48881)
* spec.py: ensure == is false if true modulo precomputed dag hash (#48889)
* llvm: fix @15 %apple-clang@16 (#48887)
* spiner: add v1.6.3 (#48871)
* spiner: update package logic
* singularity-eos: remove spiner cuda_arch propagation
* spiner: add version 1.6.3
* sherpa: +hepmc3root only when +root (#48827)
* sherpa: +hepmc3root only when +root
* sherpa: fix style
* salt: add v0.3.0 (#48877)
* salt: Add v0.3.0 of SALT
This version contains important bug fixes for building and parsing
projects containing Fortran
* salt: Be more explicit about dependency types
- llvm+clang+flang is needed at build, link and runtime for the
correct operation of SALT
- Testing with llvm@master ( llvm > 19.x ) shows that SALT is
currently incompatible with the latest llvm API so an updated salt
will be required when LLVM 20 is released
* openturbine: add new package (#48683)
* PyTorch: add v2.6.0 (#48794)
* mummer4: patching to allow building with %gcc@13: (#38292)
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fms: add 2025.01, 2024.03 (#48812)
* py-shapely: add v2.0.7 (#48810)
* spectre: add v2025.01.30 (#48803)
Co-authored-by: sxs-bot <sxs-bot@users.noreply.github.com>
* py-numba: Add version 0.61 (#48837)
* amdfftw: fix broken build, adjust flags for performance tuning (#48754)
With CFLAGS, the code path in the amdfftw build system will bypass the logic around AMD_ARCH.
---------
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
* justbuild: add v1.4.3 (#48898)
* nvpl-blas, nvpl-lapack: add v0.4.0.1, v0.3.0 (#48901)
* hep stack: build also with cuda and rocm where possible (#48528)
* lcio: add v2.22.4 (#48895)
* Add a message for CMake incremental build (#48905)
* Add a message for CMake incremental build
Requested message to explain CMake phase is getting skipped.
* [@spackbot] updating style on behalf of psakievich
* Update import
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* dcap: depends_on libxcrypt (#48903)
* Add new version of r-curl (#48912)
* pika: Add 0.32.0 (#48897)
* icu4c: no cxxstd flag option on Windows (#48510)
* ICU4C: Don't reference a spec variant on a platform on which it's not defined
* icu4c: no cxx flag on Windows
* dla-future-fortran: add v0.3.0 (#48900)
* simgrid: add v3.36 (#48909)
* dyninst: cleanup package (#47637)
* Use more idiomatic construct, shorten recipe
* Remove deprecated versions, and associated patches
* Remove v10.0.0
* Windows: Update default config for stage location (#48511)
Current location is within the Spack prefix, which causes builds to
pollute VCS with stage artifacts and generally inflates the Spack
install prefix.
This PR moves it to the user cache location now that we can
consistently support paths with spaces on Windows.
* py-maturin: add v1.8.2 and refined dependencies (#48915)
* clingo-bootstrap: fix +optimized build (#48931)
* fix regression `apple-clang` vs `%apple-clang`
* use f-strings
* remove --verbose flag from LDFLAGS
* Fix regression due to dyninst update (#48935)
* trexio: fix issues with autotools build system (#48923)
* package_base.py: remove use_cray_compiler_names (#48932)
* Apply workarounds for oneAPI compiler for ascent problem with build (#48918)
* Apply workarounds for oneAPI compiler for ascent problem with build
* Use the way with use patch through the PR address
* stylecheck - missing comma
* libgcrypt: fix enforced -O0 (#48940)
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Co-authored-by: Shane Nehring <snehring@iastate.edu>
* serialbox: add version 2.6.2 (#48937)
* nwchem: add master (#48919)
* Add possibility to build nwchem from master branch
* add oneapi@2025: patch for @7.2.3
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
* Python: add new versions (#48950)
* Python: add new versions
* black
* reframe: add v4.6.4 -> v4.7.2 (#48242)
* go: add v1.23.6 (#48955)
* qmcpack: add v4.0.0 (#48921)
* py-einops: add v0.8.1 (#48954)
* flux-sched: add v0.42.1 (#48952)
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
* Quantum ESPRESSO: add v7.4.1 (#48949)
* duckdb: add v1.2.0 (#48902)
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* mapl: add v2.53.1, v2.54.1 (#48944)
* log.py: remove setenv calls (#48933)
* nim: add v2.2.2 (#48929)
* Update GFE packages (#48899)
* OpenMPI: add version 4.1.8 (#48922)
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
* extrae: tighten dependencies on boost for +dyninst (#48938)
* py-iterative-stats: add 0.1.1 (#48959)
* import-check: bump (#48968)
* ports-of-call: add v1.6.0, v1.7.0, v1.7.1 (#48870)
* acts dependencies: new versions as of 2025/02/10 (#48969)
This commit adds detray v0.88.0 and GeoModel v6.9.0.
* cbtf-krell: Update Boost dependency (#47133)
* Update Boost
* Add gotcha
* Add patch for build errors
* Allow building with latest Dyninst
* Fix patch url
* lua-sol2: Apply workaround for oneAPI compiler for problem with build (#48920)
* Bump up the version for rocm-6.3.2 release (#48787)
* Bump up the version for rocm-6.3.2 release
* rocm-openmp-extras update and style correction
* Updating mivisionx, omniperf, rccl & rocprofiler-systems
* Updating hipsparselt & rocm-opencl
* rocprofiler-systems on gcc-13 and rvs commit instead of patch
* Updated rocjpeg & rocm-examples for 6.3.2
* ROCPROFSYS_BUILD_DYNINST & DYNINST_BUILD_TBB are required only with gcc-13
---------
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
* spack compiler find: detect `flang-new` and `flang` in newer LLVM versions (#48914)
* rivet: patch missing header in 3.1.10 (#48977)
* concretizer: reduce search space with static analysis (#48729)
Currently, when we setup the ASP problem for `clingo`, we don't take into account the configuration. This results in setting up ASP problems that are larger than necessary, with possibly redundant information, and higher concretization times.
This PR tries to improve things by adding an opt-in feature that computes the _possible dependencies_ of a solve taking also into account the current configuration, and avoids adding possible dependencies that we are certain can't be in the final solution.
The feature can be activated with:
```yaml
concretizer:
static_analysis: true
```
Examples of simple rules to discard dependencies are:
- Dependencies that are not buildable, and for which no binary is present (e.g. `cray-mpich` etc. on non Cray systems)
- Dependencies that are not for the current platform (e.g. `msmpi` on non Windows platforms)
- Conditional dependencies that cannot be activated, because of some user requirement (e.g. `cuda` etc. if the user requires `~cuda` in configuration)
- Virtual providers that cannot be used, because of a requirement on a virtual
The speed-up these rules seem to give depends on the use case at hand, but if the configuration is updated properly, they are noticeable.
Since in cases where there is no rule to exclude packages upfront, reuse is active, and this option is activated, it's possible to see some minor slow down, the feature has been added as opt-in, so it's turned off by default.
* spack.util.elf: catch seek errors (#48972)
* hep: rivet: require hepmc=3 (#48976)
* Fix performance issue on macOS (#48997)
archspec.cpu.host() is not memoized, so compute
it as less as possible.
---------
Co-authored-by: alalazo <alalazo@users.noreply.github.com>
* style.py: fix false negative in redundant import statements (#48980)
* PyTorch: build flash attention by default, except in CI (#48521)
* PyTorch: build flash attention by default, except in CI
* Variant is boolean, only available when +cuda/+rocm
* desc -> _desc
* kokkos et al. : don't monkeypatch spec in callbacks (#48916)
Currently, a few packages using kokkos rely on
kokkos itself monkeypatching its own spec to
provide some attribute.
In this commit we change this attribute to be
defined on the package, and never be monkeypatched.
* gmake: add empty libs property, remove link deptypes from dependents (#48995)
* package_hash.py: move metadata_attrs inline out of package_base (#48981)
* gmake: fix def libs/headers (#49009)
* Spec.package_class -> spack.repo.PATH.get_pkg_class (#48985)
* libfabric: use the class variable to get the list of fabrics (#49007)
Suggested by: alalazo <alalazo@users.noreply.github.com>
Signed-off-by: Justin Cook <jscook@lbl.gov>
* py-transformers: add new versions (#49000)
* py-transformers: add new versions
* py-tokenizers: add new versions
* Apply suggestions from code review
* lcio: Add latest 2.22.5 tag (#48991)
* concretize.lp: don't warn about deprecation when external (#49008)
* Spec.is_virtual -> spack.repo.PATH.is_virtual (#48986)
* g4vg: add 'develop' branch (#49003)
* g4vg: add develop version
* celeritas: add develop version
* Fix style
* REVERTME: move celeritas changes to another branch
* Get full repo
* remove unneeded variable
* Remove spack.repo.PATH.is_virtual call from SpecBuildInterface.(#48984)
This PR is effectively a breaking change extracted from #45189, which removes
support for spec["mpi"] if spec itself is openmpi / mpich that could provide mpi;
from the Spec instance we don't have any parent it provides it to,
hence it's a KeyError.
* Spec.validate_detection -> spack.detection.path.validate_detection (#48987)
* hep: add missing language dependencies (#48963)
* highfive: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989283
* lhapdf: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989283
* vc: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989140
* davix: add dependency on C, C++
https://gitlab.spack.io/spack/spack/-/jobs/14989131
* pandorasdk: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989130
* veccore: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989118
* pythia6: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989116
* jwt-cpp: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989115
* collier: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989115
* hepmc: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989112
* clhep: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14989075
* fastjet: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14981340
* gosam-contrib: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14978873
* thepeg: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997553
* cepgen: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997552
* podio: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997552
* pandoramonitoring: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997552
* lcio: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997513
* geant4: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14997202
* evtgen: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14996817
* apfel: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/14996779
* collier: add dependency on C, C++
https://gitlab.spack.io/spack/spack/-/jobs/14996770
* vecgeom: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/15003840
* dd4hep: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/15003839
* opendatadetector: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/15007666
* acts: add dependency on C
https://gitlab.spack.io/spack/spack/-/jobs/15007827
* hepmc: remove dependency on fortran
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* thepeg: remove fortran dep
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* acts: add a conditional build dependency
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* opendatadetector: add comment to explain C dep
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* apptainer: get_full_repo for branch main (#49002)
* unifyfs: Apply workaround for oneAPI compiler for problem with build (#48962)
* solver: add type-hints to OutputConfiguration (#48979)
* Ci: ensure file path comparsion uses posix paths (#47033)
Git always produces posix paths, ensure we're always comparsing apples to apples by normalizing paths compared to git output to posix.
* postgresql: add v17.2 (#47811)
* postgresql: add version 17.2
* postgresql: install flex, bison and perl when building versions 17 and up
* postgresql: do not install perl by default when building versions 17 and up
* cray-mpich: adding partial GTL support (#45830)
cray-mpich now has a rocm variant. You can use gtl_lib in the
flag_handler like so:
```python
def flag_handler(self, name, flags):
wrapper_flags = []
environment_flags = []
build_system_flags = []
if self.spec.satisfies("+rocm"):
if self.spec.satisfies("^cray-mpich"):
gtl_lib = self.spec["cray-mpich"].package.gtl_lib
build_system_flags.extend(gtl_lib.get(name) or [])
# hipcc is not wrapped, we need to pass the flags via the
# build system.
build_system_flags.extend(flags)
return (wrapper_flags, environment_flags, build_system_flags)
```
---------
Co-authored-by: Richard Berger <rberger@lanl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Richard Berger <richard.berger@outlook.com>
* lammps: use the Cray GTL (#46090)
* lammps: add 20241119 and 20250204 releases (#48978)
* plog: add new package (#48975)
* binutils: add debuginfod variant + update deps (#49011)
* h5hut: Remove H5_USE_110_API for newer versions (#48885)
* h5hut: Remove H5_USE_110_API for newer versions
* h5hut: style reformat and add maintainer
* h5hut: correct version syntax for <v2.x.x
* h5hut: Bump default version to 2.0.0rc7 and remove older rc candidates
* Update var/spack/repos/builtin/packages/h5hut/package.py
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
---------
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
* spack debug create-db-tarball: remove after test failures (#49025)
* bubblewrap: add versions up to v0.11.0 (#49023)
* gftl: add v1.15.2 (#48992)
* py-torchgeo: pyvista dep has been removed (#48990)
* py-memray: add v1.15.0 (#48989)
* yosys: add v0.50 (#48983)
* laszip: Add version 3.4.4. (#48982)
* py-numpy: add v2.2.3 (#49029)
* pandora{pfa,sdk,monitoring}: add new versions and allow setting the C++ standard (#48300)
* pandoramonitoring: add v3.6.0; pandorapfa: add v4.11.2
Remove variables that are not being used in pandorasdk. Use the C++ standard
from ROOT when possible and pass -Wno-error to override the -Werror that will
typically fail with a new standard. Add a cxxstd variant for pandorasdk
* Fix style
* Update var/spack/repos/builtin/packages/pandorasdk/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Fix style
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* easi@1.5.2 (#49013)
* Put in maintainers for Kokkos Tools Spack package (#49018)
* Kokkos Tools package.py: fix maintainers
* Kokkos Tools package.py: remove white space between first and second maintainer in comma-separated list
* [@spackbot] updating style on behalf of vlkale
* Correct maintainers syntax
Co-authored-by: Richard Berger <richard.berger@outlook.com>
---------
Co-authored-by: vlkale <vlkale@users.noreply.github.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Richard Berger <richard.berger@outlook.com>
* zlib package: Ensure correct lib search on Windows (#48512)
* Name of zlib's library differs on Windows; also account for name
differing when building +shared
* `zlib`'s `.libs` implementation was searching for the runtime
libraries (the .dlls) and should be searching for link-time libs
* Update openfast, amr-wind, and nalu-wind packages (#48994)
* acts: conflicts ^geant4@11.3: when @:35 (#49028)
* yq: add versions 4.44.5 and 4.44.6 and 4.45.1 (#49027)
* openblas: .libs() uses self.libraries attribute (#48942)
Currently this is hardcoded to the same value as listed in the class
definition. If one ever overrides this attribute, such as:
```
packages:
openblas:
package_attributes:
libraries = [ 'libopenblaso64', ]
```
this patch will make sure that override also in the
`spec['openblas'].libs()` call. (Which happens in `hypre`, likely
others).
( see
https://spack.readthedocs.io/en/latest/packages_yaml.html#assigning-package-attributes
)
Thanks to becker33 for debugging help in Slack
* RepoSplit/tests: update repo tests relying on builtin package repo to only use mock repos (#48926)
* RepoSplit/tests: update repo tests relying on builtin
* test_repo_last_mtime: skip on windows due to mtime issues in CI
* MesonPackage: depends_on pkgconfig (#46955)
meson's `dependency` function often uses pkg-config to locate a dependency, and may even fall back to cmake. The former case is very common, and since packagers often forget to add the tiny pkgconfig package as a build dep, we do it for them.
* Spec.__getitem__: restrict to direct deps + transitive runtime deps (#49016)
With this change spec["pkg"] searches only direct dependencies and transitive link/run
dependencies, ordered by depth. This avoids situations where we pick up unwanted
deps of build/test deps.
To reach those, you need to do spec["build_dep"]["pkg"] explicitly.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* views: normalize paths on case insensitive file systems (#47370)
On macOS, prefix_a/file and prefix_b/FILE map to the same file view/file or view/FILE.
This commit ensures that we test whether a view is created on a case insensitive filesystem and handle projection conflicts accordingly.
* Allow tuning max_dupes for build dependencies (#48948)
Up to now, Spack was allowing all build-tools that
may appear in the DAG to have 2 max_dupes.
This is not needed in practice for most of them,
and adding them out of caution just increases
grounding and concretization time.
This PR makes the value of max_dupes configurable
per package, and sets only a few known packages to
2 max_dupes by default.
In case user needs different values, they can
tune the configuration for their use case.
* vtk: fix 9.4.1 concretization (#48946)
* seacas: conflict 2024-06-27 with windows
* vtk: fix 9.4.1 seacas dependency
* cairo: add new version and update build system (#48822)
* update cairo for new meson build system
* update patch range. remove old conflict
* style
* update pango to reflect the changes in cairo
* refine depends
* style
* add lzo depends
* add +shared
* non self-referential variant requireme
* style
* Move +shared variant back to just autotools as meson automatically handles it
* clarify patch when=
* update based on reviews. switch from conflicts to requires to enforce variant synchronization
* refine conflicts and requires
* better group build deps together
* comment for meson build lower version bound
* clarifying comments
* clarify version ranges, enforce build_system with version ranges
* style
* cairo: no need to require for build_systems
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* fzf: add v0.60.0, improve styling (#49059)
* fd: improve documentation and styling to help newer maintainers (#49058)
* direnv: add master, fix up package for better documentation (#49053)
* GDAL: add v3.10.2 (#49042)
* pbwt: new package (#49055)
* pbwt: add v2.1, v2.0
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* [@spackbot] updating style on behalf of teaguesterling
* Update package.py
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* py-xarray-regrid: Add new package (#48834)
* Add py-xarray-regrid and required dep flox
* remove boiler plate
* Add missing py310 dep
* py-flox, py-xarray-regrid: add type=("build", "run") to python dependency
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* build(deps): bump isort in /.github/workflows/requirements/style (#48746)
Bumps [isort](https://github.com/pycqa/isort) from 5.13.2 to 6.0.0.
- [Release notes](https://github.com/pycqa/isort/releases)
- [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pycqa/isort/compare/5.13.2...6.0.0)
---
updated-dependencies:
- dependency-name: isort
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps): bump black from 24.10.0 to 25.1.0 in /lib/spack/docs (#48780)
Bumps [black](https://github.com/psf/black) from 24.10.0 to 25.1.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/24.10.0...25.1.0)
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* surfer: new package (#48432)
* Add waveform viewer, surfer for RTL simulations
* Ran black over the code following style check failure
* build(deps): bump black in /.github/workflows/requirements/style (#48779)
Bumps [black](https://github.com/psf/black) from 24.10.0 to 25.1.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/24.10.0...25.1.0)
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* new package: jujutsu (#48231)
* lis: add v2.0.28 -> v2.1.7 (#48308)
* Added LIS 2.1.7
* Added LIS versions from 2.0.28 to 2.1.7
* apply black v25.1.0 (#49076)
* build(deps): bump isort from 5.13.2 to 6.0.0 in /lib/spack/docs (#48747)
Bumps [isort](https://github.com/pycqa/isort) from 5.13.2 to 6.0.0.
- [Release notes](https://github.com/pycqa/isort/releases)
- [Changelog](https://github.com/PyCQA/isort/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pycqa/isort/compare/5.13.2...6.0.0)
---
updated-dependencies:
- dependency-name: isort
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Spec.__contains__: restrict to direct build and transitive runtime deps (#49072)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* mochi-margo/mochi-thallium: new versions (#49037)
* mochi-margo/mochi-thallium: new versions
* mochi-thallium: fixing style
* mochi-thallium: fixing required dependency on mochi-margo versions
* Add new recipe aotriton for rocm. (#49038)
* add new reciple aotriton for rocm. used for py-torch
* update the git info
* fix style error
* fix style error
* fix style error
* address review comments
* fix style error
* update maintainers (#48295)
* acts dependencies: new versions as of 2025/02/17 (#49073)
This commit adds detray v0.88.1, covfie v0.12.0 and v0.12.1, as well as
ACTS v37.1.0.
* magma: remove cuda_arch constraint on 2.9.0+ (#49019)
* plsm: new package (#48875)
* Adding plsm package
* [@spackbot] updating style on behalf of PhilipFackler
* Removing redundant text
* Add description
* Add blank line
* Add cuda_arch and update int64 handling
* gnutls: add v3.8.9 (#49062)
* gnutls: add v3.8.9
* gnutls: address super small nitpick
* gnutls: fix git url
* qemacs: add v6.4.1, fix +doc (#48722)
* py-fastjsonschema: add 2.17 to 2.21.1; note python dependencies (#47926)
* py-fastjsonschema: add 2.17 to 2.21.1; note python dependencies
* py-importlib-resources: add v5.13 to 6.4
* Revert "py-importlib-resources: add v5.13 to 6.4"
This reverts commit 1df208874c799b99dcfc43f13ae85f9324c59b52.
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Signed-off-by: Justin Cook <jscook@lbl.gov>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Etienne Ndamlabin <88906611+endamlabin@users.noreply.github.com>
Co-authored-by: Etienne Ndamlabin <jean-etienne.ndamlabin-mboula@inria.fr>
Co-authored-by: Sreenivasa Murthy Kolam <sreenivasamurthy.kolam@amd.com>
Co-authored-by: Thomas Bouvier <contact@thomas-bouvier.io>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Co-authored-by: G-Ragghianti <33492707+G-Ragghianti@users.noreply.github.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: David Schneller <12698011+davschneller@users.noreply.github.com>
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: snehring <7978778+snehring@users.noreply.github.com>
Co-authored-by: Buldram <buldram@proton.me>
Co-authored-by: jmuddnv <143751186+jmuddnv@users.noreply.github.com>
Co-authored-by: Thomas-Ulrich <ulrich@geophysik.uni-muenchen.de>
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Filippo Spiga <spiga.filippo@gmail.com>
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Co-authored-by: Richard Berger <rberger@lanl.gov>
Co-authored-by: Jonathon Anderson <17242663+blue42u@users.noreply.github.com>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Eric Berquist <727571+berquist@users.noreply.github.com>
Co-authored-by: wspear <wspear@cs.uoregon.edu>
Co-authored-by: wspear <wspear@users.noreply.github.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Chris Marsh <chrismarsh.c2@gmail.com>
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
Co-authored-by: psakievich <psakiev@sandia.gov>
Co-authored-by: Brian Spilner <Try2Code@users.noreply.github.com>
Co-authored-by: Dominic Hofer <6570912+dominichofer@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: danielsjensen1 <dsjense@sandia.gov>
Co-authored-by: Till Ehrengruber <till.ehrengruber@cscs.ch>
Co-authored-by: Henri Menke <henri@henrimenke.de>
Co-authored-by: pauleonix <paul.grosse-bley@ziti.uni-heidelberg.de>
Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com>
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov>
Co-authored-by: Taillefumier Mathieu <29380261+mtaillefumier@users.noreply.github.com>
Co-authored-by: Mathieu Taillefumier <mathieu.taillefumier@free.fr>
Co-authored-by: Piotr Sacharuk <107190444+PiotrSacharuk@users.noreply.github.com>
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: rfbgo <109985755+rfbgo@users.noreply.github.com>
Co-authored-by: Felix Thaler <thaler@cscs.ch>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Co-authored-by: japlews <22622327+japlews@users.noreply.github.com>
Co-authored-by: George Young <A-N-Other@users.noreply.github.com>
Co-authored-by: Stephen Nicholas Swatman <stephen@v25.nl>
Co-authored-by: Davis Herring <herring@lanl.gov>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
Co-authored-by: Izaak "Zaak" Beekman <contact@izaakbeekman.com>
Co-authored-by: ddement <ddement@gatech.edu>
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
Co-authored-by: Matt Thompson <matthew.thompson@nasa.gov>
Co-authored-by: SXS Bot <31972027+sxs-bot@users.noreply.github.com>
Co-authored-by: sxs-bot <sxs-bot@users.noreply.github.com>
Co-authored-by: AMD Toolchain Support <73240730+amd-toolchain-support@users.noreply.github.com>
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
Co-authored-by: Alberto Sartori <alberto.sartori@huawei.com>
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
Co-authored-by: Vinícius <viniciusvgp@gmail.com>
Co-authored-by: Teague Sterling <teaguesterling@users.noreply.github.com>
Co-authored-by: Shane Nehring <snehring@iastate.edu>
Co-authored-by: Sergey Kosukhin <sergey.kosukhin@mpimet.mpg.de>
Co-authored-by: Robert Mijakovic <robert.mijakovic@gmail.com>
Co-authored-by: Paul <bryantpj@ornl.gov>
Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
Co-authored-by: Vanessasaurus <814322+vsoch@users.noreply.github.com>
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
Co-authored-by: Howard Pritchard <howardp@lanl.gov>
Co-authored-by: jgraciahlrs <gracia@hlrs.de>
Co-authored-by: Fernando Ayats <ayatsfer@gmail.com>
Co-authored-by: Tim Haines <thaines.astro@gmail.com>
Co-authored-by: renjithravindrankannath <94420380+renjithravindrankannath@users.noreply.github.com>
Co-authored-by: Peter Scheibel <scheibel1@llnl.gov>
Co-authored-by: alalazo <alalazo@users.noreply.github.com>
Co-authored-by: Tara Drwenski <tdrwenski@users.noreply.github.com>
Co-authored-by: Justin Cook <jscook@lbl.gov>
Co-authored-by: jean-francois-sa <jleblancrichard@simplyanalytics.com>
Co-authored-by: etiennemlb <eti.malaboeuf@gmail.com>
Co-authored-by: Richard Berger <richard.berger@outlook.com>
Co-authored-by: Dmitri Smirnov <dmixsmi@gmail.com>
Co-authored-by: John Biddiscombe <biddisco@cscs.ch>
Co-authored-by: Dave Keeshan <96727608+davekeeshan@users.noreply.github.com>
Co-authored-by: Rémi Lacroix <remi.lacroix@idris.fr>
Co-authored-by: Vivek Kale <11766050+vlkale@users.noreply.github.com>
Co-authored-by: vlkale <vlkale@users.noreply.github.com>
Co-authored-by: Marc T. Henry de Frahan <marc.henrydefrahan@nrel.gov>
Co-authored-by: Robert Maaskant <RobertMaaskant@users.noreply.github.com>
Co-authored-by: Matthew Lesko <matthew.w.lesko@nasa.gov>
Co-authored-by: Paul Gessinger <hello@paulgessinger.com>
Co-authored-by: Vicente Bolea <vicente.bolea@kitware.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pranav Sivaraman <pranavsivaraman@gmail.com>
Co-authored-by: Filippo Barbari <filippo.barbari@gmail.com>
Co-authored-by: Matthieu Dorier <mdorier@anl.gov>
Co-authored-by: Nai-Yuan Chiang <sorakid507@gmail.com>
Co-authored-by: Cameron Rutherford <rcamruzz@amazon.com>
Co-authored-by: Philip Fackler <49726797+PhilipFackler@users.noreply.github.com>
Co-authored-by: dmagdavector <david.magda@vectorinstitute.ai>
* update cairo for new meson build system
* update patch range. remove old conflict
* style
* update pango to reflect the changes in cairo
* refine depends
* style
* add lzo depends
* add +shared
* non self-referential variant requireme
* style
* Move +shared variant back to just autotools as meson automatically handles it
* clarify patch when=
* update based on reviews. switch from conflicts to requires to enforce variant synchronization
* refine conflicts and requires
* better group build deps together
* comment for meson build lower version bound
* clarifying comments
* clarify version ranges, enforce build_system with version ranges
* style
* cairo: no need to require for build_systems
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Up to now, Spack was allowing all build-tools that
may appear in the DAG to have 2 max_dupes.
This is not needed in practice for most of them,
and adding them out of caution just increases
grounding and concretization time.
This PR makes the value of max_dupes configurable
per package, and sets only a few known packages to
2 max_dupes by default.
In case user needs different values, they can
tune the configuration for their use case.
On macOS, prefix_a/file and prefix_b/FILE map to the same file view/file or view/FILE.
This commit ensures that we test whether a view is created on a case insensitive filesystem and handle projection conflicts accordingly.
With this change spec["pkg"] searches only direct dependencies and transitive link/run
dependencies, ordered by depth. This avoids situations where we pick up unwanted
deps of build/test deps.
To reach those, you need to do spec["build_dep"]["pkg"] explicitly.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
meson's `dependency` function often uses pkg-config to locate a dependency, and may even fall back to cmake. The former case is very common, and since packagers often forget to add the tiny pkgconfig package as a build dep, we do it for them.
Currently this is hardcoded to the same value as listed in the class
definition. If one ever overrides this attribute, such as:
```
packages:
openblas:
package_attributes:
libraries = [ 'libopenblaso64', ]
```
this patch will make sure that override also in the
`spec['openblas'].libs()` call. (Which happens in `hypre`, likely
others).
( see
https://spack.readthedocs.io/en/latest/packages_yaml.html#assigning-package-attributes
)
Thanks to becker33 for debugging help in Slack
* Name of zlib's library differs on Windows; also account for name
differing when building +shared
* `zlib`'s `.libs` implementation was searching for the runtime
libraries (the .dlls) and should be searching for link-time libs
* pandoramonitoring: add v3.6.0; pandorapfa: add v4.11.2
Remove variables that are not being used in pandorasdk. Use the C++ standard
from ROOT when possible and pass -Wno-error to override the -Werror that will
typically fail with a new standard. Add a cxxstd variant for pandorasdk
* Fix style
* Update var/spack/repos/builtin/packages/pandorasdk/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Fix style
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* h5hut: Remove H5_USE_110_API for newer versions
* h5hut: style reformat and add maintainer
* h5hut: correct version syntax for <v2.x.x
* h5hut: Bump default version to 2.0.0rc7 and remove older rc candidates
* Update var/spack/repos/builtin/packages/h5hut/package.py
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
---------
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
cray-mpich now has a rocm variant. You can use gtl_lib in the
flag_handler like so:
```python
def flag_handler(self, name, flags):
wrapper_flags = []
environment_flags = []
build_system_flags = []
if self.spec.satisfies("+rocm"):
if self.spec.satisfies("^cray-mpich"):
gtl_lib = self.spec["cray-mpich"].package.gtl_lib
build_system_flags.extend(gtl_lib.get(name) or [])
# hipcc is not wrapped, we need to pass the flags via the
# build system.
build_system_flags.extend(flags)
return (wrapper_flags, environment_flags, build_system_flags)
```
---------
Co-authored-by: Richard Berger <rberger@lanl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Richard Berger <richard.berger@outlook.com>
* postgresql: add version 17.2
* postgresql: install flex, bison and perl when building versions 17 and up
* postgresql: do not install perl by default when building versions 17 and up
This PR is effectively a breaking change extracted from #45189, which removes
support for spec["mpi"] if spec itself is openmpi / mpich that could provide mpi;
from the Spec instance we don't have any parent it provides it to,
hence it's a KeyError.
* g4vg: add develop version
* celeritas: add develop version
* Fix style
* REVERTME: move celeritas changes to another branch
* Get full repo
* remove unneeded variable
Currently, a few packages using kokkos rely on
kokkos itself monkeypatching its own spec to
provide some attribute.
In this commit we change this attribute to be
defined on the package, and never be monkeypatched.
Currently, when we setup the ASP problem for `clingo`, we don't take into account the configuration. This results in setting up ASP problems that are larger than necessary, with possibly redundant information, and higher concretization times.
This PR tries to improve things by adding an opt-in feature that computes the _possible dependencies_ of a solve taking also into account the current configuration, and avoids adding possible dependencies that we are certain can't be in the final solution.
The feature can be activated with:
```yaml
concretizer:
static_analysis: true
```
Examples of simple rules to discard dependencies are:
- Dependencies that are not buildable, and for which no binary is present (e.g. `cray-mpich` etc. on non Cray systems)
- Dependencies that are not for the current platform (e.g. `msmpi` on non Windows platforms)
- Conditional dependencies that cannot be activated, because of some user requirement (e.g. `cuda` etc. if the user requires `~cuda` in configuration)
- Virtual providers that cannot be used, because of a requirement on a virtual
The speed-up these rules seem to give depends on the use case at hand, but if the configuration is updated properly, they are noticeable.
Since in cases where there is no rule to exclude packages upfront, reuse is active, and this option is activated, it's possible to see some minor slow down, the feature has been added as opt-in, so it's turned off by default.
* Bump up the version for rocm-6.3.2 release
* rocm-openmp-extras update and style correction
* Updating mivisionx, omniperf, rccl & rocprofiler-systems
* Updating hipsparselt & rocm-opencl
* rocprofiler-systems on gcc-13 and rvs commit instead of patch
* Updated rocjpeg & rocm-examples for 6.3.2
* ROCPROFSYS_BUILD_DYNINST & DYNINST_BUILD_TBB are required only with gcc-13
---------
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
Current location is within the Spack prefix, which causes builds to
pollute VCS with stage artifacts and generally inflates the Spack
install prefix.
This PR moves it to the user cache location now that we can
consistently support paths with spaces on Windows.
With CFLAGS, the code path in the amdfftw build system will bypass the logic around AMD_ARCH.
---------
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
* salt: Add v0.3.0 of SALT
This version contains important bug fixes for building and parsing
projects containing Fortran
* salt: Be more explicit about dependency types
- llvm+clang+flang is needed at build, link and runtime for the
correct operation of SALT
- Testing with llvm@master ( llvm > 19.x ) shows that SALT is
currently incompatible with the latest llvm API so an updated salt
will be required when LLVM 20 is released
`relocate_links` warns when the target is absolute and not matched by
any prefix from the prefix to prefix map.
This can lead to false positives, cause the prefix to prefix map does
not contain trivial/identity entries whenever a package is installed to
its original location.
Since relocate_links is the odd one out there (we don't warn about
similar issues with rpaths, etc), just remove the warning.
Codecov needs to see the token secret when uploading, so we have to
add this line to the workflow YAML:
```yaml
with:
token: ${{ secrets.CODECOV_TOKEN }}
```
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Remove variable from cmake.py
#48775 left a dangling variable that was not caught in CI but by the eyes of @haampie. Restructure variable to local method.
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
* Update lib/spack/spack/build_systems/cmake.py
* Update lib/spack/spack/build_systems/cmake.py
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword
* Revert "Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword"
This reverts commit faf9b8ce85.
* Apply workaround for oneAPI compiler
* style problem resolved
* use spec.satisfies syntax
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Add ruff configuration to `pyproject.toml`.
This allows `ruff format` in the Spack repository to format all the files we care about,
with our line length of 99, the exceptions we already put in place, and excluding things
we don't auto-format, like vendored dependencies.
Right now it'll reformat 175 or so files, but only slightly, in places where `ruff` differs from
`black`. For the most part I like the ruff format decisions better than `black`, but none of
the changes seem too severe.
This does not change `spack style` -- I figure that can come later but this at least will
let people start playing with `ruff`.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
These are some changes that `ruff check --fix` would make that the current
`spack style` also agrees with. Make the changes now so that the `ruff`
change is less disruptive.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
The import-check action now presents problematic import statements
introduced by the PR better.
The idea is roughly:
* Let (V₁, E₁) be the graph of modules as vertices and import statements
as edges before the change
* Let (V₂, E₂) be the graph after the code change, which is typically a small
perturbation of (V₁, E₁).
* X₁ = FAS(V₁, E₁) is the feedback arc set before (a minimal set of edges to
delete to make it acyclic)
* X₂ = FAS(V₂, E₂ ∖ X₁) is the feedback arc set after deletion of the minimal
set of edges that made the old graph acyclic.
* X₃ = FAS(V₂, E₂) is the feedback arc set after
Previously I displayed X₁ and X₃ and users had to diff themselves.
Now, I'm showing X₂, which is a small set, typically directly related to
code changes.
However, it can be that a small code change adding say 2 problematic imports
creates a completely different solution X₃ that only requires deletion of just 1
different import. In that case the user is informed that they can potentially do
less work.
So for PR #48784 the output is now:
> The overall number of problematic import statements increased by 1 from 31 to 32.
> This is likely a direct consequence of the following import statements:
>
> ```
> spack/config imports: spack.spec, spack.util.path, spack.util.remote_file_cache
> ```
>
> However, instead of removing 3 import statements, it is sufficient to remove only 1
> import statement from the following list:
>
> ```
> spack/concretize imports: spack.bootstrap, spack.solver.asp
> spack/environment imports: spack.bootstrap, spack.environment
> spack/fetch_strategy imports: spack.version.git_ref_lookup
> spack/install_test imports: spack.build_environment, spack.package_base
> spack/modules imports: spack.modules
> spack/platforms imports: spack.config
> spack/relocate imports: spack.bootstrap
> spack/repo imports: spack.package_base, spack.patch, spack.tag
> spack/spec imports: spack.binary_distribution, spack.compiler, spack.compilers, spack.concretize, spack.environment, spack.hash_types, spack.provider_index, spack.repo, spack.spec_parser, spack.store, spack.traverse, spack.variant, spack.version.git_ref_lookup
> spack/subprocess_context imports: spack.environment
> spack/util/gpg imports: spack.bootstrap
> spack/util/package_hash imports: spack.package_base
> spack/util/path imports: spack.config, spack.environment
> spack/util/remote_file_cache imports: spack.util.web
> ```
from which the user can figure out that
`spack/util/remote_file_cache imports: spack.util.web` is the "bottleneck" now.
* Add versions 2 and 3 of py-sphinx-rtd-theme.
Allow for versions of py-sphinx greater than 6.
Fix the Python version for older versions that depend on distutils.
Get the py-docutils dependency from the py-sphinx recipe.
* Depend purely on the py-docutils dependency in py-sphinx.
* More refined dependency versioning.
* Fixed versioning for py-sphinx and py-docutils.
Currently, environments created from manifest files with relative includes result in broken
references to config files.
This PR modifies `spack env create` to create local copies in the new environment of any local
config files from relative paths in the environment manifest passed as an init file.
This PR does not change the behavior if the include is an absolute path or if the include is from
a relative path outside the environment directory, but it does warn about missing relative includes if
they are inside the environment directory.
Includes regression test and short blurb in docs.
* Added salt variant to tau
* Update package.py
* [@spackbot] updating style on behalf of wspear
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* Create SALT package.py
Added a package for the SALT Source AnaLysis Toolkit
@zbeekman
* [@spackbot] updating style on behalf of wspear
* Update package.py
Line wrap
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* py-mdit-py-plugins: Add new versions 0.3.5, 0.4.2
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* py-myst-parser: Add new versions 0.19.0 to 4.0.0
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* hpctoolkit: Add +docs variant and manpages
This commit unconditionally enables manpages for the HPCToolkit tools.
The new `+docs` variant enables additional documentation, specifically
the user's manual. Both require new build-time dependencies.
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
---------
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* package api: drop wildcard re-export
To ensure package repos are forward/backward compatibility with Spack,
we should explicitly export all symbols we want to expose in the public
package API, and drop `from spack.something import *` because
removal/addition to the public API will go unnoticed.
Also `llnl.util.filesystem` has some methods that shouldn't be exposed
in the package API, so better to enumerate a subset explicitly.
* remove flatten_dependencies / install_dependency_symlinks
* JAX: add v0.4.34
* Disable search for clang
* Update CUDA flags
* Add py-jax 0.4.33, comment out until py-jaxlib 0.4.33 is also released
* Fix GCC build
* Try TF_NVCC_CLANG
* py-jax: add v0.4.34
* jax no longer has separate tags for jaxlib
* Install compiled wheel
* Join path before glob
* Wheel is in spack stage, not tmp path
* Add 0.4.35
* Add newer versions
* Build system has been refactored yet again
* Drop clang
* Fix build with source tarball, rocm support
* Support GCC
* Remove clang-specific compiler flags
* enable_cuda flag was removed
* Fix logic
* py-jax: add v0.4.38
* Add patch to fix GCC support
* Patch no longer needed
* Skip patching, directly pass flags
* New flags
* Remove unused import
* Patch changed
* Use older version of patch
* Newer patch
* Add CUDA symlink
* Symlink more directories
* Recursive symlink
* Import function
* Recursive search
* Undo cuda changes
* Add v0.5.0
* I quit
* Improve definition of a few placeholder packages
These packages are placeholders for vendor provided software,
that is not buildable, and should be declared as external.
* ibm-java: remove package, as asked by maintainer
Regressed in #47126
Spack was not interpreting mirrors using relative path with respect to the
metadata directory.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* test_no_matching_compiler_specs: does not need mock_low_high_config,
since mutable_config is already used at class level
* bindist.py: setup a configuration that doesn't super-impose builtin.mock
over builtin
* builder.py: use a mock configuration for the tests
This commit adds version 6.8.0 of GeoModel. As far as I can tell from
the change notes, there are no changes required to the build
configuration or dependencies.
This adds a new configuration section called `env_vars:` that can be set in an environment.
It looks very similar to the existing `environment:` section that can be added to `modules.yaml`,
but it is global for an entire spack environment. It's called `env_vars:` to deconflate it with spack
environments (the term was too overloaded).
The syntax looks like this:
```yaml
spack:
specs:
- cmake%gcc
env_vars:
set:
ENVAR_SET_IN_ENV_LOAD: "True"
```
Any of our standard environment modifications can be added to the `env_vars` section, e.g.
`prepend_path:`, `unset:`, `append_path:`, etc. Operations in `env_vars:` are performed
on `spack env activate` and undone on `spack env deactivate`.
* Reduce the size of outputted go built binaries
* Remove unused import from go package
* go: remove comment from setup dependents build env
* Add back missing imports after rebase
* Backward compat with Python 3.9 for socket.timeout
* Forward compat with Python [unknown] as HTTPResponse.geturl is deprecated
* Catch timeout etc from .read()
* Some minor simplifications: json.load(...) takes file object in binary mode.
* Fix CDash code which does error handling wrong: non-2XX responses raise.
The conflict is too strict and prevents usable combinations from being
concretized. Some packages depend on pika, but do not include pika
headers in HIP device code files, and can thus be compiled even with the
bad combination of HIP and GCC versions.
* py-awkward: add dependency on py-importlib-metadata and py-fsspec
* Implement suggested constraint and dependency type changes
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
On macOS you cannot unconditionally pass `-rpath`, it can conflict with
`-r`.
mpicc is not the place to inject rpaths to compiler runtime
libraries. That's up to the compiler's config files (spec files in gcc,
config files in llvm).
Also remove some patches that are redundant in newer versions of
openmpi.
Should be sufficient to set CONFIG_SITE to /dev/null before running configure to
prevent any config site file to be loaded, which currently causes non-determinism in builds.
Setting the variable CONFIG_SITE prevents configure files to check $prefix/share/config.site,
$prefix/etc/config.site, $ac_default_prefix/share/config.site, $ac_default_prefix/etc/config.site
so we don't have to patch a block of code.
* Address quoting issue that casuses dev_paths containing @ symbols to parse as versions
* Fix additional location were dev_path was imporperly constructed
The dev_path must be quoted to avoid parsing issues when
paths contain '@' symbols. Also add tests to catch
regression of this behavior
* Format to fix line length
* fix failing tests
* Fix whitespace error
* Add binary_compatibility fixture to test
* Change string formatting to avoid multiline f string
* Update lib/spack/spack/test/concretization/core.py
* Add tmate debug session
* Revert "Add tmate debug session"
This reverts commit 24e2f77e3c.
* Move test and refactor to use env methods
* Update lib/spack/spack/test/cmd/develop.py
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
Since macOS 15 `ld -single_module` warns with a deprecation message,
which makes configure scripts believe the flag is unsupported. That
in turn triggers a code path where `archive_cmds` is set to
```
$CC -r -keep_private_externs -nostdlib ... -dynamiclib
```
instead of just
```
$CC -dynamiclib ...
```
This code path was meant to trigger only on ancient macOS <= 14.4 where
libtool had to add `-single_module`, which is the default since macos
14.4, and is now apparently deprecated because the flag is a no-op for
more than 15 years.
The wrong `archive_cmds` causes actual problems combined with a bug in
OpenMPI's compiler wrapper (`CC=mpicc`), which appends `-rpath` flags,
which cause an error when combined with the `-r` flag added by the
autotools.
Spack's compiler wrapper doesn't do this, but it's likely there are
other compiler wrappers out there that are not aware that `-r` and
`-rpath` cannot be combined.
The fix is to change defaults: `lt_cv_apple_cc_single_mod=yes`.
* BF+ENH: Add wxwidgets 3.2.6 and py-wxpython 4.2.2
Improves compat with newer Python (>3.9) and numpy.
Fix error during configure step (even on at least some older
versions) by including 'pkgconfig' as a build dep so gtk+ is found.
* BF: Make wxpython use spack built wxwidgets instead of rebuilding
* Update var/spack/repos/builtin/packages/py-wxpython/package.py
Avoid using too new version of py-setuptools as license file format is currently not compatible.
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* for config: let `syaml_dict` inherit from `dict` instead of `OrderedDict`. `syaml_dict` now only exists as a mutable wrapper for yaml related metadata.
* for spec serialization / hashing: use `dict` directly
This is possible since we only support cpython 3.6+ in which dicts are ordered.
This improves performance of hash computation a bit. For a larger spec I'm getting 9.22ms instead of 11.9ms, so 22.5% reduction in runtime.
* OSError.errno apparently is int | None, but already part of __str__ anyway so drop
* fix disambiguate_spec_from_hashes
* List[Spec].sort() complains, ignore it
* fix typing of KnownCompiler, and use it in asp.py
Adds packages for the Arkouda server (`arkouda`) and Arkouda python client (`py-arkouda`).
Arkouda server and client are divided into separate packages to allow for them to be
installed independently of one another.
Future work remains to add a `+dev` variant to `py-arkouda`, but that will require additional
supporting packages made available through spack (e.g. for `py-pytest-env`
Modifications:
* Fix a severe bug where a spliced spec has the same hash as the original build spec
* Removed CacheManager in favor of install_mockery. The latter sets up a temporary store, and removes it at the end
of the test.
* Deleted a couple of helper functions e.g. _has_dependencies
* Checked that SolverException is raised, rather than Exception (more specific)
* Extended, and renamed the splicing_setup fixture (now called install_specs)
* Added more specific assertions in each test
One test is currently flaky, due to some instability when sorting multiple specs. It's currently marked xfail
The methods spack.spec.Spec.concretize and spack.spec.Spec.concretized
are deprecated in favor of spack.concretize.concretize_one.
This will resolve a circular dependency between the spack.spec and
spack.concretize in the next Spack release.
* add python bindings variant to chapel
* fix chpl_home, update chapel frontend rpath in venv
* fix chpl frontend shared lib rpath hack
* patch chapel env var line length limit
* patch chapel python shared lib location by chapel version
* unhack chpl frontend lib rpath
* fix postinstall main version number file path
* update chapel version number to 2.3
* use chapel releases instead of source tarballs, remove dead code
* Chapel 2.3 adds support for LLVM 19
* Bundled LLVM always requires CMake 3.20:
* Apply 2.3 patch for LLVM include search path
Fixes build errors for `chapel@2.3+rocm` of the form:
```
compiler/llvm/llvmDebug.cpp:174:9: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
compiler/llvm/llvmDebug.cpp:254:15: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
```
* fix style
* Fix misreporting of test_hello failures
`test_part` was aliasing the enclosing test name, leading to confusing and incorrect reporting on test failure.
* Ensure `libxml2` optional dep of `hwloc` is also added to `PKG_CONFIG_PATH` in the run env
* patch chplCheck for GPU + multilocale configs, install docs
* Adjust patch for checkChplInstall
* Ensure `CHPL_DEVELOPER` is unset for `~developer`
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
* Improve support using external modules with zsh
Spack integrates with external modules by launching a python subprocess
to scrape the path from the module command. In zsh, subshells do not
inheret functions defined in the parent shell (only environment
variables). This breaks the module function in module_cmd.py.
As a workaround, source the module commands using $MODULESHOME prior to
running the module command.
* Fix formatting
* Fix flake error
* Add guard around sourcing module file
* Add improved unit testing to module src command
* Correct style
* Another attempt at style
* formatting again
* formatting again
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
* bdsim: update to point to the new location in github and add 1.7.7
* Remove the C++ standard patch
* Remove the cmake_args and add a comment instead
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
`spack find` and other commands that read the DB will fail if the databse is in a
read-only location, because `FailureTracker` and `Database` can try to create their
parent directories when they are constructed, even if the DB root is read-only.
Instead, we should only write to the filesystem when needed.
- [x] don't create parent directories in constructors
- [x] have write operations create parents if needed
- [x] in tests, `chmod` databases to be read-only unless absolutely needed.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Adding the PACE library to the LAMMPS build
* Adding the PACE library for ML-PACE in LAMMPS
---------
Co-authored-by: Richard Berger <rberger@lanl.gov>
* Add py-protobuf@4.24.4 (needed for py-cylc-flow@8.3.6).
* Add py-cylc-flow@8.3.6 and enable png output when creating graphs
by requesting variant pangocairo for graphviz dependency.
* Add corresponding versions of py-metomi-rose@2.3.2,
py-cylc-rose@1.4.2, py-cylc-uiserver@1.4.2.
* Add myself as maintainer to all the cylc-related packages.
This PR allows using the subscript notation directly on packages. The
intent is to reduce the boilerplate needed to retrieve package
properties from nodes other than root.
New year, new versions of the ACTS dependencies. This time, we have a
new version of detray, a new version of algebra plugins, as well as a
new version of ACTS itself.
* Add type-hints to `spack.util.executable.Executable`
* Add type-hint to input
* Use overload, and remove assertions at calling sites
* Bump mypy to v1.11.2 (working locally), Python to 3.13
* root: Restrict patch range
* root: Set minimum gcc version for cxxstd=20
* root: fix gcc range when cxxstd 20
Co-authored-by: Paul Gessinger <hello@paulgessinger.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* add parse tool, json, and load json
bug fix
add variants and conflictions for g4tendl
* correct format
* correct format
* update version mapping
* remove 1.0
* Update var/spack/repos/builtin/packages/geant4-data/package.py
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* add options
* depends on it
* fix typo
---------
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* e4s cray rhel ci stack: re-enable and update for new cpe, should fix cray libsci issue
* only run e4s-cray-rhel stack
* Mkae Autotools build_system point at correct build_directory
* remove selective enable of cray-rhel stacks
* restore SPACK_CI_DISABLE_STACKS
* use dot prefix to hide cray-sles jobs instead of comment-out
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* edm4hep: Add lower clang version bounds
EDM4hep 0.99.1 introduced usage of consteval. While this is technically
supported in clang versions below 17, the implementation seems to be
incomplete and fail compilation of EDM4hep 0.99.1 and up.
* Update var/spack/repos/builtin/packages/edm4hep/package.py
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
---------
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
* py-py-spy: add 0.4.0
* py-py-spy: port from Package to CargoPackage
* CargoPackage: respect make_jobs
* rust: respect make_jobs during build and install
* spack style
* CargoBuilder: fix make_jobs syntax
* CargoBuilder: don't write to $HOME, use stage dir
* rivet: add version 3.1.11 with some back-port fixes
Some back-ported fixes from the 4.x series. Most importantly the
possibility to build against HepMC 3.3.0.
* yoda: Add version 1.9.11
* Add a conflict with hepmc=2
* Unify hepmc@3.3.0 conflicts into one statement
* Remove conflict and update conditional hepmc variant
* Use caret to signify conflict with dependency
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Make conflict and msg more specific
* Add dedicated version dependency to yoda
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* gromacs: remove dependency on Fortran
Fortran was removed from GROMACS core since 4.6. There are a few contrib
files around, but they are not built anyway.
Also fix a couple typos.
* [@spackbot] updating style on behalf of al42and
---------
Co-authored-by: al42and <al42and@users.noreply.github.com>
* zoltan: Ignore errors about incompatible pointer type with gcc@14
In gcc 14 -Werror=incompatible-pointer-types is now the default.
Silence it until it gets fixed in zoltan upstream.
* zoltan: Fix linking error when built with ifx
* mpich: gather in a single place env modifications needed by mpich derivatives
MPICH, and its derivatives, share a lot of copy/paste code to setup the
environment during the different stages of the package life-cycle.
This commit gathers the common modifications in a single place (a mixin class),
living in the Mpich package, and makes derivatives import, and reuse, it.
* Fix docs for Python < 3.13
* openloops: Add cmodel to user config file to override setting added in 2.1.2.
* openloops: Change cmodel from small to large following author recommendation.
A few changes to tarball creation (for build caches):
- do not run file to distinguish binary from text
- file is slow, even when running it in a batched fashion -- it usually reads all bytes and has slow logic to categorize specific types
- we don't need a highly detailed file categorization; a crude categorization of elf, mach-o, text suffices.
detecting elf and mach-o is straightforward and cheap
- detecting utf-8 (and with that ascii) is highly accurate: false positive rate decays exponentially as file size increases. Further it's not only the most common encoding, but the most common file type in package prefixes.
iso-8859-1 is cheaply (but heuristically) detected too, and sufficiently accurate after binaries and utf-8 files are classified earlier
- remove file as a dependency of Spack in general, which makes Spack itself easier to install
- detect file type and need to relocate as part of creating the tarball, which is more cache friendly and thus faster
mpicxx_shared_libraries seems a relic of #1550, and is
not currently used by any builtin package.
Thus, cleanup the recipes, and avoid monkey-patching
spec objects.
Python >= 3.13 does not have the crypt variant anymore. Still no matter
if the test for crypt succeeds (which it can on Fedora providing its own
crypt module for Python 3.13) or fails, it will add +crypt or ~crypt,
which both fail because the variant only exists until Python 3.12.
Co-authored-by: Richard Berger <rberger@lanl.gov>
* silo: variant python needs python
* Dependency to Python did not resolve the "Python.h" header not being found
Added the -I path to Python header to the compiler.
Having silo depend on python was not sufficient to get the path to python.
Maybe there is a smarter way to do that, but this one works.
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* thepeg: Correctly specify rivet version for dependency
* herwig3: Add latest version 7.3.0
* thepeg: Make sure to have consistent hepmc version
rivet and thepeg need to have the same hepmc version otherwise things
will not compile
* libzip links with external libs found on host feelpp/spack#6
* add maintainers
* fix style
* use multi-build system
/cc @wdconinc
* fix style
* rm space and rename variant bz2 to bzip2
/cc @wdconinc
* fix variant name for bzip2
* zstd is supported in libzip@:1.8
* fix style
* fix style
* fix style
* rm deprecated version and versions that cannot be found easily
use only cmake from now on
* fix style
* fix style
* use variant when option for zstd
* Update var/spack/repos/builtin/packages/libzip/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* fix style
/cc @wdconinc
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
`kcov` was removed in Ubuntu 24.04, and it is no longer
installable via `apt` in our CI images. Instal it via
Linuxbrew instead, at least until it comes back to Ubuntu.
`subversion` is also not installed on ubuntu 24 by default,
so we have to install it manually.
- [x] Add linuxbrew to linux tests
- [x] Install `kcov` with brew
- [x] Install subversion with `apt`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Fix silent error when reporting builds to CDash
CDash has a 191 char maximum for build names. When this
is exceeded, CDash silently fails to correctly process the
reported XML. This truncates CDash build names to 190 chars
and emits a warning indicating it is doing so to prevent
such errors from occuring.
* test/reporters.py: add unittest for buildname len issue
* test/reporters.py: rename cdash buildname test
* ci/common.py: fix syntax causing breaking test
It appears that the CDash reporter is expecting a string
as the buildname.
* Update lib/spack/spack/reporters/cdash.py
Fix warning message to reflect actual issue.
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* ci/common.py: fix function call to actually call function
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: psakievich <psakiev@sandia.gov>
* py-typer: add version 0.15.1 and "standard" optional dependencies
* py-typer: remove variant that only exists in source, not sdist. Remove trailing .0 from versions.
* Improve variant robustness for dd4hep and edm4hep
Now variants won't be "false" if there's a typo.
* Use libs instead of manual prefix paths
* Improve cmake for another hep package
* Fix variant use and style
* Use directories for ODD
* r-rlas is a dependency for r-lidr
* new package r-lidr w/ suggests to address masking issues
* fixed flake8 issues and added maintainers
* removed boost import statement for flake sake
* Score-P: Replace with-or-without, document options that are not currently explicitly mapped in package for mpi and shmem.
* trim long lines
---------
Co-authored-by: wrwilliams <wrwilliams@users.noreply.github.com>
Improve our typing by updating some todo locations in the code to use
`Literal` instead of a simple `str`.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Fix issue reported by some users regarding some build dependencies.
* Remove invalid configure-time flag that was recently introduced.
Signed-off-by: Samuel K. Gutierrez <samuel@lanl.gov>
There are still more fix ups required for the missing libs to work as
expected in CI. Dropping the error requirement in favor of moving to a
log scraping method until we can verify all package issues have been
addressed correctly.
* Set the "build_jobs" on concretization/generate for CI
build_jobs also controls the concretization pool size. Set this
in the config section for CI generate.
This config is overwritten by build_job CI using the SPACK_BUILD_JOBS
environment variable. This implicitly will drop the default build
CPU request on all "default" grouped build jobs from (max) 16 to 8.
* Add default allocations for build jobs
* Add common jobs and concretize args to ci generate and rebuild
* CI: Specify parallel concretize and build jobs via argument
* Increase power and cray concretization limits
Lowering limits for these stacks creates timeout
* Increase default pool size to 8
intermittent timeouts with 4 CPU
* Add reduced requests for windows for now
This turns some variant-specific methods for dealing with when-keyed dictionaries into
more generic versions, in preparation for conditional version definitions.
`_by_name`, `_names`, etc. are replaced with generic methods for transforming
when-keyed dictionaries:
* `_by_subkey()`
* `_subkeys()`
* `_num_definitions()`
* `_definitions()`
* `_remove_overridden_defs()`
And the variant accessors are refactored to use these methods underneath.
To do this, types like `WhenDict` had to be generified, and some `TypeVars`
were added for sortable keys and values.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
We are using more and more typing features in Spack, and without features like
protocols, typing core is becoming harder and harder.
I think it's worth vendoring `typing_extensions` for this. It will get us a number of
useful capabilities:
* `Literal`
* `TypedDict`
* `Protocol`
among others.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This commit adds a config option `config:shared_linking:missing_library_policy:error/warn/ignore` which will cause installation errors or warnings when ELF executables or libraries need shared libraries which cannot be resolved from RPATH search paths. The default is to ignore.
This is a safeguard against accidentally linking to system libraries instead of Spack libraries. It makes it more likely that build cache installs work on different machines. It works only at the level of libraries, not at the level of symbols. Some system dependencies are allowed (e.g. kernel and libc).
Packages can (but are discouraged to) set `unresolved_libraries` to a list of patterns of sonames/library names that are know to be unresolvable in RPATHs. In the future this could be made more fine-grained in a non-breaking way by allowing a dictionary of patterns `lib => [deps]`.
Extracted #45189
Common test setup has been extracted in fixtures. Some matrix
dimensions moved from being "compiler" to be "targets".
Use --fake install for packages in test.
The `_normal` attribute on specs is no longer used and has no meaning.
It's left over from part of the original concretizer.
The `concrete` constructor argument is also not used by any part of core.
- [x] remove `_normal` attribute from `Spec`
- [x] remove `concrete` argument from `Spec.__init__`
- [x] remove unused `check_diamond_normalized_dag` function in tests
- [x] simplify `Spec` constructor and docstrings
I tried to add typing to `Spec` here, but it creates a huge number of type issues
because *most* things on `Spec` are optional. We probably need separate `Spec` and
`ConcreteSpec` classes before attempting that.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* nim: fix deps, deprecate and patch old versions
* Fix runtime dependencies for produced binaries:
- Add -rpaths pointing at dependencies to
std wrapper modules
- Set version constraints for OpenSSL
- Added SQLite3 variant for <2.0
* Parallelize build with make
* Deprecate 1.0.10 due to CVEs
* Deprecate 1.9.3 as it's an old development version
* Backport patch for CVE-2021-21372 to <1.2.10/<1.4.4,
CVE-2021-21374 to 1.4.2 and CVE-2021-46872 to 1.4.*
* Avoid empty low ranges that include devel
* Add previously missing CVE comment
* Keep "link" type for dynamic libraries for MSVC
* Omit "run" type for library dependencies
* Disable SQLite variant by default
* Fix version ranges
Had assumed they were exclusive, but they're inclusive
* Correct version range for sqlite variant
Difference doesn't matter outside of development versions
* Move patches to use GitHub URLs instead of files
* Retry CI
* append ?full_index=1
Previously the pip setup would delete the visitmodule during the install
step. This was fixed by forcing the pip setup to only run once before
the dependents are created.
Add missing encoding=utf-8 to various open calls. This makes
files like spec.json, spack.yaml, spack.lock, config.yaml etc locale
independent w.r.t. text encoding. In practice this is not often an
issue since Python 3.7, where the C locale is promoted to
C.UTF-8. But it's better to enforce UTF-8 explicitly, since there is
no guarantee text files are written in the right encoding.
Also avoid opening in text mode if it can be avoided.
* `f.tell` on a `TextIOWrapper` does not return the offset in bytes, but
an opaque integer that can only be used for `f.seek` on the same
object. Spack assumes it's a byte offset.
* Do not open in a locale dependent way, but assume utf-8 (and allow
users to override that)
* Use tempfile to generate a backup/temporary file in a safe way
* Comparison between None and str is valid and on purpose.
Follow-up to #47956
* Rename `token.py` -> `tokenize.py`
* Rename `parser.py` -> `spec_parser.py`
* Move common code related to iterating over tokens into `tokenize.py`
* Add "unexpected character token" (i.e. `.`) to `SpecTokens` by default instead of having a separate tokenizer / regex.
* Python: deprecate 3.8
* Remove preference for EOL Python versions
* Explicitly deprecate things requiring EOL Python
* More deprecations
* deprecate old versions of slepc, py-petsc4py, py-slepc4py in sync with old versions of petsc
---------
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
The use of `^` in `depends_on` directives has never been allowed, since
the dawn of Spack.
Up to now, we used to have an audit to catch this kind of issue, mainly
because in that way we could easily collect all issues and report them
to packagers at once.
Due to implementation details, this audit doesn't work if a dependency
without a `^` is followed by the same dependency with a `^`.
This PR makes this pattern an error, which will be reported eagerly, and
removes the corresponding audit. It also fixes a package using the wrong
idiom.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
So far, the ESMF package recipe in spack assumes that the spack
compilers clang and apple-clang are using gfortran as the Fortran
compiler. But with the latest improvements to the LLVM compilers,
we need to also support clang with flang.
Reorganize the pipeline generation aspect of the ci module,
mostly to separate the representation, generation, and
pruning of pipeline graphs from platform-specific output
formatting.
Introduce a pipeline generation registry to support generating
pipelines for other platforms, though gitlab is still the only
supported format currently.
Fix a long-existing bug in pipeline pruning where only direct
dependencies were added to any nodes dependency list.
* Set the "build_jobs" on concretization/generate for CI
build_jobs also controls the concretization pool size. Set this
in the config section for CI generate.
This config is overwritten by build_job CI using the SPACK_BUILD_JOBS
environment variable. This implicitly will drop the default build
CPU request on all "default" grouped build jobs from (max) 16 to 8.
* Add default allocations for build jobs
* Add common jobs and concretize args to ci generate and rebuild
* CI: Specify parallel concretize and build jobs via argument
* Increase power and cray concretization limits
Lowering limits for these stacks creates timeout
* py-nbclassic: add v1.1
* py-nbclassic: reduce explicit dependencies for v1.1.0
Having all the 'excess' packages listed did not break anything, as
they were needed for `py-jupyter-server` (pulled in via `py-notebook-shim`)
anyway, but the change makes it more clear on why things are being pulled in.
* acts dependencies: new versions as of 2024/12/08
This commit includes a new version of ACTS, as well as new versions of
the ACTS algebra plugins, covfie, detray, and geomodel.
* Fixes
* covfie: depends_on cmake@3.21: when @0.11:
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Add py-param@2.1.1
* Add py-panel@1.5.2
* Add py-bokeh@3.5.2
* New package py-datashader
* New package py-geoviews
* New package py-holoviews
* WIP: new package py-uxarray
* New package py-antimeridian
* New package py-dask-expr
* New package py-spatialpandas
* New package py-hvplot
* Add dependency on py-dask-expr for 'py-dask@2024.3: +dataframe'
* Added all dependencies for py-uxarray; still having problems with py-dask +dataframe / py-dask-expr
* Fix style errors in many packages
* Clean up comments and fix style errors in var/spack/repos/builtin/packages/py-dask-expr/package.py
* In var/spack/repos/builtin/packages/py-dask/package.py: since 2023.8, the dataframe variant requires the array variant
* Fix style errors in py-uxarray package
- [x] Clean up arguments on the `resource` directive.
- [x] Add type annotations
- [x] Add `resource` to type annotations on `PackageBase`
- [x] Fix up `resource` docstrings
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Some of the class-level annotations were wrong, and some were missing. Annotate all the
functions here and fix the class properties to match what's actually happening.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
The first argument to each Spack directive is not a `PackageBase` instance but a
`PackageBase` class object, so fix the type annotations to reflect this.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`Optional` shouldn't be part of `PatchesType` -- it's clearer to specify `Optional` it
in the methods that need their arguments to be optional.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* update py-numl and py-nugraph recipes
this commit adds the develop branch as a valid option for each of these two packages. in order to enable this, package tarballs are now retrieved from the github source repository instead of pypi, and their checksums and the build system have been updated accordingly.
* rename versions "develop" -> "main" to be consistent with branch name
* nim: add latest versions
In addition:
- Create separate build and install phases.
- Remove koch nimble call as it's redundant with koch tools.
- Install all additional tools bundled with Nim instead of only Nimble.
* Fix 1.6 version
* nim: add devel
In addition:
- Fix build accessing user config/cache
* Bump Kokkos and Kokkos-kernels to 4.5.00
* petsc@:3.22 add a conflict with this new version of kokkos
* Update kokkos/kokkos-kernel dependency
---------
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
the nis module was removed in python 3.13
we had it default to ~nis
no package requires +nis
required dependencies for +nis were missing
so better to remove the nis module entirely.
* acts dependencies: new versions as of 2024/11/25
This commit adds a new version of detray and two new versions of vecmem.
* acts dependencies: new versions as of 2024/12/02
This commit adds version 38 of ACTS and a new version of detray.
In preparation for adding `when=` to `version()`, I'm cleaning up the types in
`version_types` and making sure the methods here pass `mypy` checks. This started as an
attempt to use `ConcreteVersion` outside of `spack.version` and grew into a larger type
refactor.
The hierarchy now looks like this:
* `VersionType`
* `ConcreteVersion`
* `StandardVersion`
* `GitVersion`
* `ClosedOpenRange`
* `VersionList`
Note that the top-level thing can't easily be `Version` as that is a method and it
returns only `ConcreteVersion` right now. I *could* do something fancy with `__new__` to
make `Version` a synonym for the `ConcreteVersion` constructor, which would allow it to
be used as a type. I could also do something similar with `VersionRange` but not sure if
it's worth it just to make these into types.
There are still some places where I think `GitVersion` might not be handled properly,
but I have not attempted to fix those here.
- [x] Add a top-level `VersionType` class that all version types extend from
- [x] Define and document common methods and rich comparisons on `VersionType`
- [x] Replace complicated `Union` types with `VersionType` and `ConcreteVersion` as needed
- [x] Annotate most methods (skipping `__getitem__` and friends as the typing is a pain)
- [x] Fix up the `VersionList` constructor a bit
- [x] Add cases to methods that weren't handling all `VersionType`s
- [x] Rework some places to clarify typing for `mypy`
- [x] Simplify / optimize _next_version
- [x] Make StandardVersion.string a property to enable lazy comparison
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This visitor accepts the sub-dag of all nodes and unique edges that have
deptype X directly from given roots, or deptype Y transitively for any
of the roots.
LLVM can be a transitive link dependency of hip through gl's dependency mesa, which uses it for software rendering.
In this case make sure llvm-amdgpu is found with find_package(LLVM) and
find_package(Clang) by setting LLVM_ROOT and Clang_ROOT.
That makes the patch of find_package's HINTS redundant, so remove that.
It did not work anyways, because CMAKE_PREFIX_PATH has higher precedence
than HINTS.
Explcitly sets the CMake variables Faodel_INCLUDE_DIRS and Faodel_LIBRARY_DIRS when +faodel.
This seems to be needed for recent versions of seacas (seacas@2021-04-02:), but should be safe
to do for all versions.
For Faodel_INCLUDE_DIRS, it looks like Faodel has header files under $(Faodel_Prefix)/include/faodel,
but seacas is not including the "faodel" part in #includes. So add both $(Faodel_Prefix)/include
and $(Foadel_Prefix)/include/faodel
Co-authored-by: payerle <payerle@users.noreply.github.com>
* c/c++ flags should have been modified for all 2023.x.y versions, but
upper bound was too low
* Fortran flags should have been modified for all 2024.x.y versions, but
likewise the upper bound was too low
* gromacs: announce deprecation policy and start to implement
* Style it up
* [@spackbot] updating style on behalf of mabraham
* Bump versions used in CI
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Permit configuring GROMACS with support for mdrun to trace its timing
regions by calling the ITT API. This permits tools like VTune and
unitrace to augment their analysis with GROMACS-specific annotation.
external googletest breaks dependents because they end up with
ITK_LIBRARIES set to `GTest::GTest;GTest::Main`, which then end up
literally in a nonsensical link line `-lGTest::GtTest`.
the vendored googletest produces a cmake config file where
`ITKGoogleTest_LIBRARIES` is empty.
* gromacs: oneapi does not always require gcc
* Support intel_provided_gcc only with %intel classic compiler
Require gcc only when needed with %intel
* New approach depending on gcc-runtime directly
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
---------
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
* Visit: Add new versions 3.4.0 and 3.4.1
* Adios2: Restrict python, 3.11 doesn't not work for older Adios2
* VisIt: Set the VTK_VERSION for @3.4:
Older versions of VTK used the VTK_{MAJOR, MINOR}_VERSION variables for
VTK detection. VisIt >= 3.4 uses the full string VTK_VERSION.
* CI: Don't build llvm-amdgpu for non-HIP stack
* VisIt: v3.4.1 handles newer Adios2 correctly
* Visit: Add missing links in HDF5, set correct VTK version configuration parameter
* VisIt: Add py-pip requirement and patch visit with configuration changes
* HDF5 symlinks move when inside of callback
* VisIt ninja install fails with python module. Using make does not
* VisIt 3.4 has a high minimum cmake requirement
* HDF5: Early return when not mpi for mpi symlinks
* HDF5: Use platform agnostic method for creating legacy compatible MPI symlinks
* Fix VISIT_VTK_VERSION handling for 8.2.1a hack
* This extends PR #47285 to properly include some of the required version constrains of cgal 6 incl C++ standard. It also adds the new no-gmp backend as a variant.
* fix style
* disable cgal@6 +demo variant as the demos require qt6 which is not in spack
* disable the gmp variant until clarity on how its supposed to work is provided. bound shared and header_only variants to relevant versions
* Fix missing msvc compiler limit, fix variant left in
* Add more comments. Better describe the gmp variant. Remove testing code
* fix style
The default build of clang on darwin couldn't actually build anything
because of a lack of a sysroot built in. Also several compilation
errors finding the system libc++ cropped up, much like those in GCC, and
have been fixed.
* packages: Update 'yambo'
* add call to 'resource' method to download Ydriver and iotk during fetch instead of during build
* air-gapped installation could be performed since version 5.2.1
* add versions 5.2.3 and 5.2.4
* remove some inexistant configure options for versions "@5:"
* add a sanity_check on 'bin/yambo'
The gni libfabric provider works on some Cray systems, but not all. For
example, Slingshot-based machines use a different libfabric provider
(cxi). Therefore libfabric/gni should not be a dependency when using
Cray PMI.
* celeritas: remove deprecated versions through 0.3
* celeritas: deprecate old versions
* celeritas: add c++20 option
* Propagate vecgeom CUDA requirements
* Remove outdated conflicts and format it
Automatic splicing say `Spec` grow a `__len__` method but it's only used
in one place and it's not clear the semantics are useful elsewhere. It also
runs the risk of Specs one day being confused for other types of containers.
Rather than introduce a new function for one algorithm, let's use a more
specific method in the splice code.
- [x] Use topological ordering in `_resolve_automatic_splices` instead of
sorting by node count
- [x] delete `Spec.__len__()` and `Spec.__bool__()`
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
`spack spec` output has looked like this for a while:
```console
> spack spec /v5fn6xo /wd2p2v7
Input spec
--------------------------------
- /v5fn6xo
Concretized
--------------------------------
[+] openssl@3.3.1%apple-clang@16.0.0~docs+shared build_system=generic certs=mozilla arch=darwin-sequoia-m1
[+] ^ca-certificates-mozilla@2023-05-30%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
...
Input spec
--------------------------------
- /wd2p2v7
Concretized
--------------------------------
[+] py-six@1.16.0%apple-clang@16.0.0 build_system=python_pip arch=darwin-sequoia-m1
[+] ^py-pip@23.1.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
```
But the input spec is right there on the CLI, and it doesn't add anything to the output.
Also, since #44843, specs concretized in the CLI line can be unified, so it makes sense
to display them as we did in #44489 -- as one multi-root tree instead of as multiple
single-root trees.
With this PR, concretize output now looks like this:
```console
> spack spec /v5fn6xo /wd2p2v7
[+] openssl@3.3.1%apple-clang@16.0.0~docs+shared build_system=generic certs=mozilla arch=darwin-sequoia-m1
[+] ^ca-certificates-mozilla@2023-05-30%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] ^gmake@4.4.1%apple-clang@16.0.0~guile build_system=generic arch=darwin-sequoia-m1
[+] ^perl@5.40.0%apple-clang@16.0.0+cpanm+opcode+open+shared+threads build_system=generic arch=darwin-sequoia-m1
[+] ^berkeley-db@18.1.40%apple-clang@16.0.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=darwin-sequoia-m1
[+] ^bzip2@1.0.8%apple-clang@16.0.0~debug~pic+shared build_system=generic arch=darwin-sequoia-m1
[+] ^diffutils@3.10%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^libiconv@1.17%apple-clang@16.0.0 build_system=autotools libs=shared,static arch=darwin-sequoia-m1
[+] ^gdbm@1.23%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^readline@8.2%apple-clang@16.0.0 build_system=autotools patches=bbf97f1 arch=darwin-sequoia-m1
[+] ^ncurses@6.5%apple-clang@16.0.0~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=darwin-sequoia-m1
[+] ^pkgconf@2.2.0%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^zlib-ng@2.2.1%apple-clang@16.0.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=darwin-sequoia-m1
[+] ^gnuconfig@2022-09-17%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] py-six@1.16.0%apple-clang@16.0.0 build_system=python_pip arch=darwin-sequoia-m1
[+] ^py-pip@23.1.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] ^py-setuptools@69.2.0%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[-] ^py-wheel@0.41.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
...
```
With no input spec displayed -- just the concretization output shown as one consolidated
tree and multiple roots.
- [x] remove "Input Spec" section and "Concretized" header from `spack spec` output
- [x] print concretized specs as one BFS tree instead of multiple
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
This PR provides complementary 2 features:
1. An augmentation to the package language to express ABI compatibility relationships among packages.
2. An extension to the concretizer that can synthesize splices between ABI compatible packages.
1. The `can_splice` directive and ABI compatibility
We augment the package language with a single directive: `can_splice`. Here is an example of a package `Foo` exercising the `can_splice` directive:
class Foo(Package):
version("1.0")
version("1.1")
variant("compat", default=True)
variant("json", default=False)
variant("pic", default=False)
can_splice("foo@1.0", when="@1.1")
can_splice("bar@1.0", when="@1.0+compat")
can_splice("baz@1.0+compat", when="@1.0+compat", match_variants="*")
can_splice("quux@1.0", when=@1.1~compat", match_variants="json")
Explanations of the uses of each directive:
- `can_splice("foo@1.0", when="@1.1")`: If `foo@1.0` is the dependency of an already installed spec and `foo@1.1` could be a valid dependency for the parent spec, then `foo@1.1` can be spliced in for `foo@1.0` in the parent spec.
- `can_splice("bar@1.0", when="@1.0+compat")`: If `bar@1.0` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `bar@1.0+compat` in the parent spec
- `can_splice("baz@1.0", when="@1.0+compat", match_variants="*")`: If `baz@1.0+compat` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `baz@1.0+compat` in the parent spec, provided that they have the same value for all other variants (regardless of what those values are).
- `can_splice("quux@1.0", when=@1.1~compat", match_variants="json")`:If `quux@1.0` is the dependency of an already installed spec and `foo@1.1~compat` could be a valid dependency for the parent spec, then `foo@1.0~compat` can be spliced in for `quux@1.0` in the parent spec, provided that they have the same value for their `json` variant.
2. Augmenting the solver to synthesize splices
### Changes to the hash encoding in `asp.py`
Previously, when including concrete specs in the solve, they would have the following form:
installed_hash("foo", "xxxyyy")
imposed_constraint("xxxyyy", "foo", "attr1", ...)
imposed_constraint("xxxyyy", "foo", "attr2", ...)
% etc.
Concrete specs now have the following form:
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "foo", "attr1", ...)
hash_attr("xxxyyy", "foo", "attr2", ...)
This transformation allows us to control which constraints are imposed when we select a hash, to facilitate the splicing of dependencies.
2.1 Compiling `can_splice` directives in `asp.py`
Consider the concrete spec:
foo@2.72%gcc@11.4 arch=linux-ubuntu22.04-icelake build_system=autotools ^bar ...
It will emit the following facts for reuse (below is a subset)
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "hash", "foo", "xxxyyy")
hash_attr("xxxyyy", "version", "foo", "2.72")
hash_attr("xxxyyy", "node_os", "ubuntu22.04")
hash_attr("xxxyyy", "hash", "bar", "zzzqqq")
hash_attr("xxxyyy", "depends_on", "foo", "bar", "link")
Rules that derive abi_splice_conditions_hold will be generated from
use of the `can_splice` directive. They will have the following form:
can_splice("foo@1.0.0+a", when="@1.0.1+a", match_variants=["b"]) --->
abi_splice_conditions_hold(0, node(SID, "foo"), "foo", BaseHash) :-
installed_hash("foo", BaseHash),
attr("node", node(SID, SpliceName)),
attr("node_version_satisfies", node(SID, "foo"), "1.0.1"),
hash_attr("hash", "node_version_satisfies", "foo", "1.0.1"),
attr("variant_value", node(SID, "foo"), "a", "True"),
hash_attr("hash", "variant_value", "foo", "a", "True"),
attr("variant_value", node(SID, "foo"), "b", VariVar0),
hash_attr("hash", "variant_value", "foo", "b", VariVar0).
2.2 Synthesizing splices in `concretize.lp` and `splices.lp`
The ASP solver generates "splice_at_hash" attrs to indicate that a particular node has a splice in one of its immediate dependencies.
Splices can be introduced in the dependencies of concrete specs when `splices.lp` is conditionally loaded (based on the config option `concretizer:splice:True`.
2.3 Constructing spliced specs in `asp.py`
The method `SpecBuilder._resolve_splices` implements a top-down memoized implementation of hybrid splicing. This is an optimization over the more general `Spec.splice`, since the solver gives a global view of exactly which specs can be shared, to ensure the minimal number of splicing operations.
Misc changes to facilitate configuration and benchmarking
- Added the method `Solver.solve_with_stats` to expose timers from the public interface for easier benchmarking
- Added the boolean config option `concretizer:splice` to conditionally load splicing behavior
Co-authored-by: Greg Becker <becker33@llnl.gov>
We added unification semantics for parsing specs from the CLI, but there are a couple
of special cases in which we can avoid calls to the concretizer for speed when the
specs can all be resolved by lookups.
- [x] special case 1: solving a single spec
- [x] special case 2: all specs are either concrete (come from a file) or have an abstract
hash. In this case if concretizer:unify:true we need an additional check to confirm
the specs are compatible.
- [x] add a parameterized test for unifying on the CI
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* libunwind: Add 1.7.2 and 1.8.1
* libunwind: Remove deprecated 1.1 version
* libunwind: Add newer *-stable branches: Remove 1.5-stable branch as well as cleanup.
* libunwind: Use GitHub url for all versions
* libunwind: Add conflict for PPC and 1.8.*
* libunwind: Add conflict for aarch64 and 1.8:
Build fails with
aarch64/Gos-linux.c: In function '_ULaarch64_local_resume':
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
}
^
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
make[2]: *** [Makefile:4795: aarch64/Los-linux.lo] Error 1
* added updated versions
* added pyhmmer
* updated infernal
* fix blast-plus for apple-clang
* fix py-biopython build on apple-clang
* remove erroneous biopython dep: build issue is with python 3.8, not biopython
* deepsig python 3.9: expanding unnecessary python restrictions
* add pyrodigal
* fix unnecessarily strict diamond version
* builds and updates: blast-plus indexing broken, still need to test db download and bakta pipeline
* builds and runs
* revert blast-plus changes: remove my personal hacks to get blast-plus to build
* fix the build error during compilation of rocdecode.was dependent on libva-devel packag
* address review comment
* address review changes.commit the changes
* Add two_level_namespace variant (default is disabled) for MacOS to enable building
executables and libraries with two level namespace enabled.
* Addressed reviewer comments.
* Moved two_level_namespace variant ahead of the patch that uses that variant to
get concretize to work properly.
* Removed extra print statements
* soqt: Add SoQt package
The geomodel package needs this if visualization is turned on.
* make qt versions explicit
* use virtual dependency for qt
* pr feedback
Remove myself as maintainer
Remove v1.6.0
Remove unused qt variant
This addresses part [1] of #46345#44713 introduced a bug where all non-spec query parameters like date
ranges, -x, etc. were ignored when an env was active.
This fixes that issue and adds tests for it.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
`spack mirror add` and `set` now have flags `--oci-password-variable`, `--oci-password-variable`, `--s3-access-key-id-variable`, `--s3-access-key-secret-variable`, `--s3-access-token-variable`, which allows users to specify an environment variable in which a username or password is stored.
Storing plain text passwords in config files is considered deprecated.
The schema for mirrors.yaml has changed, notably the `access_pair` list is generally replaced with a dictionary of `{id: ..., secret_variable: ...}` or `{id_variable: ..., secret_variable: ...}`.
the py-oracledb package only has a single outdated version available in its recipe. this PR adds a much broader range of versions and their corresponding checksums.
* add more versions of py-oracledb
* update py-oracledb recipe
* add py-cython version dependencies
* tweak py-cython version dependencies
* remove older versions of py-oracledb
This filters any selected executable ending with `-ocl` from the list of executables being probed as candidate for external `llvm` installations.
I couldn't reproduce the entire issue, but with a simple script:
```
#!/bin/bash
touch foo.o
echo "clang version 10.0.0-4ubuntu1 "
echo "Target: x86_64-pc-linux-gnu"
echo "Thread model: posix"
echo "InstalledDir: /usr/bin"
exit 0
```
I noticed the executable was still probed:
```
$ spack -d compiler find /tmp/ocl
[ ... ]
==> [2024-11-11-08:38:41.933618] '/tmp/ocl/bin/clang-ocl' '--version'
```
and `foo.o` was left in the working directory. With this change, instead the executable is filtered out of the list on which we run `--version`, so `clang-ocl --version` is not run by Spack.
- [x] Get rid of a call to `parser.quote_if_needed()` during solver setup, which
introduces a circular import and also isn't necessary.
- [x] Rename `spack.variant.Value` to `spack.variant.ConditionalValue`, as it is *only*
used for conditional values. This makes it much easier to understand some of the
logic for variant definitions.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`conditional()`, which defines conditional variant values, and the other ways to declare
variant values should probably be in a layer above `spack.variant`. This does the simple
thing and moves *just* `conditional()` to `spack.directives` to avoid a circular import.
We can revisit the public variant interface later, when we split packages from core.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* lua: add +pcfile support for @5.4: versions, without using a version-dependent patch
* lua: always generate pcfile, remove +pcfile variant from all packages
* lua: minor fixes
* rpm: minor fix
* Add 5.030 and remove the requirement to patch verilator, the problem has be fixed in this rev
* Update var/spack/repos/builtin/packages/verilator/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* benchmark: enable shared libraries by default
The existing behaviour of Google Benchmark yiels static objects which
are of little use for most projects. This PR changes the spec to use
dynamic libraries instead.
* Add shared variant
* librdkafka: added missing dependency on curl
This PR adds a missing dependency on curl in librdkafka.
* librdkafka: added dependency on openssl and zlib
* Added support for Codeplay AMD Plugin for Intel OneAPI Compilers
* [@spackbot] updating style on behalf of kaanolgu
* Adding 2025.0.0
* removed HOME and XDG_RUNTIME_DIR
* [@spackbot] updating style on behalf of kaanolgu
---------
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
No ROOT `builtin` should ever be set to true if possible, because that
builds an existing library that spack may not know about.
Furthermore, using `builtin_glew` forces the package to be on, even when
not building x/gl/aqua on macos. This causes build failures.
Caused by https://github.com/spack/spack/pull/45632#issuecomment-2276311748 .
Currently, if a package has a dependency from another repository and patches it,
generation of the patch cache will fail. Concretization succeeds if a fixed patch
cache is in place.
- [x] don't assume that patched dependencies are in the same repo when indexing
- [x] add some test fixtures to support multi-repo tests.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* spack.compiler: cache output
* compute libc from the dynamic linker at most once per spack process
* wrap compiler cache entry in class, add type hints
* test compiler caching
* ensure tests do not populate user cache, and fix 2 tests
* avoid recursion: cache lookup -> compute key -> cflags -> real_version -> cache lookup
* allow compiler execution in test that depends on get_real_version
If a package `foo` doesn't implement `libs`, the default was to search recursively for `libfoo` whenever asking for `spec[foo].libs` (this also happens automatically if a package includes `foo` as a link dependency).
This can lead to some strange behavior:
1. A package that is normally used as a build dependency (e.g. `cmake` at one point) is referenced like
`depends_on(cmake)` which leads to a fully-recursive search for `libcmake` (this can take
"forever" when CMake is registered as an external with a prefix like `/usr`, particularly on NFS mounts).
2. A similar hang can occur if a package is registered as an external with an incorrect prefix
- [x] Update the default library search to stop after a maximum depth (by default, search
the root prefix and each directory in it, but no lower).
- [x]
The following is a list of known changes to `find` compared to `develop`:
1. Matching directories are no longer returned -- `find` consistently only finds non-dirs,
even at `max_depth`
2. Symlinked directories are followed (needed to support max_depth)
3. `find(..., "dir/*.txt")` is allowed, for finding files inside certain dirs. These "complex"
patterns are delegated to `glob`, like they are on `develop`.
4. `root` and `files` arguments both support generic sequences, and `root`
allows both `str` and `path` types. This allows us to specify multiple entry points to `find`.
---------
Co-authored-by: Peter Scheibel <scheibel1@llnl.gov>
This PR adds a sub-command to `spack env` (`track`) which allows users to add/link
anonymous environments into their installation as named environments. This allows
users to more easily track their installed packages and the environments they're
dependencies of. For example, with the addition of #41731 it's now easier to remove
all packages not required by any environments with,
```
spack gc -bE
```
#### Usage
```
spack env track /path/to/env
==> Linked environment in /path/to/env
==> You can activate this environment with:
==> spack env activate env
```
By default `track /path/to/env` will use the last directory in the path as the name of
the environment. However users may customize the name of the linked environment
with `-n | --name`. Shown below.
```
spack env track /path/to/env --name foo
==> Tracking environment in /path/to/env
==> You can activate this environment with:
==> spack env activate foo
```
When removing a linked environment, Spack will remove the link to the environment
but will keep the structure of the environment within the directory. This will allow
users to remove a linked environment from their installation without deleting it from
a shared repository.
There is a `spack env untrack` command that can be used to *only* untrack a tracked
environment -- it will fail if it is used on a managed environment. Users can also use
`spack env remove` to untrack an environment.
This allows users to continue to share environments in git repositories while also having
the dependencies of those environments be remembered by Spack.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
You can now provide multiple roots to a single `find()` call and all of
them will be searched. The roots can overlap (e.g. can be parents of one
another).
This also adds a library function for taking a set of regular expression
patterns and creating a single OR expression (and that library function
is used in `find` to improve its performance).
Set command line scopes last in _main, so they are higher scopes
Restore the global configuration in a spawned process by inspecting
the result of ctx.get_start_method()
Add the ability to pass a mp.context to PackageInstallContext.
Add shell-tests to check overriding the configuration:
- Using both -c and -C from command line
- With and without an environment active
* edm4hep: Add json variant for newer versions
Explicit option has been added to EDM4hep so we now expose it via a
variant as well. We keep the old behavior where we unconditionally
depended on nlohmann-json and implicitly built JSON support if we could
detect it cmake stage
* Fix condition statement in when clause
* Use open version range to avoid fixing to single version
---------
Co-authored-by: Valentin Volkl <valentin.volkl@cern.ch>
Variants can now be propagated from a dependent package to (transitive) dependencies,
even if the source or transitive dependencies have the propagated variants.
For example, here `zlib` doesn't have a `guile` variant, but `gmake` does:
```
$ spack spec zlib++guile
- zlib@1.3%gcc@12.2.0+optimize+pic+shared build_system=makefile arch=linux-rhel8-broadwell
- ^gcc-runtime@12.2.0%gcc@12.2.0 build_system=generic arch=linux-rhel8-broadwell
- ^gmake@4.4.1%gcc@12.2.0+guile build_system=generic arch=linux-rhel8-broadwell
```
Adding this property has some strange ramifications for `satisfies()`. In particular:
* The abstract specs `pkg++variant` and `pkg+variant` do not intersect, because `+variant`
implies that `pkg` *has* the variant, but `++variant` does not.
* This means that `spec.satisfies("++foo")` is `True` if:
* for concrete specs: `spec` and its dependencies all have `foo` set if it exists
* for abstract specs: no dependency of `spec` has `~foo` (i.e. no dependency contradicts `++foo`).
* This also means that `Spec("++foo").satisfies("+foo")` is `False` -- we only know after concretization.
The `satisfies()` semantics may be surprising, but this is the cost of introducing non-subset
semantics (which are more useful than proper subsets here).
- [x] Change checks for variants
- [x] Resolve conflicts
- [x] Add tests
- [x] Add documentation
---------
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* pythia8: Include patch for C++20 / Clang
Pythia8 vendors some FJCore sources that are as of Pythia8 312
incompatible with C++20 on clang. This adds a patch that makes it
compatible in these scenarios
* Add issue link
* rename setup_cxxstd function
* Remove an accidental printout
* Apply patch to all compilers, add lower bound
* `find(..., max_depth=...)` can be used to control how many directories at most to descend into below the starting point
* `find` now enters every unique (symlinked) directory once at the lowest depth
* `find` is now repeatable: it traverses the directory tree in a deterministic order
In the pure `ld` case, we weren't actually parsing `RPATH` arguments separately as we
do for `ccld`. Fix this by adding *another* nested case statement for raw `RPATH`
parsing.
There are now 3 places where we deal with `-rpath` and friends, but I don't see a great
way to unify them, as `-Wl,`, `-Xlinker`, and raw `-rpath` arguments are all ever so
slightly different.
Also, this Fixes ordering of assertions to make `pytest` diffs more intelligible.
The meaning of `+` and `-` in diffs changed in `pytest` 6.0 and the "preferred" order
for assertions became `assert actual == expected` instead of the other way around.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`cc` divides most paths up into system paths, spack managed paths, and other paths.
This gets really repetitive and makes the code hard to read. Simplify the script
by adding some functions to do most of the redundant work for us.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This PR has two small contributions:
- It adds another phase to the timer for concrectization, "construct_specs", to actually see the time the concretizer spends interpreting the `clingo` output to build the Python object for a concretized spec.
- It adds the method `Solver.solve_with_stats` to expose the timers that were already in the concretizer to the public solver API. `Solver.solve` just becomes a special case of `Solver.solve_with_stats` that throws away the timing output (which is what it was already doing).
These changes will make it easier to benchmark concretizer performance and provide a more complete picture of the time spent in the concretizer by including the time spent interpreting clingo output.
Currently, the `geant4-data` spec creates symlink to all of its
dependencies, and it does so by globbing their `share/` directories.
This works very well for the way Spack installs these, but it doesn't
work for anybody wanting to use e.g. the Geant4 data on CVMFS. See pull
request #47298. This commit changes the way the `geant4-data` spec
works. It no longer blindly globs directories and makes symlinks, but it
asks its dependencies specifically for the name of their data directory.
This should allow us to use Spack to use the CVMFS installations as
externals.
* zabbix: add v5.0.44, v6.0.34, v7.0.4 (fix CVEs)
* [@spackbot] updating style on behalf of wdconinc
* zabbix: use f-string
* zabbix: fix f-string quoting
* zabbix: use mysql-client
* @wdconic, this fixes the mysql client virtual for me
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
The idea is to go from most to least used: backward compat -> forward compat -> pinning on major or major.minor version -> pinning specific, concrete versions.
Further, the following
```python
# backward compatibility with Python
depends_on("python@3.8:")
depends_on("python@3.9:", when="@1.2:")
depends_on("python@3.10:", when="@1.4:")
# forward compatibility with Python
depends_on("python@:3.12", when="@:1.10")
depends_on("python@:3.13", when="@:1.12")
depends_on("python@:3.14")
```
is better than disjoint when ranges causing repetition of the rules on dependencies, and requiring frequent editing of existing lines after new releases are done:
```python
depends_on("python@3.8:3.12", when="@:1.1")
depends_on("python@3.9:3.12", when="@1.2:1.3")
depends_on("python@3.10:3.12", when="@1.4:1.10")
depends_on("python@3.10:3.13", when="@1.11:1.12")
depends_on("python@3.10:3.14", when="@1.13:")
When compiled without MPI support, a fake mpi header is autogenerated during configure/build. The header is missing one symbol in version 1.9.5. The problem has since been fixed upstream.
A simular problem does also occur for 1.9.4. Unfortunately, the patch does not work for 1.9.4 and I also don't know if further fixes would be required for 1.9.4. Therefore, only the newest version 1.9.5 is patched.
Update HDF5 version for develop branch to develop-2.0 to match the new
version in the develop branch.
Remove develop-1.16 as it has been decided to make next release HDF5 2.0.0.
This PR changes the semantic of == for spec so that:
hdf5++mpi == hdf5+mpi
won't hold true anymore. It also changes the constrain semantic, so that a
non-propagating variant always override a propagating variant. This means:
(hdf5++mpi).constrain(hdf5+mpi) -> hdf5+mpi
Before we had a very weird semantic, that was supposed to be tested by unit-tests:
(libelf++debug).constrain(libelf+debug+foo) -> libelf++debug++foo
This semantic has been dropped, as it was never really tested due to the == bug.
According to https://github.com/root-project/root/issues/7160, if
`-Dcocoa=ON` build must also be configured with `-Dopengl=ON`, since
otherwise the build encounters missing includes. This is/was a silent
failure in ROOT CMake, but I believe has been made an explicit failure
some time this year.
Currently, the schema reads:
from:
- type:
environment: path_or_name
but this can't be extended easily to other types, e.g. to buildcaches,
without duplicating the extension keys. Use instead:
from:
- type: environment
path: path_or_name
Currently, the `concretizer:unify:` config option only affects environments.
With this PR, it now affects any group of specs given to a command using the `parse_specs(*, concretize=True)` interface.
- [x] implementation in `parse_specs`
- [x] tests
- [x] ensure all commands that accept multiple specs and concretize use `parse_specs` interface
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* don't concretize in CI if changed packages are not in stacks
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Generate noop job when no specs to rebuild due to untouched pruning
* Add test to verify skipping generate creates a noop job
* Changed debug for early exit
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
This changes `Spec` serialization to include information about propagation for abstract specs.
This was previously not included in the JSON representation for abstract specs, and couldn't be
stored.
Now, there is a separate `propagate` dictionary alongside the `parameters` dictionary. This isn't
beautiful, but when we bump the spec version for Spack `v0.24`, we can clean up this and other
aspects of the schema.
* geant4: make downloading data dependency optional
This PR makes downloading the data repository of the Geant4 spec
optional by adding a sticky, default-enabled variant which controls the
dependency on `geant4-data`. This should not change the default
behaviour, but should allow users to choose whether or not they want the
data directory.
* Add comment
* Update env variable
* Generic docs
* Buildable false
- Merging sycl2020usm and sycl2020acc into sycl2020 and the submodel=acc/usm variant is introduced
- implementation is renamed to option
- impl ( fortran implementation options) renamed to foption
- sycl_compiler_implementation and thrust_backend
- stddata,stdindices,stdranges to a single std with std_submodel introduction
- std_use_tbb to be boolean; also changed model filtering algorithm to make sure that it only picks model names
- Modified comments to clear confusion with cuda_arch cc_ and sm_ prefix appends
- Deleted duplicate of cuda_arch definition from +omp
- CMAKE_CXX_COMPILER moved to be shared arg between all models except tbb and thrust
- Replaced sys.exit with InstallError and created a dictionary to simplify things and eliminate excess code lines doing same checks
- Replaced the -mcpu flags to -march since it is deprecated now
- Replaced platform.machine with spec.target
- Removing raja_backend, introducing openmp_flag,removing -march flags,clearing debugging print(), removing excess if ___ in self.spec.variants
- [FIX] Issue where Thrust couldn't find correct compiler (it requires nvcc)
- [FIX] Fortran unsupported check to match the full string
- [FIX] RAJA cuda_arch to be with sm_ not cc_
- dir= option is no longer needed for kokkos
- dir is no longer needed
- [omp] Adding clang support for nvidia offload
- SYCL2020 offload to nvidia GPU
- changing model dependency to be languages rather than build system
- removing hardcoded arch flags and replacing with archspec
- removing cpu_arch from acc model
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* Introduce support for ArmPL and ACfL 24.10
This patch introduces the possibility of installing armpl-gcc
and acfl 24.10 through spack. It also addressed one issue observed
after PR https://github.com/spack/spack/pull/46594
* Fix Github action issues.
- Remove default URL
- Reinstate default OS for ACfL to RHEL.
Fixes an issue reported where `spack env depfile` + `make -j` would
non-deterministically refuse to mark all environment roots explicit.
`update_explicit` had the pattern
```python
rec = self._data[key]
with self.write_transaction():
rec.explicit = explicit
```
but `write_transaction` may reinitialize `self._data`, meaning that
mutating `rec` won't mutate `self._data`, and the changes won't be
persisted.
Instead, use `mark` which has a correct implementation.
Also avoids the essentially incorrect early return in `update_explicit`
which is a pattern I don't think belongs in database.py: it branches on
possibly stale data to realize there is nothing to change, but in reality
it requires a write transaction to know that for a fact, but that would
defeat the purpose. So, leave this optimization to the call site.
The already concrete specs in an environment are now among the reusable specs for the concretizer.
This includes concrete specs from all include_concrete environments.
In addition to this change to the default reuse, `environment` is added as a reuse type for
the concretizer config. This allows users to specify:
spack:
concretizer:
# Reuse from this environment (including included concrete) but not elsewhere
reuse:
from:
- type: environment
# or reuse from only my_env included environment
reuse:
from:
- type:
environment: my_env
# or reuse from everywhere
reuse: true
If reuse is specified from a specific environment, only specs from that environment will be reused.
If the reused environment is not specified via include_concrete, the concrete specs will be retried
at concretization time to be reused.
Signed-off-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Currently the order in which hooks are run is arbitrary.
This can be fixed by sorted(list_modules(...)) but I think it is much
more clear to just have a static list.
Hooks are not extensible other than modifying Spack code, which
means it's unlikely people maintain custom hooks since they'd have
to fork Spack. And if they fork Spack, they might as well add an entry
to the list when they're continuously rebasing.
The idea is that `spack -e env add ./concrete-spec.json` would list the
full hash in the specs, so that (a) it's not ambiguous and (b) it could
in principle results in constant time lookup instead of linear time
substring match in large build caches.
* r-*: updates to latest versions
* r-*: add new dependencies
* r-proj: fix docstring line length
* r-list: add homepage
* r-*: add more dependencies
* r-rmpi: use virtual dependencies, conflict openmpi@5:
* r-cairo: require cairo +png; +pdf for some versions; cairo +fc when +ft
* r-proj: set LD_LIBRARY_PATH since rpath not respected
* Added packages to for intel-2025.0.0 release
* fix style
* pin mkl to 2024.2.2
until e4s can upgrade to 2025 compiler and ginkgo compatibility issue can be resolved.
---------
Co-authored-by: Robert Cohn <rscohn2@gmail.com>
* Add a descriptor to have a class level constant
This descriptor helps intercept places where we set a value on instances.
It does not really behave like "const" in C-like languages, but is the
simplest implementation that might still be useful.
* Add a descriptor to deprecate properties/attributes of an object
This descriptor is used as a base class. Derived classes may implement a
factory to return an adaptor to the attribute being deprecated. The
descriptor can either warn, or raise an error, when usage of the deprecated
attribute is intercepted.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
This adds the current latest version of py-uv. While working on this, I also
found that uv (including older versions) has a build dependency on cmake which
was not specified in the package, so I added it as a dependency.
I also found that on my machine, the build process had trouble finding cmake,
so I set the path to it explicitly as an environment variable.
* py-jupyter-core: set environment variables for extensions
* Changes committed by gh-spack-pr
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Currently, `spack solve` has different spec selection semantics than `spack spec`.
`spack solve` currently does not allow specifying a single spec when an environment is active.
This PR modifies `spack solve` to inherit the interface from `spack spec`, and to use
the same spec selection logic. This will allow for better use of `spack solve --show opt`
for debugging.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* gdk-pixbuf: delete old versions, make mesonpackage
goal is to get rid of `std_meson_args` global, but clean up package
while at it.
`setup_dependent_run_environment` was removed because it did not depend
on the dependent spec, and would result in repeated env variable
changes.
* atk: idem
* fix a dependent
* ML CI: Linux aarch64
* Add config files
* No aarch64 tag
* Don't specify image
* Use amazonlinux image
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
* Update and require
* GCC is too old
* Fix some builds
* xgboost doesn't support old GCC + cuda
* Run on newer Ubuntu
* Remove mxnet
* Try aarch64 range
* Use main branch
* Conflict applies to all targets
* cuda only required when +cuda
* Use tagged version
* Comment out tf-estimator
* Add ROCm, use newer Ubuntu
* Remove ROCm
---------
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
Originally, concretization failed if the splice config points to an invalid replacement.
This PR defers the check until we know the splice is needed, so that irrelevant splices
with bad config cannot stop concretization.
While I was at it, I improved an error message from an assert to a ValueError.
* Normalize Spack Win entrypoints
Currently Spack has multiple entrypoints on Windows that in addition to
differing from *nix implementations, differ from shell to shell on
Windows. This is a bit confusing for new users and in general
unnecessary.
This PR adds a normal setup script for the batch shell while preserving
the previous "click from file explorer for spack shell" behavior.
Additionally adds a shell title to both powershell and cmd letting users
know this is a Spack shell
* remove doskeys
#44588 we added logic to suppress deprecation warnings for the
Intel classic compilers. This depended on matching against
* The compiler names (looking for icc, icpc, ifort)
* The compiler version
When using an Intel compiler with fortran wrappers, the first check
always fails. To support using the fortran wrappers (in combination
with the classic Intel compilers), we remove the first check and
suppress if just the version matches. This works because:
* The newer compilers like icx can handle (ignore) the flags that
suppress deprecation warnings
* The Cray wrappers pass the underlying compiler version (e.g. they
report what icc would report)
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* Update var/spack/repos/builtin/packages/openjdk/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Connection objects are Python version, platform and multiprocessing
start method independent, so better to use those than a mix of plain
file descriptors and inadequate guesses in the child process whether it
was forked or not.
This also allows us to delete the now redundant MultiProcessFd class,
hopefully making things a bit easier to follow.
This allows the following
```python
cache.init_entry("my/cache")
with cache.read_transaction("my/cache") as f:
data = f.read() if f is not None else None
```
mirroring `write_transaction`, which returns a tuple `(old, new)` where
`old` is `None` if the cache file did not exist yet.
The alternative that requires less defensive programming on the call
site would be to create the "old" file upon first read, but I did not
want to think about how to safely atomically create the file, and it's
not unthinkable that an empty file is an invalid format (for instance
the call site may expect a json file, which requires at least {} bytes).
This PR is in response to a question in the `environments` slack channel (https://spackpm.slack.com/archives/CMHK7MF51/p1729200068557219) about inadequate CLI help/documentation for one specific subcommand.
This PR uses the approach I took for the descriptions and help for `spack test` subcommands. Namely, I use the first line of the relevant docstring as the description, which is shown per subcommand in `spack env -h`, and the entire docstring as the help. I then added, where it seemed appropriate, help. I also tweaked argument docstrings to tighten them up, make consistent with similar arguments elsewhere in the command, and elaborate when it seemed important. (The only subcommand I didn't touch is `loads`.)
For example, before:
```
$ spack env update -h
usage: spack env update [-hy] env
positional arguments:
env name or directory of the environment to activate
optional arguments:
-h, --help show this help message and exit
-y, --yes-to-all assume "yes" is the answer to every confirmation request
```
After the changes in this PR:
```
$ spack env update -h
usage: spack env update [-hy] env
update the environment manifest to the latest schema format
update the environment to the latest schema format, which may not be
readable by older versions of spack
a backup copy of the manifest is retained in case there is a need to revert
this operation
positional arguments:
env name or directory of the environment
optional arguments:
-h, --help show this help message and exit
-y, --yes-to-all assume "yes" is the answer to every confirmation request
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Commit aa0825d642 accidentally added a semicolon
to the ANSI escape sequence even if the color code was `None` or unknown, breaking the
bold, uncolored font-face. This PR restores the old behavior.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* Add type hints to all query* methods
* Inline docstrings
* Change defaults from `any` to `None` so they can be type hinted in old Python
* Pre-filter on given hashes instead of iterating over all db specs
* Fix a bug where the `--origin` option of uninstall had no effect
* Fix a bug where query args were not applied when searching by concrete spec
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Fixes a change in behavior/bug in
70412612c7, where partial environment
installs would mark the selected spec as explicitly installed, even if
it was not a root of the environment.
The desired behavior is that roots by definition are the to be
explicitly installed specs. The specs on the `spack -e ... install x`
command line are just filters for partial installs, so leave them
implicitly installed if they aren't roots.
ci: Remove deprecated logic from the ci module
Remove the following from the ci module, schema, and tests:
- deprecated ci stack and handling of old ci config
- deprecated mirror handling logic
- support for artifacts buildcache
- support for temporary storage url
* ParaView: Explicitly set the ENABLE_MPI on/off
* Disallow MPI in the DAG when ~mpi
* @5.13 uses 'remove_children', use pugixml@1.11:, See #47098
* cloud_pipelines/stacks/data-vis-sdk: paraview +raytracing: add +adios2 +fides
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
When building ParaView with ADIOS2 and allowing VTK-m to be
built, also build Fides. This reads ADIOS2 files with a
particular JSON schema, but it requires VTK-m to read data.
* Libmng: Restore Autotools system
CMake, when building its Qt gui, depends on Qt, which in turn, depends on libmng, a CMake based build. To avoid this obvious cyclic dependency, we re-introduce libmng's autotools build into Spack and require when building Qt as a CMake dependency, libmng is built with autotools
* Ensure autotools constraint is limited to non-Windows
* refactor qt-libmng relation from CMake
This commit remove all the uses of spec.compiler that
can be easily substituted by a more idiomatic approach,
e.g. using spec.satisfies or directives
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* py-sphinxcontrib-spelling: new package
* Dependency enchant: Add missing dep on pkgconfig
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Both `multiprocessing.connection.Connection.__del__` and `io.IOBase.__del__` called `os.close` on the same file descriptor. As of Python 3.13, this is an explicit warning. Ensure we close once by usef `os.fdopen(..., closefd=False)`
* stacks: add a stack for devtools on darwin
After getting this whole mess building on darwin, let's keep it that
way, and maybe make it so we have some non-ML darwin binaries in spack
as well.
* reuse: false for devtools
* dtc: fix darwin dylib name and id
On mac the convention is `lib<name>.<num>.dylib`, while the makefile
creates a num suffixed one by default. The id in the file is also a
local name rather than rewritten to the full path, this fixes both
problems.
* node-js: make whereis more deterministic
* relocation(darwin): catch Mach-O load failure
The MachO library can throw an exception rather than return no headers,
this happened in an elf file in the test data of go-bootstrap. Trying
catching the exception and moving on for now. May also need to look
into why we're trying to rewrite an elf file.
* qemu: add darwin flags to clear out warnings
There's a build failure for qemu in CI, but it's invisible because of
the immense mass of warning output. Explicitly specify the target macos
version and remove the extraneous unknown-warning-option flag.
* dtc: libyaml is also a link dependency
libyaml is required at runtime to run the dtc binary, lack of it caused
the ci for qemu to fail when the library wasn't found.
fixes#47101
The bug was introduced in #33495, where `spack find was not updated,
and wasn't caught by unit tests.
Now a Database can accept a custom predicate to select the installation
records. A unit test is added to prevent regressions. The weird convention
of having `any` as a default value has been replaced by the more commonly
used `None`.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* root: fix variant detection for external
A few fixes (possibly non-exhaustive) to `spack external find root`
Several variants have had `when=` clauses added that need to be
propagated to determine_variants. The previously used
Version.satifies("") method also has been removed, it seems. It's
slightly cumbersome that there is no self.spec to use in
determine_variants, but comparisons using Version(version_str) work at least
* remove debug printout
* sccache: add new package
* sccache: add older versions and minimum rust versions
* sccache: add more minimum rust versions
* sccache: add sccache executable and tag as build-tools
* sccache: add dist-server
* sccache: add determine_version and determin_variants
* sccache: add sccache-dist executable
* sccache: fix style
* Update var/spack/repos/builtin/packages/sccache/package.py
* In case building very old sccache <= 5 is not needed with these older rust version is not needed, they can be omitted.
* sccache: drop older versions
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* sccache: add openssl dependency
* sccache: openssl is a linux only dependency?
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Older builds of Boost were failing on Windows because they were
adding --without-... flags for libraries that did not exist in those
versions. So:
* lib variants are updated with version range info (current range
info for libs is not comprehensive, but represents changes over the
last few minor versions up to 1.85)
* On Windows, --without-... options are omitted for libraries when they
don't exist for the version of boost being built. Non-Windows uses
a different approach, which was not affected because the new libraries
were not activated by default. It would benefit from similar attention
though to avoid potential future issues.
#44327 made sure to always run `set_package_py_globals` on all
packages before running `setup_dependent_package` for any package,
so that packages implementing the latter could depend on variables
like `spack_cc` being defined.
This ran into an undocumented dependency: `std_cmake_args` is set in
`set_package_py_globals` and makes use of `cmake_prefix_paths` (if it
is defined in the package); `py-torch`es implementation of
`cmake_prefix_paths` depends on a variable set by
`setup_dependent_package` (`python_platlib`).
This generally restores #44327, and corrects the resulting issue by
moving assignment of `std_cmake_args` to after both actions have been
run.
* py-clip-anytorch: new package
* py-clip-anytorch: ran black
py-langchain-core: ran black
py-pydantic: ran black
py-dalle2-pytorch: ran black
* [py-clip-anytorch] fixed license(checked_by)
* Apply suggestion from Wouter on fixing CI
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Alex C Leute <acl2809@rit.edu>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* py-pytorch-warmup: new package
* py-clip-anytorch: ran black
py-langchain-core: ran black
py-pydantic: ran black
py-dalle2-pytorch: ran black
---------
Co-authored-by: Alex C Leute <acl2809@rit.edu>
Some unused methods in VTK-m resulted in compile errors. These were
not discovered because many compilers ignore unused methods in templated
classes, but the SYCL compiler for Aurora gave an error.
Turns out `os=...` of the spec and `MACOSX_DEPLOYMENT_TARGET` are kept
in sync, and the env variable is used to initialize
`CMAKE_MACOSX_DEPLOYMENT_TARGET`.
In bootstrap code we set the env variable, so these bits are redundant.
---------
Co-authored-by: haampie <haampie@users.noreply.github.com>
* Remove the implicit CORE-AVX512 since the CPU specific flags are added by the
compiler wrappers.
* Add `-i_use-path` to help `ifx` find `lld` even if `-gcc-name` is set in
`ifx.cfg`. This file is written by `intel-oneapi-compilers` package to find the
correct `gcc`. Not being able to find `ldd` is a bug in `ifx`. @rschon2 found
this workaround.
* e4s external rocm ci: upgrade to v6.2.1
* use ghcr.io/spack/spack/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.2.1:2024.10.08
* magma +rocm: add entry for v6.2.1
Extrae normally separates the C and MPI fortran interception libs, but
for mixed C/Fortran applications a combined lib is needed.
Co-authored-by: fpanichi <fpanichi@amd.com>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* [ftgl] Restrict GCC 14+ patch to apply only to GCC 14+
The patch added by #46927 should only be applied where it is needed:
with GCC 11 it causes a compilation failure where none previously
existed.
* Fix the contraint for applying unsiged char patch to ^freetype@2.13.3:
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
The purpose of this CI job is to ensure that we
can use a modern clingo to concretize specs, if
e.g. it was installed in a virtual environment
with pip.
Since there is no need to re-test unrelated parts
of Spack, reduce the number of tests we run to just
concretize.py
* lua: update luarocks resource to 3.11.1
We have kept an older 3.8 for some time, but that now uses an incorrect
value for the deployment target for macos, causing builds for bundles to
succeed but in such a way that they can't be linked to applications by
`ld` but only loaded by dlopen. This fixes that, and also generally
updates the tool.
* lua-luajit-openresty: add new version fix LUA_PATH
Adds a newer version of openresty's luajor, and adds the slightly odd
extra share path they use that contains the `jit.*` modules. Without
that, things that use bytecode-saving and other jit routines (like
neovim) fail.
* lua-lpeg: fix lpeg build to work for neovim on OSX
Normally luarocks builds all lua libraries as bundles on macos, this
makes sense, but means they can't then be linked by LD into executables
the way neovim expects to do. I'm not sure how this ever worked, if it
did. This patch adds the appropriate variables to have luarocks build
the library as a shared librar, and subsequently fix the id with
install_name_tool (the built-in functionality for this does not
trigger).
This also adds a symlink from `liblpeg.dylib` to `lpeg.so` because
neovim will not build on macos without it. See corresponding upstream
pull request at https://github.com/neovim/neovim/pull/30749
Remove the constraint for concrete specs and simply take the
max(version) if a version is not given. This should default to the
highest infinity version which is also the logical best guess for
doing development.
* Remove concrete verision constriant for develop, set docs
* Add unit-test
* Update lib/spack/docs/environments.rst
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
* Update lib/spack/spack/cmd/develop.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Consolidate env collection in cmd
* Style
---------
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Remove the `build-tools` tag of python, otherwise these types of
concretizations are possible:
```
py-root
^py-pip
^python@3.12
^python@3.13
```
So, a package would be configured with py-pip using python 3.12, but
installed for 3.13, which does not work.
* Add new version for master branch
Added new version for master branch. Also added additional functions to ensure tempo will actually run. Tempo assumes the stage directory sticks around and references numerous files and directory there. That has been corrected here only if using the master version. The LWA-10-2020 version will also have this problem but they may have additional setup in their compute/Spack environment to address this issue already so I did not modify anything when that's the version. Example of what happens in the LWA-10-17-2020 version regarding missing files is given below
user@cs:~/spack/bin$ tempo
more: cannot open /tempo.hlp: No such file or directory
* Updated to fix format errors
Flake8 check found errors. Fixed those formatting issues
* Additional format change
Removed redundant setup_dependent_run_environment missed in previous update
* Update url to use https: https is the usual transport and is needed to support checkout behind some firewalls
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
The CMake builder in Spack actually adds incorrect rpaths. They are
unfiltered and incorrectly ordered compared to what the compiler wrapper
adds.
There is no need to specify paths to dependencies in `CMAKE_INSTALL_RPATH`
because of two reasons:
1. CMake preserves "toolchain" rpaths, which includes the rpaths injected
by our compiler wrapper.
2. We use `CMAKE_INSTALL_RPATH_USE_LINK_PATH=ON`, so libraries we link
to are rpath'ed automatically.
However, CMake does not create install rpaths to directories in the package's
own install prefix, so we set `CMAKE_INSTALL_RPATH` to the educated guess
`<prefix>/{lib,lib64}`, but omit dependencies.
Some assertion are not testing DAG invariants, and they are passing only
because of the simple structure of the builtin.mock repository on develop.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* update versions of Neo4j and Redis deps
* deprecating older versions due to security vulnerabilities
* [@spackbot] updating style on behalf of kchilleri
* Update var/spack/repos/builtin/packages/redis/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* adding previous urls to use archives on project websites
* [@spackbot] updating style on behalf of kchilleri
* adding new required maven version
* label when to use specific maven versions
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Krishna Chilleri <krishnachilleri@lanl.gov>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Add versions 1.14.4-3, 1.14.5, develop-1.16, and update develop-1.15 to
develop-1.17.
* Remove unused list_url, list_depth, and style disapproved spaced in url
= "...
* One more style fix.
This PR allows users to configure explicit splicing replacement of an abstract spec in the concretizer.
concretizer:
splice:
explicit:
- target: mpi
replacement: mpich/abcdef
transitive: true
This config block would mean "for any spec that concretizes to use mpi, splice in mpich/abcdef in place of the mpi it would naturally concretize to use. See #20262, #26873, #27919, and #46382 for PRs enabling splicing in the Spec object. This PR will be the first place the splice method is used in a user-facing manner. See https://spack.readthedocs.io/en/latest/spack.html#spack.spec.Spec.splice for more information on splicing.
This will allow users to reuse generic public binaries while splicing in the performant local mpi implementation on their system.
In the config file, the target may be any abstract spec. The replacement must be a spec that includes an abstract hash `/abcdef`. The transitive key is optional, defaulting to true if left out.
Two important items to note:
1. When writing explicit splice config, the user is in charge of ensuring that the replacement specs they use are binary compatible with whatever targets they replace. In practice, this will likely require either specific knowledge of what packages will be installed by the user's workflow, or somewhat more specific abstract "target" specs for splicing, to ensure binary compatibility.
2. Explicit splices can cause the output of the concretizer not to satisfy the input. For example, using the config above and consider a package in a binary cache `hdf5/xyzabc` that depends on mvapich2. Then the command `spack install hdf5/xyzabc` will instead install the result of splicing `mpich/abcdef` into `hdf5/xyzabc` in place of whatever mvapich2 spec it previously depended on. When this occurs, a warning message is printed `Warning: explicit splice configuration has caused the the concretized spec {concrete_spec} not to satisfy the input spec {input_spec}".
Highlighted technical details of implementation:
1. This PR required modifying the installer to have two separate types of Tasks, `RewireTask` and `BuildTask`. Spliced specs are queued as `RewireTask` and standard specs are queued as `BuildTask`. Each spliced spec retains a pointer to its build_spec for provenance. If a RewireTask is dequeued and the associated `build_spec` is neither available in the install_tree nor from a binary cache, the RewireTask is requeued with a new dependency on a BuildTask for the build_spec, and BuildTasks are queued for the build spec and its dependencies.
2. Relocation is modified so that a spack binary can be simultaneously installed and rewired. This ensures that installing the build_spec is not necessary when splicing from a binary cache.
3. The splicing model is modified to more accurately represent build dependencies -- that is, spliced specs do not have build dependencies, as spliced specs are never built. Their build_specs retain the build dependencies, as they may be built as part of installing the spliced spec.
4. There were vestiges of the compiler bootstrapping logic that were not removed in #46237 because I asked alalazo to leave them in to avoid making the rebase for this PR harder than it needed to be. Those last remains are removed in this PR.
Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* cuda: Add 12.6.2
* Update cuda build system
- Remove gcc@6 conflict that was only a deprecation (probably has be added again with cuda@13)
- Update cuda_arch support by CUDA version
- Kepler support has ended with cuda@12
- Recently added 90a Hopper "experimental" features architecture was
missing the dependency on cuda@12:
adds the lima-vm project, in order to make that useful adds a newer
version of qemu so qemu VMs can work, and builds qemu with flags that
allow it to do things like give the VMs networking and virtfs
filesystems.
Also adds vde as a dependency of qemu.
* CMake: Improve incremental build speed.
CMake automatically embeds an updated configure step into make/ninja that will be called during the build phase. By default if a `CMakeCache.txt` file exists in the build directory CMake will use it and this + `spec.is_develop` is sufficient evidence of an incremental build.
This PR removes duplicate work/expense from CMake packages when using `spack develop`.
* Update cmake.py
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
meant self not spec...
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* fix typo in variable name in hepmc3 variant
* set cxx standard to 14 when using protobuf
* add myself to hepmc3 maintainer list
* hepmc3: Applied suggestion of @acecbs for spec.satisfies("+protobuf") (agreed!)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* hepmc3: cxx_standard for protobuf
only set cxx standard to meet protobuf minimum (14) if not also rootio variant as that sets the cxx standard to match the root public API standard requirements
* Environment.clear: ensure clearing is passed through to manifest
* test/cmd/env: make test_remove_commmand round-trip to disk
* cleanup vestigial variables
Some Windows Python installations may store the Python exe in Scripts/
rather than the base directory. Update `.command` to search in both
locations on Windows. On all systems, the search is now done
recursively from the search root: on Windows, that is the base install
directory, and on other systems it is bin/.
* Add libALL support
* cabana: also require ALL
* cabana: Bugfix: Fix spec for cmake>=3.26 to be @3.26: and HDF5 support requires MPI
* cabana: MPI requires C: Add depends_on("c", type="build", when="+mpi")
* cabana: +mpi requires C, but at least for some CMake versions, Cabana's enable of C is too late. Patch it.
* cabana: simplify disabling of find_package's for disabled options and improve comment
* cabana: +grid of 0.6.0 does not compile with gcc-13: It misses iostream includes
* cabana: +test requires googletest at build time: gtest is a linked library(not a plugin or tool)
* cabana: 0.6.0+cuda requires kokkos@3.7:, see https://github.com/ECP-copa/Cabana/releases
* cabana: As 0.6.0+grid does not support gcc-13 and newer, I think it's good to add 0.6.1 and 0.7.0?
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* added py-evodiff and dependencies
* deleted the FIXME
* fixed style issues
* added versions for biotite dependencies; added hash to py-hatch-vcs
* added python version for py-hatch-cython
* updated biotraj dependencies
* - added versions for the packages and dependencies
- added more dependencies for py-hatch
- added rust versions
- added py-uv as a new package
* updated pacakges and their dependencies according to the PR review by @meyersbs
* typo fix for hatchling version; fix the minimum required setuptools version for evodiff
* added 1.9.0 and 1.7.0 userpath versions; required as a dependency
* added mlflow as a dependency
* changed biopython to an optional dependency according to review from @meyersbs; variant esmfold
* Updated Specs
- Pinned biotraj to 1:1 for py-biotite
- Added numpy and other dependencies for py-biotraj; they are dependent
on the versions
- Excluded py-mlflow as a dependency for package py-evodiff; missing
usage in
package.
- Removed versioned dependencies from py-fair-esm
- Added a version to py-packaging
- Added py-setuptools as a dependency in py-userpath
- Added sha256 as hashes for py-uv
* style changes
* Remove "modify_object_macholib"
According to documentation, this function is used when installing
Mach-O binaries on linux. The implementation seems questionable at
least, and the code seems to be never hit (Spack currently doesn't
support installing Mach-O binaries on linux).
* Fix relocation on macOS, when store projection changes
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Enable tests for symlink-based views (this works with almost no
modifications to the view logic). View logic is not yet robust
for hardlink/junction-based views, so those are disabled for now
(both in the tests and as subcommands to `spack view`).
* add new darshan-runtime variants
- `lustre` variant enables instrumentation of Lustre files
* requires Lustre headers, so Lustre is a proper dependency
for this variant
- `log_path` variant allows setting of a centralized log directory
for Darshan logs (commonly used for facility deployments)
* when this variant is used, the `DARSHAN_LOG_DIR_PATH` env var
is no longer used to set the log file path
- `group_readable_logs` variant sets Darshan log permissions to
allow reads from users in the same group
* add mmap_logs variant to enable usage of mmap logs
Also adds support for Paraview and CMake to build with Qt support on
Windows.
The remaining edits are to enable building of Qt itself on Windows:
* Several packages needed to update `.libs` to properly locate
libraries on Windows
* Qt needed a patch to allow it to build using a Python with a space
in the path
* Some Qt dependencies had not been ported to Windows yet
(e.g. `harfbuzz` and `lcms`)
This PR does not provide a sufficient GL for Qt to use Qt Quick2, as
such Qt Quick2 is disabled on the Windows platform by this PR.
---------
Co-authored-by: Dan Lipsa <dan.lipsa@kitware.com>
* Bugfix/Installer: properly track task queueing
* Move ordinal() to llnl.string; change time to attempt
* Convert BuildTask to use kwargs (after pkg); convert STATUS_ to BuildStats enum
* BuildTask: instantiate with keyword only args after the request
* Installer: build request is required for initializing task
* Installer: only the initial BuildTask cannnot have status REMOVED
* Change queueing check
* ordinal(): simplify suffix determination [tgamblin]
* BuildStatus: ADDED -> QUEUED [becker33]
* BuildTask: clarify TypeError for 'installed' argument
* madgraph5amc: add v3.5.6, add a preferred version and remove urls
* Fix format
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
We run tests for more python versions on `develop` than we do for PRs, so codecov
project status is nearly always failing. There is about a 1% difference in max coverage
between `develop` tests and PR tests, so we should increase the project threshold to 2%
to allow for this difference.
The purpose of the project test on PRs is just to make sure that nothing done on the PR
massively affects coverage of code not covered by the PR. This is valuable, but rare. It
only really affects PRs that deal with test or coverage configuration.
- [x] change project coverage threshold from .2% to 2%
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Add v3.0.0 version for libfirefly package
* [@spackbot] updating style on behalf of tbhaxor
---------
Co-authored-by: tbhaxor <tbhaxor@users.noreply.github.com>
`spack gc` has so far been a global or environment-specific thing.
This adds the ability to restrict garbage collection to specific specs,
e.g. if you *just* want to get rid of all your unused python installations,
you could write:
```console
spack gc python
```
- [x] add `constraint` arg to `spack gc`
- [x] add a simple test
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Avoid that hdf5 searches all search paths for ZLIB.cmake config files (inluding /usr/lib), before it looks for zlib without cmake config files, which is how Spack installs it
* WarpX: 24.10
This updates WarpX and dependencies for the 24.10 release.
New features:
- EB runtime control: we can now compile with EB on by default,
because it is not an incompatible binary option anymore
- Catalyst2 support: AMReX/WarpX 24.09+ support Catalyst2 through
the existing Conduit bindings
* Fix Typo in Variant
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve Python Dep Version Ranges
* Add Missing `-DWarpX_CATALYST`
* AMReX: Missing CMake Options for Vis
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
This allows us to keep the workflow file tidier, and avoid
using indirections to perform platform specific operations.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* py-mpi4py: add v4.0.0
* sensei: update mpi4py dependency
build with py-mpi4py@4.0.0 due to fatal no such file or directory error
* petsc4py: update license, and remove C++/Fortran dependency
There was a bit of mystery surrounding the arguments for `_setup_pkg_and_run`. It passes
two file descriptors for handling the `gmake`'s job server in child processes, but they are
unsed in the method.
It turns out that there are good reasons to do this -- depending on the multiprocessing
backend, these file descriptors may be closed in the child if they're not passed
directly to it.
- [x] Document all args to `_setup_pkg_and_run`.
- [x] Document all arguments to `_setup_pkg_and_run`.
- [x] Add type hints for `_setup_pkg_and_run`.
- [x] Refactor exception handling in `_setup_pkg_and_run` so it's easier to add type
hints. `exc_info()` was problematic because it *can* return `None` (just not
in the context where it's used). `mypy` was too dumb to notice this.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* new builtin package: ambertools
* fixes for the style test
* yet more changes for the style test
* hope this is the last fix for the style test
* netlib-xblas is a dependency, it needs a depends_on("m4", type="build")
* ambertools: Add new setuptool dependency, limit python to <= 3.10 (does not build with 3.11+)
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
We mostly use `spack style` and `spack style --fix`, but it's nice to also be able to
run plain old `black .` in the repo.
- [x] Fix includes and excludes `pyproject.toml` so that we *only* cover files we expect
to be blackened.
Note that `spack style` is still likely the better way to go, because it looks at `git
status` and tells black to only check files that changed from `develop`. `black` with
`pyproject.toml` won't do that. Of course, you can always manually specify which files
you want blackened.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* postgresql: Add icu4c dependency for versions 16+
* postgresql: make ICU an option
* postgresql: ICU variant only needed for v16+
* postgresql: Check for negated option
Check for negated option instead of negating the test
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* [py-flash-attn] Add version 2.6.3
* Update dependencies according to the latest version
* Add max_jobs environmental variable to avoid oom error
---------
Co-authored-by: aurianer <8trash-can8@protonmail.ch>
* Revert "`cc`: ensure that RPATHs passed to linker are unique"
This reverts commit 2613a14c43.
* Revert "`cc`: simplify ordered list handling"
This reverts commit a76a48c42e.
Updated the terminology for the two types of environments to be
consistent with that used in the tutorial for the last three years.
Additionally:
* changed 'anonymous' to 'independent in environment command+test for consistency.
* Update package.py
Update to pull Totalview tar files from AWS instead of requiring the user to download ahead of time. Use new license type, RLM license. Only allow for installs of versions using the new license type. 2024.1 and 2024.2. User selects the platform with the version as it is down from the TotalView downloads website.
* Update package.py
Update to pass style test
* Update package.py
fixing syle
* Updating to pass style check
removing more spaces to pass style check
* final style fixes
fixing the last 2 style errors
* Typo
Typo correction to pass style check
* REmove new line
removing new line character
* Ran black to reformat
Ran black to clear errors
* Changing to use sha256
Updating to use sha256 checksums for all TotalView files.
* acts dependencies: new versions as of 2024/09/30
This commit adds new versions of acts, actsvg, and detray.
* Add vecmem version, patch detray version
#45205 already removed previous use of single letter packages
from unit tests, in view of reserving `c` as a language (see #45191).
Some use of them has been re-introduced accidentally in #46382, and
is making unit-tests fail in the feature branch #45189 since there
`c` is a virtual package.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Kokkos: adding some sanity checks
We can pretty much guarentee that if bin, include or lib directory
is missing, something is wrong. Additionally KokkosCore_config.h
and Kokkos_Core.hpp. I guess technically we could look for all
public headers at least but that seems a bit overkill as well?
* Kokkos Kernels: adding sanity checks
* Remove check for lib directory since it might end up being lib64
* Also remove lib from kokkos-kernels sanity check
* Add latest releases of Camp, RAJA, Umpire, CHAI and CARE
* Address review comments + blt requirement in Umpire
* CARE @develop & @main: Submodules -> False
* Changes in Umpire
* Changes in RAJA
* Changes in CHAI
* Changes in RAJA: prefer 'spec.satisfies' to 'in spec'
This is due to a non-equivalence in Spack with providers like mpi.
See e.g. https://github.com/spack/spack/pull/46126
* Changes in Umpire: prefer 'spec.satisfies' to 'in spec'
This is due to a non-equivalence in Spack with providers like mpi.
See e.g. https://github.com/spack/spack/pull/46126
* Changes in CARE:
Still need to update to CachedCMakePackage based on RADIUSS Spack Configs version
* Missing change in RAJA + changes in fmt
* Fix synta
* Changes in Camp
* Fix style
* CHAI: when ~raja, turn off RAJA in build system
* Fix: Ascent@0.9.3 does not support RAJA@2024.07.0
* Enforce same version constraint on Umpire as for RAJA
* Enforce preferred version of vtk-m in ascent 0.9.3
* Migrate CARE package to CachedCMakePackage
* Fix style in CARE package
* CARE: Apply changes for uniform implementation accross RADIUSS projects
* Caliper: move to CachedCMakePackage, from RADIUSS Spack Configs
* Adapt RAJA Perf to spack CI
* Activate CHAI, CARE and RAJAPerf in Spack CI
* Fixes and diffs with RADIUSS Spack Configs
* Caliper: fix
* Caliper : fix + RAJAPerf : style
* RAJAPerf: fixes
* Update maintainers
* raja-perf: fix license header
* raja-perf: Fix variant naming openmp_target -> omptarget
* raja-perf: style and blt dependency versions
* CARE: benchmark and examples off by default (like tests)
* CARE: fix missing variable
* Update var/spack/repos/builtin/packages/raja-perf/package.py
* CARE: fix branch name
* Revert changes in MFEM to pass CI
* Fix CXX17 condition in RAJA + add sycl option in RAJAPerf
---------
Co-authored-by: Rich Hornung <hornung1@llnl.gov>
* cbindgen: new package
* Attempting to add rust dependencies for cbindgen
* adding rust-toml min rust version
* Removing dependencies that don't install with cargo
* cleanup broken packages
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
On sysroot systems like gentoo prefix, as well as nix/guix, our "is
system path" logic is broken cause it's static.
Talking about "the system paths" is not helpful, we have to talk
about default search paths of the dynamic linker instead.
If glibc is recent enough, we can query the dynamic loader's default
search paths, which is a much more robust way to avoid registering
rpaths to system dirs, which can shadow Spack dirs.
This PR adds an **additional** filter on rpaths the compiler wrapper
adds, dropping rpaths that are default search paths. The PR **does
not** remove any of the original `is_system_path` code yet.
This fixes issues where build systems run just-built executables
linked against their *not-yet-installed libraries*, typically:
```
LD_LIBRARY_PATH=. ./exe
```
which happens in `perl`, `python`, and other non-cmake packages.
If a default path is rpath'ed, it takes precedence over
`LD_LIBRARY_PATH`, and a system library gets loaded instead
of the just-built library in the stage dir, breaking the build. If
default paths are not rpath'ed, then LD_LIBRARY_PATH takes
precedence, as is desired.
This PR additionally fixes an inconsistency in rpaths between
cmake and non-cmake packages. The cmake build system
computed rpaths by itself, but used a different order than
computed for the compiler wrapper. In fact it's not necessary
to compute rpaths at all, since we let cmake do that thanks to
`CMAKE_INSTALL_RPATH_USE_LINK_PATH`. This covers rpaths
for all dependencies. The only install rpaths we need to set are
`<install prefix>/{lib,lib64}`, which cmake unfortunately omits,
although it could also know these. Also, cmake does *not*
delete rpaths added by the toolchain (i.e. Spack's compiler
wrapper), so I don't think it should be controversial to simplify
things.
https://docs.sylabs.io/guides/main/admin-guide/configfiles.html#loop-devices
shared loop devices: This allows containers running the same image
to share a single loop device. This minimizes loop device usage and
helps optimize kernel cache usage.
Enabling this feature can be particularly useful for large MPI jobs.
The current `Spec.splice` model is very limited by the inability to splice specs that
contain multiple nodes with the same name. This is an artifact of the original
algorithm design predating the separate concretization of build dependencies,
which was the first feature to allow multiple specs in a DAG to share a name.
This PR provides a complete reimplementation of `Spec.splice` to avoid that
limitation. At the same time, the new algorithm ensures that build dependencies
for spliced specs are not changed, since the splice by definition cannot change
the build-time information of the spec. This is handled by splitting the dependency
edges and link/run edges into separate dependencies as needed.
Signed-off-by: Gregory Becker <becker33@llnl.gov>
* CI: Add documentation for adding new stacks and runners
* More docs for runner registration
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
Co-authored-by: Bernhard Kaindl <contact@bernhard.kaindl.dev>
This PR shorten the string representation for concrete specs,
in order to make it more legible.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
macOS Sequoia's linker will complain if RPATHs on the CLI are specified more than once.
To avoid errors due to this, make `cc` only append unique RPATHs to the final args list.
This required a few improvements to the logic in `cc`:
1. List functions in `cc` didn't have any way to append unique elements to a list. Add a
`contains()` shell function that works like our other list functions. Use it to implement
an optional `"unique"` argument to `append()` and an `extend_unique()`. Use that to add
RPATHs to the `args_list`.
2. In the pure `ld` case, we weren't actually parsing `RPATH` arguments separately as we
do for `ccld`. Fix this by adding *another* nested case statement for raw `RPATH`
parsing. There are now 3 places where we deal with `-rpath` and friends, but I don't
see a great way to unify them, as `-Wl,`, `-Xlinker`, and raw `-rpath` arguments are
all ever so slightly different.
3. Fix ordering of assertions to make `pytest` diffs more intelligible. The meaning of
`+` and `-` in diffs changed in `pytest` 6.0 and the "preferred" order for assertions
became `assert actual == expected` instead of the other way around.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`cc` divides most paths up into system paths, spack managed paths, and other paths.
This gets really repetitive and makes the code hard to read. Simplify the script
by adding some functions to do most of the redundant work for us.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* axom/stand-alone tests: build and run in test stage directory
* Removed unused glob
* axom/stand-alone tests: add example_stage_dir variable for clarity
SimpleFilesystemView was producing an error due to looking for a
<prefix>/lib/.spack folder. Also, view_destination had no effect and
wasn't called. Changed this by instead patching in the correct
installation prefix for dictionaries.
Since aspell is using the resolved path of the executable prefix, the
runtime environment variable ASPELL_CONF is set to correct the prefix
when in a view. With this change aspell can now find installed
dictionaries. Verified with:
aspell dump config
aspell dump dicts
* shorten version number validations per reviewer feedback
* rename set_lib_path per reviewer feedback
* Add E4S tag
* Set CHPL_CUDA_PATH to ensure Chapel installer finds the right package
* Update ROCm dependency for Chapel 2.2
* Fix llvm-amdgpu and CHPL_TARGET_* for llvm=bundled
* Ensure CHPL_TARGET_COMPILER is set to "llvm" when required (llvm=spack
or +cuda or +rocm).
* Ensure CHPL_TARGET_{CC,CXX} are only set when using llvm=spack or llvm=none
* Use hip.prefix to set CHPL_ROCM_PATH
Since we might not directly depend on llvm-amdgpu, thus it might
not appear in our spec
* limit m4 dependency to +gmp
* limit names of env vars created from variants
* Ensure that +cuda and +rocm variants are Sticky
The concretizer should never be permitted to select GPU support, because
it's only meaningful and functional when the appropriate hardware is actually
available, and the concretizer cannot reliably determine that.
Also: Chapel's GPU support includes alot of complicated dependencies
and constraints, so leaving that choice free to the concretizer leads to alot
of extraneous and confusing messages when failing to concretize a
non-GPU-enabled spec.
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
Add pre-built sbcl for x86 and arm for various glibc versions, making
way for an actual sblc built from source.
Also switch to use set_env in a context manager over setting the
environment variable for the build environment. I hit an issue with the
build system due to this in the sbcl package, pre-empting the same issue
here.
* dla-future: Add DLAF_ prefix to LAPACK_LIBRARY CMake variable in newer versions
* dla-future: Use spec.satisfies to check version constraint for LAPACK_LIBRARY variable prefix
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
---------
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
* py-sphinx-tabs: new version 3.4.5
* py-sphinx-design: new versions 0.5.0, 0.6.0, and 0.6.1
* py-requests: new version 2.32.3
* py-dnspython: new version 2.6.1
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* py-hatch-vcs: new version 0.4.0
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Python 3.12 removed the `distutils` module, which is being required
by the build process of LLVM <= 14: Conflict with it for +python.
Fix build to not pick host tools like an incompatible Python from host
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
gcc on ubuntu has fix-cortex-a53-843419 set by default - this causes linking
issues (symbol relocation errors) for tf, even when compiling for different
cpus.
If `add_padding()` is allowed to return a path with a trailing path
separator, it will get collapsed elsewhere in Spack. This can lead to
buildcache entries that have RPATHS that are too short to be replaced by
other users whose install root happens to be padded to the correct
length. Detect this and replace the trailing path separator with a
concrete path character.
Signed-off-by: Samuel E. Browne <sebrown@sandia.gov>
Also: set the build and install directories to the source directory
because the build system unfortunately expects the `src_ext` directory
to be under the current working directory when building the bundled
third-party libraries, even when the configure script is run from
another directory.
@scemama pointed out that 'make' just calls 'dune' which is already
parallel, so make itself should not have more than one job.
opam@:2.1 need 'make lib-ext' for cmdliner, above it's obsolete.
Co-authored-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
The detection logic for the prefix used in py-bind11 if broken for spack
resulting in an empty prefix. However, the package provides an escape
hatch in the form of `prefix_for_pc_file`. Use this escape hatch to
provide the correct path; spack will always know better than pybind11's
CMake.
Co-authored-by: Robert Underwood <runderwood@anl.gov>
We've seen `getfqdn()` cause slowdowns on macOS in CI when added elsewhere. It's also
called by database.py every time we write the DB file.
- [x] replace the call with a memoized version so that it is only called once per process.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This PR introduces a new heuristic for the solver, which behaves better when
compilers are treated as nodes. Apparently, it performs better also on `develop`,
where compilers are still node attributes.
The new heuristic:
- Sets an initial priority for guessing a few attributes. The order is "nodes" (300),
"dependencies" (150), "virtual dependencies" (60), "version" and "variants" (30), and
"targets" and "compilers" (1). This initial priority decays over time during the solve, and
falls back to the defaults.
- By default, it considers most guessed facts as "false". For instance, by default a node
doesn't exist in the optimal answer set, or a version is not picked as a node version etc.
- There are certain conditions that override the default heuristic using the _priority_ of
a rule, which previously we didn't use. For instance, by default we guess that a
`attr("variant", Node, Variant, Value)` is false, but if we know that the node is already
in the answer set, and the value is the default one, then we guess it is true.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This includes a test_linux699 variant which "activates" a version
that pulls from a repository other than the official repository.
This version is required to work with Linux kernel version
6.9.9 or later. Future official `msr-safe` versions are expected
to support later Linux kernel versions.
* opendatadetector: Add an env variable pointing to the share directory
* Rename the new variable to OPENDATADETECTOR_DATA and use join_path
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
The `spack.target.Target` class is a weird entity, that is just needed to:
1. Sort microarchitectures in lists deterministically
2. Being able to use microarchitectures in hashed containers
This PR removes it, and uses `archspec.cpu.Microarchitecture` directly. To sort lists, we use a proper `key=` when needed. Being able to use `Microarchitecture` objects in sets is achieved by updating the external `archspec`.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Introduce the bufr_query library from NOAA-EMC (#461)
This PR adds in a new package.py script for the new bufr_query library from NOAA-EMC. This is being used by JEDI and other applications.
* Add explicit build dependency spec to the pybind11 depends_on spec
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Convert patch file to the URL form which pulls the changes from github.
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Added new version (0.0.3) and removed obsolete site-packages.patch file
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
While the existing getting started guide does in fact reference the
powershell support, it's a footnote and easily missed. This PR adds
explicit, upfront mentions of the powershell support. Additionally
this PR adds notes about some of the issues with certain components
of the spec syntax when using CMD.
If the spec is external, it has extra attributes. If not, we know
which names are used. In both cases we don't need to search again
for executables.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Removes `spack.package_base.PackageBase.do_{install,deprecate}` in favor of
`spack.installer.PackageInstaller.install` and `spack.installer.deprecate` resp.
That drops a dependency of `spack.package_base` on `spack.installer`, which is
necessary to get rid of circular dependencies in Spack.
Also change the signature of `PackageInstaller.__init__` from taking a dict as
positional argument, to an explicit list of keyword arguments.
Continuing the work started in #40326, his changes the structure
of Variant metadata on Packages from a single variant definition
per name with a list of `when` specs:
```
name: (Variant, [when_spec, ...])
```
to a Variant definition per `when_spec` per name:
```
when_spec: { name: Variant }
```
With this change, everything on a package *except* versions is
keyed by `when` spec. This:
1. makes things consistent, in that conditional things are (nearly)
all modeled in the same way; and
2. fixes an issue where we would lose information about multiple
variant definitions in a package (see #38302). We can now have,
e.g., different defaults for the same variant in different
versions of a package.
Some notes:
1. This required some pretty deep changes to the solver. Previously,
the solver's job was to select value(s) for a single variant definition
per name per package. Now, the solver needs to:
a. Determine which variant definition should be used for a given node,
which can depend on the node's version, compiler, target, other variants, etc.
b. Select valid value(s) for variants for each node based on the selected
variant definition.
When multiple variant definitions are enabled via their `when=` clause, we will
always prefer the *last* matching definition, by declaration order in packages. This
is implemented by adding a `precedence` to each variant at definition time, and we
ensure they are added to the solver in order of precedence.
This has the effect that variant definitions from derived classes are preferred over
definitions from superclasses, and the last definition within the same class sticks.
This matches python semantics. Some examples:
```python
class ROCmPackage(PackageBase):
variant("amdgpu_target", ..., when="+rocm")
class Hipblas(ROCmPackage):
variant("amdgpu_target", ...)
```
The global variant in `hipblas` will always supersede the `when="+rocm"` variant in
`ROCmPackage`. If `hipblas`'s variant was also conditional on `+rocm` (as it probably
should be), we would again filter out the definition from `ROCmPackage` because it
could never be activated. If you instead have:
```python
class ROCmPackage(PackageBase):
variant("amdgpu_target", ..., when="+rocm")
class Hipblas(ROCmPackage):
variant("amdgpu_target", ..., when="+rocm+foo")
```
The variant on `hipblas` will win for `+rocm+foo` but the one on `ROCmPackage` will
win with `rocm~foo`.
So, *if* we can statically determine if a variant is overridden, we filter it out.
This isn't strictly necessary, as the solver can handle many definitions fine, but
this reduces the complexity of the problem instance presented to `clingo`, and
simplifies output in `spack info` for derived packages. e.g., `spack info hipblas`
now shows only one definition of `amdgpu_target` where before it showed two, one of
which would never be used.
2. Nearly all access to the `variants` dictionary on packages has been refactored to
use the following class methods on `PackageBase`:
* `variant_names(cls) -> List[str]`: get all variant names for a package
* `has_variant(cls, name) -> bool`: whether a package has a variant with a given name
* `variant_definitions(cls, name: str) -> List[Tuple[Spec, Variant]]`: all definitions
of variant `name` that are possible, along with their `when` specs.
* `variant_items() -> `: iterate over `pkg.variants.items()`, with impossible variants
filtered out.
Consolidating to these methods seems to simplify the code a lot.
3. The solver does a lot more validation on variant values at setup time now. In
particular, it checks whether a variant value on a spec is valid given the other
constraints on that spec. This allowed us to remove the crufty logic in
`update_variant_validate`, which was needed because we previously didn't *know* after
a solve which variant definition had been used. Now, variant values from solves are
constructed strictly based on which variant definition was selected -- no more
heuristics.
4. The same prevalidation can now be done in package audits, and you can run:
```
spack audit packages --strict-variants
```
This turns up around 18 different places where a variant specification isn't valid
given the conditions on variant definitions in packages. I haven't fixed those here
but will open a separate PR to iterate on them. I plan to make strict checking the
defaults once all existing package issues are resolved. It's not clear to me that
strict checking should be the default for the prevalidation done at solve time.
There are a few other changes here that might be of interest:
1. The `generator` variant in `CMakePackage` is now only defined when `build_system=cmake`.
2. `spack info` has been updated to support the new metadata layout.
3. split out variant propagation into its own `.lp` file in the `solver` code.
4. Add better typing and clean up code for variant types in `variant.py`.
5. Add tests for new variant behavior.
Historically, every PR, push, etc. to Spack generates a bunch of jobs, each of which
uploads its coverage report to codecov independently. This means that we get annoying
partial coverage numbers when only a few of the jobs have finished, and frequently
codecov is bad at understanding when to merge reports for a given PR. The numbers of the
site can be weird as a result.
This restructures our coverage handling so that we do all the merging ourselves and
upload exactly one report per GitHub actions workflow. In practice, that means that
every push to every PR will get exactly one coverage report and exactly one coverage
number reported. I think this will at least partially restore peoples' faith in what
codecov is telling them, and it might even make codecov handle Spack a bit better, since
this reduces the report burden by ~7x.
- [x] test and audit jobs now upload artifacts for coverage
- [x] add a new job that downloads artifacts and merges coverage reports together
- [x] set `paths` section of `pyproject.toml` so that cross-platform clone locations are merged
- [x] upload to codecov once, at the end of the workflow
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* kokkos, kokkos-kernels, kokkos-nvcc-wrapper: add v4.4.01
* trilinos: update @[master,develop] dependency on kokkos
==> Error: InstallError: For Trilinos@[master,develop], ^kokkos version in spec must match version in Trilinos source code. Specify ^kokkos@4.4.01 for trilinos@[master,develop] instead of ^kokkos@4.4.00.
* petsc: configure requires rocm-core/rocm_version.h to detect ROCM_VERSION_MAJOR.ROCM_VERSION_MINOR.ROCM_VERSION_PATCH
* mfem: add dependency on rocprim (as needed via petsc dependency)
In file included from linalg/petsc.cpp:19:
In file included from linalg/linalg.hpp:65:
In file included from linalg/petsc.hpp:48:
In file included from /scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/petsc-3.22.0-7dsxwizo24ycnqvwnsscupuh4i7yusrh/include/petscsystypes.h:530:
In file included from /scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/rocthrust-6.1.2-ux5nmi4utw27oaqmz3sfjmhb6hyt5zed/include/thrust/complex.h:30:
/scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/rocthrust-6.1.2-ux5nmi4utw27oaqmz3sfjmhb6hyt5zed/include/thrust/detail/type_traits.h:29:10: fatal error: 'rocprim/detail/match_result_type.hpp' file not found
29 | #include <rocprim/detail/match_result_type.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Update seacas package.py
Adding libcatalyst variant to seacas package
When seacas is installed with "seacas +libcatalyst"
then a dependency on the spack package "libcatalyst"
(which is catalyst api 2 from kitware) is added, and
the appropriage cmake variable for the catalyst TPL
is set. The mpi variant option in catalyst (i.e. build
with mpi or build without mpi) is passed on to
libcatalyst. The default of the libcatalyst variant
is false/off, so if seacas is installed without the
"+libcatalyst" in the spec it will behave exactly as
it did before the introduction of this variant.
* shortened line 202 to comply with < 100 characters per line style requirement
* py-httpx: New version
* [py-httpx] fix when for dependencies
* [py-httpx] organized dependencies
* [py-httpx] added version 0.27.2
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
* Automated deployment to update package flux-sched 2024-09-05
* flux-sched: add back check for run environment
* flux-sched: add conflict for gcc and clang above 0.37.0
---------
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
autotools packages with a configure script should generate the libtool
executable, there's no point in `depends_on("libtool", type="build")`.
the libtool executable in `<libtool prefix>/bin/libtool` is configured
for the wrong toolchain (libtools %compiler instead of the package's
%compiler).
Some package link to `libltdl.so`, which is fine, but had a wrong
dependency type.
See https://github.com/spack/spack/pull/46314#discussion_r1752940332.
This further simplifies `cxxstd` variant handling in `acts` by removing superfluous
version constraints from dependencies for `geant4` and `root`.
The version constraints in the loop are redundant with the conditional variant
values here:
```python
_cxxstd_values = (
conditional("14", when="@:0.8.1"),
conditional("17", when="@:35"),
conditional("20", when="@24:"),
)
_cxxstd_common = {
"values": _cxxstd_values,
"multi": False,
"description": "Use the specified C++ standard when building.",
}
variant("cxxstd", default="17", when="@:35", **_cxxstd_common)
variant("cxxstd", default="20", when="@36:", **_cxxstd_common)
```
So we can simplify the dependencies in the loop to:
```python
for _cxxstd in _cxxstd_values:
for _v in _cxxstd:
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +geant4")
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +fatras_geant4")
depends_on(f"root cxxstd={_v.value}", when=f"cxxstd={_v.value} +tgeo")
```
And avoid the potential for impossible variant expressions.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Openmpi provider statements were changed in #46102. The package change
was fine in and of itself, but apparently one of our tests depends on
the precise constraints used in those statements. I updated the test
to remove the checks for constraints that were removed.
In #44425, we add stricter variant audits that catch expressions that can never match.
This fixes 13 packages that had this type of issue.
Most packages had lingering spec expressions from before conditional variants with
`when=` statements were added. For example:
* Given `conflicts("+a", when="~b")`, if the package has since added
`variant("a", when="+b")`, the conflict is no longer needed, because
`a` and `b` will never exist together.
* Similarly, two packages that depended on `py-torch` depended on
`py-torch~cuda~cudnn`, which can't match because the `cudnn` variant
doesn't exist when `cuda` is disabled. Note that neither `+foo` or `~foo`
match (intentionally) if the `foo` variant doesn't exist.
* Some packages referred to impossible version/variant combinations, e.g.,
`ceed@1.0.0+mfem~petsc` when the `petsc` variant only exist at version `2`
or higher.
Some of these correct real issues (e.g. the `py-torch` dependencies would have never
worked). Others simply declutter old code in packages by making all constraints
consistent with version and variant updates.
The only one of these that I think is not all that useful is the one for `acts`,
where looping over `cxxstd` versions and package versions ends up adding some
constraints that are impossible. The additional dependencies could never have
happened and the code is more complicated with the needed extra constriant.
I think *probably* the best thing to do in `acts` is to just not to use a loop
and to write out the constraints explicitly, but maybe the code is easier to
maintain as written.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Update var/spack/repos/builtin/packages/fms/package.py: apply patch for fms@2023.03 to fix compiler options bug in cmake config, add variant shared and corresponding patch for fms@2024.02
* Fix fms package audit: use c9bba516ba.patch?full_index=1 instead of c9bba516ba.patch?full_index=1
* Update checksum of patch for fms@2023.03
* CUDA: support Grace Hopper 9.0a compute capability
* Fix other packages
* Add type annotations
* Support ancient Python versions
* isort
* spec -> self.spec
Co-authored-by: Andrew W Elble <aweits@rit.edu>
* [@spackbot] updating style on behalf of adamjstewart
---------
Co-authored-by: Andrew W Elble <aweits@rit.edu>
Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
fixes#46295
A proper solution would be a tag directive that accumulates tags
with the ones defined in base classes.
For the time being, rewrite them explicitly.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Boost:Adjust bootstrapping/b2 options as needed for Windows (the
bootstrapping phase sufficiently differs between Windows/Unix
that it is handled entirely within its own branch).
* Boost: Paths in user-config.jam should be POSIX, including on Windows
* Python: `.libs` for the Python package should return link libraries
on Windows. The libraries are also stored in a different directory.
The option config:install_missing_compilers is currently buggy,
and has been for a while. Remove it, since it won't be needed
when compilers are treated as dependencies.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fast-float: new package
* fast-float: add test dependency
* fast-float: fix doctest dependency type
* fast-float: convert deps to tuple
* fast-float: add v6.1.5 and v6.1.6
* fast-float: patch older versions to use find_package(doctest)
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
* Changed software pull location
The original package pulled a tagged release version from GitHub. That tagged version was created in 2022 and has not been updated since. It no longer runs because newer versions of numpy have removed deprecation warnings for several of their calls. The main branch for this repository has addressed these numpy issues as well as some other important fixes but no new release has been generated. Because of this and the apparent minimal development that now appears to be going on, it is probably best to always pull from the main branch
* [@spackbot] updating style on behalf of aweaver1fandm
* py-your: Changed software pull location
1. Restored original URL and version (0.6.7) as requested
2. Updated py-numpy dependency versions to be constrained based on the version of your. Because of numpy deprecations related to your version 0.6.7 need to ensure that the numpy version used is not 1.24 or greater because the depracations were removed starting with that version
* gptune: new test API
* gptune: cleanup; finish API changes; separate unrelated test parts
* gptune: standalone test cleanup with timeout constraints
* gptune: ensure stand-alone test bash failures terminate; enable in CI
* gptune: add directory to terminate_bash_failures
* gptune/stand-alone tests: use satisifes for checking variants
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add numactl 2.0.16-2.0.18
* Create link-with-latomic-if-needed-v2.0.16.patch
Add a link to libatomic, if needed, for numactl v2.0.16.
* Add some missing patches to v2.0.16
* Create numactl-2.0.18-syscall-NR-ppc64.patch
In short, we need numactl to set __NR_set_mempolicy_home_node on ppc64, if it's not already defined.
* Apply a necessary patch for v2.0.18 on PPC64
* Add libatomic patch for v2.0.16
`spack reindex` relies on projections from configuration to locate
installed specs and prefixes. This is problematic because config can
change over time, and we have reasons to do so when turning compilers
into depedencies (removing `{compiler.name}-{compiler.version}` from
projections)
This commit makes reindex recursively search for .spack/ metadirs.
* py-httpcore: Added new version
* [py-httpcore]
- added version 0.18.0
- restructured dependencies as everything has a when and
type/when ordering was all over the place
* [py-httpcore] ordered dependencies in the order listed in v1.0.5 pyproject.toml
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
When a package is running `setup_dependent_package` on a parent, ensure
that module variables like `spack_cc` are available. This was often
true prior to this commit, but externals were an exception.
---------
Co-authored-by: John Parent <john.parent@kitware.com>
* New package: py-monai
* Fixed linked issues with py-monai
* [py-monai] removed extra line
* [py-monai]
- New version 1.3.2
- ran black
- update copyright
* [py-monai] added license
* [py-monai]
- moved build only dependencies
- specified newer python requirements for newer versions
---------
Co-authored-by: vehrc <vehrc@rit.edu>
* Replace if ... in spec with spec.satisfies in f* and g* packages
* gromacs: ^amdfftw -> ^[virtuals=fftw-api] amdfftw
* flamemaster: add virtuals lapack for the amdlibflame dependency
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Allow flags from different sources (compilers, `require:`, command-line
specs, and `depends_on`) to be merged together, and enforce a consistent
order among them.
The order is based on the sources, e.g. flags on specs from the command
line always come last. Some flag order consistency issues are fixed:
1. Flags from `compilers.yaml` and the command line were always intra- and
inter-source order consistent.
2. Flags from dependents and packages.yaml (introduced via `require:`)
were not: for `-a -b` from one source and `-c` from another, the final
result might rearrange `-a -b`, and would also be inconsistent in terms
of whether `-c` came before or after.
(1) is/was handled by going back to the original source, i.e., flags are
retrieved directly from the command line spec rather than the solver.
(2) is addressed by:
* Keeping track of grouped flags in the solver
* Keeping track of flag sources in the solver on a per-flag basis
The latter info is used in this PR to enforce DAG ordering on flags
applied from multiple dependents to the same package, e.g., for this
graph:
```
a
/|\
b | c
\|/
d
```
If `a`, `b`, and `c` impose flags on `d`, the combined flags on `d` will
contain the flags of `a`, `b`, and `c` -- in that order.
Conflicting flags are allowed (e.g. -O2 and -O3). `Spec.satisifes()` has
been updated such that X satisfies Y as long as X has *at least* all of
the flags that Y has. This is also true in the solver constraints.
`.satisfies` does not account for how order can change behavior (so
`-O2 -O3` can satisfy `-O3 -O2`); it is expected that this can be
addressed later (e.g. by prohibiting flag conflicts).
`Spec.constrain` and `.intersects` have been updated to be consistent
with this new definition of `.satisfies`.
Spack can now bootstrap two new dependencies on Windows: GnuPG, and file.
These dependencies are modeled as a separate package, and they install a cross-compiled binary.
Details on how they binaries are built are in https://github.com/spack/windows-bootstrap-resources
This PR adds py-pybind11 versions 2.13.0, 2.13.1, 2.13.2, 2.13.3, and
2.13.4. It also adds a new conflict between gcc 14 and pybind versions
up to and including 2.13.1.
* Allow deprecating more than one property in config
This internal change allows the customization of errors
and warnings to be printed when deprecating a property.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fix
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Use a list comprehension for "issues"
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
fixes#40791
Currently stacks behave differently if used in unify:false
environments, which leads to inconsistencies during concretization.
For instance, we might have two abstract user specs that do not
intersect with each other map to the same concrete spec in the
environment. This is clearly wrong.
This PR removes the best effort expansion, so that user specs
are always applied strictly.
* gaudi: Specify boost components and add +fiber for v39
* gaudi: Limit fmt version to allow building master branch
* Make boost dependencies a bit more readable
* Remove patches for no longer existing versions
* Replace if ... in spec with spec.satisfies in d* and e* packages
* Use virtuals for different mpi implementations in esmf
* esmf: ^[virtuals=mpi] mpt
* extrae: ^[virtuals=mpi] intel-oneapi-mpi
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
This change aligns the build condition for parmetis with the
depends_on condition.
The current build condition of parmetis looks for "+parmetis" in
the spec which is not added by the depends_on as that adds
"^parmetis" instead.
* Adding addtional check for omptarget library for amdgpu in nvidia environment
* Avoiding registration of duplicate when built on cuda
* Adding hsa library path in LD_LIBRARY_PATH
* Correction in hsa prefix library path in LD_LIBRARY_PATH
==> Error: InstallError: For Trilinos@[master,develop], ^kokkos version in spec must match version in Trilinos source code. Specify ^kokkos@4.4.00 for trilinos@[master,develop] instead of ^kokkos@4.3.01.
* root: Add dependency on libglx
We have been trying to build the Acts package on MacOS, and in this
process we have been running into problems with the ROOT spec on that
operating system; the primary issue we are encountering is that the
compiler is unable to find the `GL/glx.h` header, which is part of glx.
It seems, therefore, that ROOT depends on libglx, but this is not
currently encoded in the spec. This commit ensures that ROOT depends on
the virtual libglx package when both the OpenCL and X11 variants are
enabled.
* Enable builtin glew on MacOS
* Allow `root+opengl+aqua~x` on macOS
dd4hep versions up to and including 1.27 had a conflict with root
versions starting from 6.31.1, as shown in
https://github.com/AIDASoft/DD4hep/issues/1210. This PR explicitly adds
that conflict to the spec.
* whizard: add a patch when using hepmc3 3.3.0 or newer
* whizard: comment with patch origin
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* podio: apply patch for gcc 14 builds
Podio versions after 0.17.0 but before 1.0.0 are broken when using gcc
14 because of a missing include, which is addressed in the podio pull
request at https://github.com/AIDASoft/podio/pull/600. This commit
patches pre-1.0.0 versions of Podio so they can be compiled with gcc 14,
which is important for building Acts.
* Style
* Style 2
* Fixes
* Add comment:
* Add sha256
This should help not selecting, by default, some niche implementation that are supposed to be externals.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This PR simplifies the code doing external spec detection by removing
the `DetectedPackage` class. Now, functions accepting or returning lists
of `DetectedPackage`, will accept or return list of specs.
Performance doesn't seem to change if we use `Spec.__reduce__` instead
of `DetectionPackage.__reduce__`.
- Honor compiler extra_rpaths (this build bypasses spack wrappers,
so the RPATHs are added explicitly as CMake options)
- Use root_cmakelists_dir instead of adding it to the command line
- Add BLT as a dependency, allowing different versions outside of what
is in the tarball
- Put a copy of host-config in the stage directory: this allows
examination of the host-config when a build fails (before, the
host-config was just stored in the install directory, which is
deleted by default on a failed build)
* Add conflict between NVSHMEM and cuFFTMp for GROMACS package
These don't work in the same build configuration.
* [@spackbot] updating style on behalf of mabraham
* Update package.py
Also constrain NVSHMEM appropriately
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
* Update package.py
* [@spackbot] updating style on behalf of mabraham
* Update package.py
* Update package.py
* Update package.py
* [@spackbot] updating style on behalf of mabraham
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
Change the signature of Spec.from_detection to set the
external prefix, and the external modules, if they are
present.
Delete "spack.package_prefs.spec_externals" since it
is unused.
This context manager was used to prevent detecting a platform
as "cray". Since now Cray machines are detected as linux, we can
remove the context manager.
This commit adds version 0.73.0 of the detray package. As this version
drops support for pre-C++20 standards, I had to update the `cxxstd`
variant logic.
Right now if you run `spack ci generate` you get a warning about CDash credentials even
if there's no CDash configuration specified. We should only warn if there was actually a
CDash config.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Source mirrors store entries by digest and add a human readable alias of the
form 'name-version'. If no digest is available, the alias is used as the primary
storage location.
Spack erroneously fetches by alias when the digest path does not exist. This is
problematic if `version(..., sha256=...)` changes in package.py, and the mirror
is populated with the old shasum. That would result in an error when a digest
is available, but in case of git versions with a modified commit sha, the wrong
sources would be fetched without error. With this PR, only the digest path is
used, not the alias, in case a digest is available. This is also a small performance
optimization, as the number of request is halved for mirrors that don't contain
the sources.
Further, for git sources the tag was used as a digest, but this is a moving
target. Only commit sha is used now.
Also whenever the alias already existed, Spack used to keep it in place when
updating the mirror cache, which means that aliases would always point to
outdated mirror entries whenever digests are modified. With this PR the alias
is moved in place.
Lastly, fix a recent regression where `Stage.disable_mirrors` disabled mirrors
but not the local download cache, which was the intention.
* VTK-m: Point to github mirror for source tarball
The gitlab.kitware.com source location seems to have intermittent
network issues. Switching the to mirror hosted at Github may alleviate
some of the timeouts.
* Update sha256 for GitHub tarballs
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
* davix: add versions 0.8.2-0.8.7 and dependencies
This commit adds new versions 0.8.2-0.8.7 of the davix package, and it
also improves the handling of embedded packages. Davix will try to build
libcurl from its own embedded version of that code, which doesn't mesh
well with Spack's design philosophy, so I've changed the CMake
configuration to disallow the builtin libcurl and use a Spack dependency
instead. Up to version 0.8.7, RapidJSON was also builtin, but version
0.8.7 allows users to specify that they want to use a pre-installed
version of RapidJSON, so this commit also adds that as a dependency for
versions 0.8.7:.
* Fix old versions
* acts: add v36.0.0, v36.1.0 and fixes
This commit makes several changes to the Acts repository, namely:
1. It adds versions 36.0.0 and 36.1.0.
2. It adds the traccc plugin and related dependencies.
3. It updates the version requirements of some dependencies.
4. It adds the Geant4 module of GeoModel.
5. It updates the C++ standard requirement.
6. It adds a new variant determining the scalar type to use.
This commit supercedes #45851. Thanks @jmcarcell for the version
updates; I have added you as co-author.
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Updates
* alphabetic cmake args
---------
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Add missing MultiMethodMeta metaclass in builders
and remove the Python 2 fallback option in favor of hard errors to catch
similar issues going forward.
The fallback option can cause about 10K stat calls due to use of
`realpath` in the inspect module, depending on how deep Spack itself is
nested in the file system, which is ... undesirable.
* code shuffling to avoid circular import
* more reshuffling
* move reserved variant names into variants module
Currently, the llvm-amdgpu package doesn't compile on MacOS, but it is
also not marked as a conflict. This causes problems because it seems
that Spack is very happy to pull in llvm-amdgpu as the default package
to satisfy any virtual libllvm dependency, which can cause dependent
specs to fail to install on MacOS. This commit marks a conflict between
this llvm package and the Darwin platform.
* root: set +webgui~http conflict from @6.28.12:
Currently, the ROOT spec correctly identifies a conflict between +webgui
and ~http, but this conflict is marked as affecting @6.29.00: only. As a
matter of fact, ROOT 6.28.12 is also affected. This commit, therefore,
updates the when clause on the conflict to @6.28.12:.
* Remove when clause entirely
* oops
* Adds an MSBuild system + Builder to the icu4c package
* Adds custom install method as MSBuild system does not vendor an
install target
* The cxxstd variant is not supported on Windows (there are no config
options you use to tell the build system what cxx standard to
build against), so the variant definition was updated to occur
everywhere except Windows
Also, this commit defines the c/cxx..._flag properties of the MSVC
compiler (although they are not used by `icu4c` and not strictly
necessary to bundle with this PR).
This commit adds a dependency on GeoModel 4.6.0 when the GeoModel plugin
is enabled. Note that the dependency is upgraded to 6.3.0 in Acts
36.1.0, but that will need to be covered in #45851.
* geomodel: add versions 5.1.0-6.4.0
This PR adds new versions 5.1.0 through 6.4.0 of the geomodel package.
It also updates the CMake configuration to use the `define_from_variant`
mechanism and it adds me as a maintainer.
* Undo cmake change
The avx2 version can be downloaded from the ORCA [forum](https://orcaforum.kofo.mpg.de/app.php/dlext/?view=detail&df_id=214#).
The version is named `avx2-6.0.0` (as opposed to the more
natural-looking `6.0.0-avx2`) to avoid the avx2 version shadowing the
non-avx2 one. Definitely open for better suggestion.
* hsakmt-roct: new test API
* hsakmt-roct: minor change to check_install script variable name
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parallel-netcdf: new test API
* parallel-netcdf: fix test args and tweak docstring and variables
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* WarpX: Python on pyAMReX
Long overdue update for WarpX: in 2024, we updated our Python
bindings to rely on the new pyAMReX package. This deprecates the old
`py-warpx` package and adds a new dependency and variant to WarpX.
Also deprecates old versions that we will not continue to support.
* Update Cloud/E4S Pipelines for WarpX
`py-warpx` is replaced by `warpx +python`
oneAPI does not support IPO/LTO: diable for `py-amrex` even though
pybind11 strongly encourages it.
* Add `+python` variant
* `esmf` package installs Python bindings when `+python` is set
Note: this does not inherit `PythonPackage`, which force an either/or
choice between the Makefile and Pip builder: it instantiates a
`PythonPipBuilder` as needed (when `+python` is set).
Paths over 260 characters in length are not handled by `shutil.rmtree`
unless they use the extended-length path syntax (using a prefix of
"\\?\").
This fixes an issue where stage cleaning fails when paths in a stage
exceed the normal 260-character limit.
This indicates that other parts of the codebase should be examined/
refactored to handle long paths.
* harfbuzz: enable freetype in MesonBuilder to facilitate depends_on("freetype")
* spack style fix
* freetype is defined as a depends_on(), so set as enabled in MesonBuilder rather than an option/flag/variant
* add back depends_on($lang) lines for new test api
* keep original order
Previously `spack buildcache push --only=package` errored in the OCI
case, but it's been requested that OCI can be used as pure storage w/o
the need for runnable container images.
This commit makes it so that
1. manifests refer only to runtime dependencies that were selected to be
pushed
2. failure to upload a blob among the selected specs does not prevent a
manifest/tag to be created for dependents: they just don't refer to
the missing blob as a layer/dependency
This fixes the following issues:
1. dependents of non-redistributable specs can now be pushed to oci
build caches without error
2. failure to upload one tarball does not cause cascading failures for
dependents whose tarballs do upload succesfully -- so it's better
best-effort behavior
3. for some people uploading with deps caused a massive amount of
fetches of their manifests (which certain registries count as a
download of an image, even though their layers are not fetched) --
being able to specify --only=package reduces the number of fetches.
Stage objects create mirrors ad-hoc from current config.
- There is no way to prevent mirrors from being used
- There is no way to restrict mirrors to source/binary, which is of
course context dependent.
- Stage is also used in build caches, where iterating over mirrors is
already implemented differently, and wouldn't work anyways cause it's
source only, and in particular it makes no sense for OCI build caches.
This commit:
1. Injects the sensible mirrors into the stage object from contexts
where it is relevant
2. Separates mirrors from cache, so that w/o mirrors download cache can
still be used
The test_foreground_background unit test has been marked
xfail for a while, meaning:
- Nobody looks at the results of the test
- It still runs every time
That test happens to hang frequently on some Apple M1 I have access to,
so here I mark it as skip.
Also went through other xfailing and skipped tests, and applied minor changes.
* Use absolute paths instead of https:// fake mirrors (this speed-up tests by avoiding requests)
* Add a fixture to gather in a single place code that is copy/pasted in a lot of tests
* General clean-up of tests and repeated code
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* hypre: get rid of depreciated run_test method
* hypre: use mkdirp directly
* hypre: use install() for ij for addition of permissions fix
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parsec: old to new test API
* parsec: restore stand-alone test subparts; preliminary test build fixes
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* n2p2: convert from old to new test API
* n2p2: Enhance stand-alone testing checks to reduce unnecessary processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* stand-alone test API update: self.cache_extra_test_sources(...) -> cache_extra_test_sources(self, ...)
* superlu: switch to new cache_extra_test_sources API
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add options for sparse checkout in GitFetcher
Newer versions of git have a beta feature called sparse checkout
that allow users to check out a portion of a large repo.
This feature will be ideal for monolithic repo projects that want to
model their infrastructure via spack. This PR implements an addition
to the GitFetcher that allows users to add a `git_sparse_paths`
attribute to package classes or versions which will then use sparse
checkout on those directories/files for the package.
* Style
* Split git clone into multiple functions
* Add sparse-checkout impl
* Internalize src clone functions
* Docs
* Adding sparse clone test
* Add test for partial clone
* [@spackbot] updating style on behalf of psakievich
* Small fixes
* Restore default branch status
* Fix attributes for package
* Update lib/spack/docs/packaging_guide.rst
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Extend unit test to multiple git versions
* style
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Feature update py-fortranformat
Add more recent versions of py-fortranformat. The currently included release (0.2.5) is from 2014. I've added the latest point release of each of the major versions from the last 4 years.
* update homepage
* magma: old to new test API
* magma: simplify stand-alone test method/part docstrings/purposes
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* initial commit to enable asan in remaining rocm packages
* remove os import
* add rocm-opencl rocm-dbgapi rocm-debug-agent
* add libclang path to LD_LIBRARY_PATH
* enable asan for rocfft
* add f-string and add +asan dependencies for hip and rocm-opencl
* add conflicts for centos7/8 and rhel 9
When sending Repo and RepoPath over to a child process,
we go through a marshaling procedure with pickle. The
default behavior for these classes is highly inefficient,
as it serializes a lot of specs that can just be
reconstructed on the other end of the pipe.
Here we write optimized procedures to __reduce__ both
classes.
* raja: new test API
* raja: tweak test method names and docstrings
* raja: restore running tests under proper directory
* raja: cleanup skiptest message and example call
* raja: Tweak expected outputs to match current
* raja: test_views -> test_stencil_offset_layout
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* mptensor: Changed skiptest, test name, and added docstring
* mptensor: make stand-alone test method name and docstring more specific
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add version 2.0.-2
* Newer version at the top
* [@spackbot] updating style on behalf of rahmans1
* py-htgettoken: reorder comments
* Use sha256sum instead of commit id for version 2.0-2 and above
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Extracted from #44419
This adds / modifies some external detection tests for compilers,
to reproduce cases that are currently tested in unit tests.
The unit tests will later be removed.
Extracted from #45638
When adding the "detectable" tag to a package class that has the
"tag" attribute inherited from a base class, we need to copy it to
avoid modifying the base class.
* kickoff attempt
* resource similar to fortran
* delete unused install_component_codeplay
* Adding conflict for versions <= 2022.2.1, moving install to package.py, adding sha256 for version 2024.2.1
* [@spackbot] updating style on behalf of kaanolgu
---------
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
"spack buildcache push" for partially installed environments pushes all it
can by default, and only dumps errors towards the end.
If --fail-fast is provided, error out before pushing anything if any
of the packages is uninstalled
oci build caches using parallel push now use futures to ensure pushing
goes in best-effort style.
* migraphx: Old to new test API
* migraphx: tweak name and docstring to be more descriptive
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* c: new test API
* gcc: provides('c')
* c: bugfix and simplification of the new stand-alone test method
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* fortran: new test API
* fortran: add provides to gcc package
* fortran: simplify stand-alone test processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* hypre-cmake: old to new test API
* hypre-cmake: update Makefile to use installed files
* hypre-cmake: make stand-alone test method name more specific
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
The old concretizer is still used to bootstrap clingo from source. If we switch to a DAG model
where compilers are treated as nodes, we need to either:
1. fix the old concretizer to support this (which is a lot of work and possibly research), or
2. bootstrap `clingo` without the old concretizer.
This PR takes the second approach and gets rid of the old concretizer code. To bootstrap
`clingo`, we store some concrete spec prototypes as JSON, select one according to the
coarse-grained system architecture, and tweak them according to the current host.
The old concretizer and related dead code are removed. In particular, this removes
`Spec.normalize()` and related methods, which were used in many unit-tests to set
up the test context. The tests have been updated not to use `normalize()`.
- [x] Bootstrap clingo concretization based on a JSON file
- [x] Bootstrap clingo *before* patchelf
- [x] Remove any use of the old concretizer, including:
* Remove only_clingo and only_original fixtures
* Remove _old_concretize and _new_concretize
* Remove _concretize_together_old
* Remove _concretize_together_new
* Remove any use of `SPACK_TEST_SOLVER`
* Simplify CI jobs
- [x] ensure bootstrapping `clingo` works on on Darwin and Windows
- [x] Raise an intelligible error when a compiler is missing
- [x] Ensure bootstrapping works on FreeBSD
- [x] remove normalize and related methods
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`setup-env.sh` is meant to be sourced, not executed directly.
By revoking execution permissions, users who accidentally execute
the script will receive an error instead of seeing no effect.
* Remove execution permission from `setup-env.sh` and friends
* Don't make output file executable in `spack commands --update-completion`
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* On Windows it is built with CMake, however CMake is built against
libuv, so libuv must depend on cmake+ownlibs to short circuit the
circular dependency
* libuv currently fetches the -dist source distribution of libuv for
certain versions because those versions contain a pre-generated
./configure script. However those distributions have all CMake
files removed, so they cannot be used to build on Windows.
Because the source distributions are different, this means the
checksums are different, and necessitates an additional version
declaration for each version we want to support with CMake.
* Update available versions
* fix style
* Reduce added versions to just a single new version as per review
* fix style
* Set dependency versions in line with pyproject and setup.py
* add new interp variant
* add wheel as required
* Add variant description
* change +interp default, actually add packaging this time
Fixes#43494
Add a set of environment variables SPACK_ALWAYS_CFLAGS (etc.) that
are always applied by the compiler wrapper.
Unlike SPACK_CFLAGS, for example, these will also be applied to
version checks (both SPACK_CFLAGS and SPACK_ALWAYS_CFLAGS will be
applied to the other invocation modes like ccld etc.).
Using this new functionality, the classic Intel and oneAPI compilers
are updated to pass compiler flags that disable warning messages
when newer versions are invoked via their older binary names
(these warnings were also generated for version checks, hence the
need for a new wrapper variable).
---------
Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
* namd:add compile options for ROCm
* Combine --rocm-prefix and its value in a single opts.extend call to ensure they remain ordered correctly and improve code robustness.
* Standardize the code and add ROCm conflicts
* add single_node_gpu conflict
---------
Co-authored-by: U-PALLAS\boudaoud <boudaoud@pc44.pallas.cines.fr>
When clang is installed I get `clang: Permission denied`.
Setting CLANG to the full path of clang fixes this.
Signed-off-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Fixes error:
```txt
3 errors found in build log:
61 ==> xmlto: Executing phase: 'build'
62 ==> [2023-10-26-09:48:35.425903] 'make' '-j16' 'V=1'
63 make all-am
64 make[1]: Entering directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
65 /bin/sh ./ylwrap xmlif/xmlif.l .c xmlif/xmlif.c -- /bin/sh /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing flex
66 /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing: line 81: flex: command not found
>> 67 WARNING: 'flex' is missing on your system.
68 You should only need it if you modified a '.l' file.
69 You may want to install the Fast Lexical Analyzer package:
70 <http://flex.sourceforge.net/>
>> 71 make[1]: *** [Makefile:757: xmlif/xmlif.c] Error 127
72 make[1]: Leaving directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
>> 73 make: *** [Makefile:584: all] Error 2
```
For example: spack install py-pyqt5 or py-pyqt6 would install pylupdate5 pyrcc5 and
pyuic5 into in python-venv's install prefix.
Fix https://github.com/spack/spack/issues/45359
* [py-configspace] fix dependency on cython
* py-cython not needed starting from 1.0.0
* added py-configspace 1.0.0 and 1.0.1
* py-configspace: fix style
* added py-configspace version 1.1.0
* added py-configspace version 1.1.1
* py-configspace: two more versions and new maintainer
* py-configspace: fixed typo
* py-configspace: added version 1.1.4
* Allow spec queries by `namespace`
Spack specs have "namespaces" that indicate what package repository they come from, but
there has not been a way to use the spec syntax to match one.
You can say things like this:
```console
spack find builtin.zlib
spack find myrepo.zlib
```
But, because namespaces are written as a dot-separated prefix on the name, you can't say
"find me all specs in namespace myrepo". The syntax doesn't allow it.
This PR allows you to specify anonymous specs with namespaces on the CLI. Specifically
you can do queries like this:
```console
spack find namespace=builtin
spack find namespace=myrepo
```
You can use this anywhere else you use spec syntax, e.g. in a config file to separate
installations based on what repo they came from:
```yaml
spack:
config:
install_tree:
root: $spack/opt/spack
projections:
namespace=myrepo: "myrepo_special_path/{name}-{hash}"
namespace=builtin: "builtin/{name}-{hash}"
```
This PR adds a special `namespace_if_anonymous` attribute to specs, which returns the
`namespace` if the spec has no name, otherwise it returns `None`. This allows us to
print the namespace for anonymous specs but to continue hiding it for most views, as
we've done so far.
This is implemented as a special case, but it's one that already exists, along with
`platform`, `os`, `target`, etc. This also reserves existing special case names for
variants so that users cannot define them in their package files. This change is
potentially breaking, but I do not think it will be common. There are no builtin
packages with a variant called `namespace`, and defining `os`, `target`, or `platform`
as a variant would've likely caused other problems if they were already being used.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* format: allow spaces in format specifiers
Key-value pair format specifiers can now contain spaces in the key. This allows us to
add spaces to format strings that are *only* present when the attribute formatted is not
``None``. Instead of writing:
```
{arch=architecture}
```
and special casing `arch=` like a sigil in `Spec.format()`, we can now write:
```
{ arch=architecture}
```
And the space is *only* printed when `architecture` is not `None`. This allows us to
remove the special case in `Spec.format()` for `arch=`.
Previously the only `key=` prefix allowed in format specifiers was `arch=`, but this PR
removes that requirement, and the `key=` part of a key-value specifier can be any name.
It does *not* have to correspond to the formatted attribute.
- [x] modify `SPEC_FORMAT_RE` to allow arbitrary keys in key-value pairs.
- [x] remove special case for `arch=` from `Spec.format()`.
- [x] modify format strings using `{arch=architecture}` to use `{ arch=architecture}`
- [x] add more tests for formatting
This PR saves other more complex attributes like compiler flags and their spacing for later.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
VC toolset versions 144 and 143 are both associated with the platform
toolset 143; this deviates from prior version choices made by the
MSVC devs; add a special case to report platform toolset version
as 143 when VC toolset version is >= 143 (this will need revisiting
for later releases).
The setSet tool (removed in version 10) has an optional dependency on readline.
The build script will use the system readline if not provided by Spack.
* dd4hep: restrict to podio@0 to prevent failures with podio@1
* acts: restrict to podio@0 to prevent failures with podio@1
* dd4hep: close when range for podio
* acts: close when range for podio
* acts: fix when range for podio
* acts: close when range for podio
* acts,dd4hep: ensure main/master still depends on podio
* Fix +rocm variant, to ensure correct dependencies on ROCm packages
and use of AMD LLVM
* Add a +pshm variant for comm=gasnet to enable fast shared-memory
comms between co-locales
* Add logic to ensure we get the native CXI libfabric network provider
on Cray EX
* Expand dependency type for package modules to encompass runtime
dependencies
* Factor logic for setting (LD_)LIBRARY_PATH and PKG_CONFIG_PATH of
runtime dependencies
* Workaround issue #44746 that causes a transitive dependency on lua
to break SLURM
* Disable nonfunctional checkChplDoc test
* Annotate some variants as conditional, to improve spack info output
and reduce confusion
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
`apple-libuuid` includes types that aren't available in other `uuid`
providers; this cause issues in consuming packages (e.g., py-matplotlib)
that use SDKs like CarbonCore.framework when they attempt to use
`util-linux-uuid` as a `uuid` provider on Mac OS.
Tweak `util-linux-uuid` to indicate that it does not provide `uuid`
on Mac OS.
This was missed in #45394 because we don't run unit tests for package PRs, and
`test_all_virtual_packages_have_default_providers`, which would've caught it, is a unit
test, not an audit.
- [x] add a default provider for `wasi-sdk` in `etc/spack/defaults/packages.yaml` (which
we require for all virtuals)
- [x] rework `test_all_virtual_packages_have_default_providers` as an audit called
`_ensure_all_virtual_packages_have_default_providers`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* pinentry: New API
* move code around
* added back version check
* Complete check_version refactor
* Honor original handling of guis (i.e., don't try if not there)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* upgrade apptainer: security fix
apptainer 1.3.1 has a high severity security issue: CVE-2024-3727
Upgrade to 1.3.2 or preferably 1.3.3
* added comment for 1.3.1
---------
Co-authored-by: Walter de Jong <walter.dejong@surf.nl>
* intel-oneapi-compilers: update description with current compilers
* Update var/spack/repos/builtin/packages/intel-oneapi-compilers/package.py
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* intel-oneapi-compilers: break docstring line
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* kokkos: new test API
* kokkos: added import llnl.util.lang as lang because earlier versions couldn't be installed without it
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* povray: new test API
* capture output and test name change
* povray: add v3.7.0.10, deprecate 3.7.0.8
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* umpire: old to new test format
* umpire: old to new test method and refactor
* indentation
* black reformat
* last minute syntax
* docstring and checks
* black format
* change test name
* method call correction
* Resolve problem with stand-alone tests (or examples) not loading library (Fixes#44960)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* sz: new test API
* fix typo; check installed executable; conform to subpart naming convention
* skip tests early if not installed; remove unnecessary "_sz" from test part names
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Library names on Windows are not typically prefixed with lib; the default
`.libs` implementation accounts for this, but `netlib-lapack` has a
custom implementation of `.libs` that did not account for this.
The `Executable` class emits a warning when you pass quoted arguments to it:
```
> spack pkg grep '"namespace"'
==> Warning: Quotes in command arguments can confuse scripts like configure.
The following arguments may cause problems when executed:
"namespace"
Quotes aren't needed because spack doesn't use a shell. Consider removing them.
If multiple levels of quotation are required, use `ignore_quotes=True`.
```
This is to warn new package authors who aren't used to calling build commands in python.
It's not useful for `spack pkg grep`, where we really are passing args on the command
line, and if we pass a quoted string, we probably meant to.
- [x] make `ignore_quotes` an instance variable, not just an argument to ``__call__`
- [x] set `ignore_quotes` to `True` in `spack pkg grep`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* package/linux-pam: dependencies
* Adding variants to linux-pam
* Updating linux-pam variants
* Fixing variants for linux-pam after testing
* clean up flag handling
* clean up terrible tabs
* cant use default_args for compiler dependencies
* Change selinux to off by default
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Adding new versions and compilation conflict for nodejs
* Update failed version for gcc14
* Updating conflicts notes for correctness/clarity/format
* Applying spack-ized versions of fix in https://github.com/nodejs/node/issues/52223 to adddress CI failures
* Update fix-old-glibc-random-headers.patch
* Update package.py
* Update fix-old-glibc-random-headers.patch
* Update fix-old-glibc-random-headers.patch
* Adding conflict for older glibc
* Fixing patch for older systems, need to undef
* Removing overly strict conflicts
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Several tests can be enabled on Windows with no changes to logic
(either the test logic or logic being tested)
* Test for `spack location` requires modification of the test logic,
but with a minor change can be enabled on Windows
using self.compiler.cc_pic_flag here results in these errors:
==> sqlite: Executing phase: 'install'
==> Error: AttributeError: 'AutotoolsBuilder' object has no attribute 'compiler'
change it to self.pkg.compiler.cc_pic_flag instead.
* imoroved-rdock: new test API
* Make test subpart names and or descriptions more accurate
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* e4s ci: enable some disabled specs
* comment out cp2k +cuda due to unsupported cuda_arch
* comment out dealii+cuda due to concretize error
* work through concretize errors
* e4s: comment out failing builds
* e4s stack: disabled non-building specs
* comment out failing specs
* comment out failing specs
* cleanup comments
* rust: rework external find to require both rustc and cargo
* rust: handle unable to parse version
* [@spackbot] updating style on behalf of wdconinc
* rust: not x or not y -> not (x and y)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: pick first rustc found
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: list comprehensions
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
* libmesh: add missing v1.7.1 release
* libmesh: avoid pulling in petsc if only +mpi or +metis is given
* libmesh: add shared variant
Co-authored-by: rbberger <rbberger@users.noreply.github.com>
The "use_store" context manager is used to swap the value
of a global variable (spack.store.STORE), while keeping
another global variable consistent (spack.config.CONFIG).
When doing that it tries to evaluate the previous value
of the store, if that was not done already. This is wrong,
since the configuration might be in an "intermediate" state
that was never meant to trigger side effects.
Remove that operation, and add a unit test to
prevent regressions.
Add language dependencies `c`, `cxx`, and `fortran`.
These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.
The `# generated` comment can be removed by package maintainers after
validating correctness.
* detray: new package
This commit adds the detray package, a detector description library for
HEP experiments that is designed to be GPU-friendly.
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* vasp: add new version 6.4.3 and rework package
* vasp: remove redundant cuda dep
* vasp: bump fftlib variant version restriction
* vasp: honor the still existing scalapack variant
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Since the the MetaPathFinder now owns a lazily constructed RepoPath object, we need to deserialize environments before the package that needs to be restored. Before we were relying on globals to be inconsistent in a way that let the entire process go.
The patch which enables icx/ifx compilers is only added for `wrf@4.4:`. This PR prints a useful message at concretization time instead of failing the installation later on.
Co-authored-by: stephenmsachs <stephenmsachs@users.noreply.github.com>
* slate: changed from old to new format
* make code tighter
* replace assert method
* SkipTest plus other cleanup
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* vecmem: add infrastructure for working with SYCL
The vecmem package uses an unorthodox build system where, instead of
expecting a SYCL-capable compiler in the `CXX` environment variable, it
expects one in `SYCLCXX`. It also needs the correct SYCL flags to be
set. This commit adds a custom build environment for the vecmem package
which allows it to be built in this way. I've also added an extra CMake
flag to ensure that the build system doesn't download any unwanted
dependencies.
* Update var/spack/repos/builtin/packages/vecmem/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
- Add explicit conflict on ppc64le for 12.5 and newer.
- Update/fix intel compiler conflict to reflect that intel@2021 is compatible
only since 11.4.1 and not since 11.1.1.
- Add intel compiler conflicts to reflect strict support matrix since
12.2.0.
As discussed in https://github.com/spack/spack/pull/44881#issuecomment-2218411735 a `spack install py-globus-cli` fails to concretize on an Ubuntu 22.04 under Windows WSL2 because of too strict of explicit dependencies.
Let's try to remove them here (since these are "just in case" and in all honesty should be handled by `py-globus-sdk` anyways).
* Now builds with latest fmt release (11.0.1)
* Missing array include in nem_spread
* Fix timestep conssitency check in file-per-rank case if one or more dbs have no timesteps.
If an external happens to be a provider of anything, the solver would
set its weight to 0, meaning that it is most preferred, even if
packages.yaml config disagrees.
That was done so that `spack external find mpich` would be sufficent to
pick it up as mpi provider.
That may have made sense for mpi specifically, but doesn't make sense
for other virtuals. For example `glibc` provides iconv, and is an
external by design, but it's better to use libiconv as a separate
package as a provider.
Therefore, drop this rule, and instead let users add config:
```
mpi:
require: [mpich]
```
or
```
mpi:
buildable: false
```
which is well-documented.
This change makes `spack external find --not-buildable` mark virtuals
provided by detected packages as non-buildable, so that it's sufficient
for users to let spack detect say mpich and have the concretizer pick it
up as mpi provider even when openmpi is "more preferred".
Consistently with ArmPL@24:, the include directory for acfl@24:
has changed to include. The change wants to update to this change
and distinguish the include path for releases previous to 24.04
and the future ones
* rocm-cmake: changed test format from old to new
* Rename cmake variable
* post-conflict resolution: remove remaining version check
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Added Release 0.0.4 and 0.0.5
* Changed requirement for gotcha
use gotcha 1.0.5 for 0.0.2 and 0.0.3
* Combine gotcha 1.0.7 for master and develop
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
in case of inheritance the static tags prop may be updated multiple
times, and it turns out builder classes magically inherit from
traditional package classes
While trying to build packages with the OneAPI compiler version 2024.1 I
ran into the following error, indicating that the compiler is unable to
find some necessary libraries:
```
/storage/Software/oneapi/2024.1/compiler/2024.1/bin/sycl-post-link: error
while loading shared libraries: libonnxruntime.1.12.22.721.so: cannot open
shared object file: No such file or directory
icpx: error: unable to execute command: No such file or directory
icpx: error: sycl-post-link command failed due to signal (use -v to see
invocation)
```
Indeed, `libonnxruntime.1.12.22.721.so` does come bundled with the
OneAPI compiler, but it is not available in the build environment by
default. In this commit, I update the custom environment created by
OneAPI to include the `lib/` directory in which these libraries reside
in the `LD_LIBRARY_PATH` environment variable.
In my enthusiasm to add dfelibs to Spack, I didn't realise that the
unit tests of dfelibs use Boost and, as such, Boost is required as a
testing dependency.
Originally if you had `x -> y -> z`, and an env with `x` in its speclist that is concretized but not installed, then `spack find -c y` would not show anything. This was intended: `spack find` has up-until-now only ever listed out installed specs (and `-c` was for adding a preamble section about roots).
This changes `spack find` so:
* `-c` makes it search through all concretized specs in the env (in a sense it is anticipated that a concretized environment would serve as a "speculative" DB and users may want to query it like they query the DB outside of envs)
* Adds a `-i/--install-status` option, equivalent to `-I` from `spack spec`
* Shows install status for either `-c` or `-i`
* As a side effect to prior point, `spack find -i` can now distinguish different installation states (upstream/external)
Examples:
```
$ spack find -r
==> In environment findtest
==> 1 root specs
- raja
==> 6 installed packages (not shown)
==> 12 concretized packages to be installed (not shown)
```
```
$ spack find
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
berkeley-db@18.1.40 bzip2@1.0.8 diffutils@3.10 gmake@4.4.1 gnuconfig@2022-09-17 libiconv@1.17
==> 6 installed packages
==> 12 concretized packages to be installed (show with `spack find -c`)
```
```
$ spack find -c
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
[+] berkeley-db@18.1.40 [+] bzip2@1.0.8 - cmake@3.29.4 [+] diffutils@3.10 [+] gmake@4.4.1 [+] libiconv@1.17 - nghttp2@1.62.0 - pkgconf@2.2.0 - readline@8.2
- blt@0.6.2 - camp@2024.02.1 - curl@8.7.1 - gdbm@1.23 [+] gnuconfig@2022-09-17 - ncurses@6.5 - perl@5.38.2 - raja@2024.02.2 - zlib-ng@2.1.6
==> 6 installed packages
==> 12 concretized packages to be installed
```
$ spack -E find
...
==> 82 installed packages
```
* Add py-datasets version 2.20.0
* Add dependency requirements for version 2.20 + refactor
* Add missing tqdm and requests versions and to install latest py-datasets
* Add missing python requirements for 2.8.0 and 2.20.0
Fix a bug where Spec.tree with cover=nodes reduces deptypes from all
in-edges, including from nodes not reachable from the root, which almost
always happens for concrete specs
`Spec.__getitem__` queries dependent edges, which almost always point to
nodes outside the sub-dag considered. It should only ever look at edges
being traversed.
When using a newer Clang for Kokkos than supported by a given CUDA version, the
CUDA compiler detection in Legion's CMake still needs to be passed
CMAKE_CUDA_FLAGS to pass the compiler check.
This modifies heuristic to decay to clingo default
over time. The hope is that this helps with specs
that have an optimal solution with a high penalty.
Let target and compiler heuristic decay too, do not
guess compiler
* dfelibs: new package
This commits adds the `dfelibs` package, which is an Acts project fork
of dfelibs.
* dfelibs: docstring typo
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Add MatrixSwitch package
Add libOMM package
Use libOMM as external in ELSI
Add include paths for all libraries
Fortran modules need one directory up (can't use libs.directories since they are just the include/ directories)
This addresses a few issues in the kentutils package:
- Issue #44372
- Cleaning up formattting and styles
- Removing old versions that are not avaialble anymore
- Removing newer versions that are also not available anymore
- The installer does not install the static libs
that are expected by packages that depend on kentutils
- There are conflicts and less-than-desirable dependencies
that can be addressed via variants
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Bumping up to ROCm 6.1.2
* Bump up rocmlir & amdsmi to 6.1.2
* Removing llvm dependency from amdsmi
* Removed redundant for loops and brought rocm-cmake in for loop
* Removing version check of deprecated versions
This adds spack.util.spack_yaml.anchorify, which takes a non-cyclic
dict/list structure, and replaces identical values with (back)
references to the first instance, so that yaml serialization will use
anchors.
`repr` is used to identify sub-dags, which in principle is quadratic
complexity in depth of the graph, but in practice the depth is O(1) so
this should not matter.
Then this is used in CI to reduce the size of generated YAML files to
30% of their original size.
* lammps: add new versions
* lammps: add directories for plugins
* lammps-example-plugin: new package
Example illustrating how to install a LAMMPS plugin, based on the
example part of the LAMMPS distribution.
If netlib-lapack is built with ~external-blas, it internally links
liblapack.so with libblas.so, meaning that whenever netlib-lapack is
used as a lapack provider, the package must also be a blas provider.
Conversely using netli-lapack as a blas provider does not imply that it
also must provide lapack, but nothing is lost disallowing that...
In practice people don't care about having their build dependencies reinstalled with say cflags=-O3 if that is what is set at the input spec, so restrict propagation to link/run deps only.
Also simplify the encoding in asp.
* Changed test method from old method to new method
* Corrected test name and added docstring
* Split test method into stand alone-tests
* Added SkipTest
* code refactor
* removed repeated code
* Remove exe from test part purpose
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve proj version selection to avoid too new of a proj with older api usage in vtk9
* Proj constraint range around 8.2.1
---------
Co-authored-by: John Parent <john.parent@kitware.com>
Fix the following on Windows:
* `spack load --list` (this printed 0 packages even if packages were
loaded)
* `spack unload <package>` (this said that the package is not loaded
even if it was)
Update unit tests for `spack load` to also run on Windows (specifically
for ".bat"). This involved refactoring a few tests to parameterize
based on whether the unit tests are being run on a Windows system
(and to account for batch syntax).
* Adding additional versions to plink2 and switching to tarballs to allow for better version detection in the future
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
The pcluster pipeline runs spack external find and autodetects the most ancient of zlib versions on the system: 1.2.7, release date May 3, 2012.
Turns out some packages require parts of the zlib api added within the last decade.
* update e4s stacks
* adios2 +rocm: disable kokkos due to spack issue #44832
* comment out mgard+cuda due to spack issue #44833
* comment out cabana, legion, arborx due to kokkos spack issue #44832
* comment out slepc, petsc due to petsc spack issue #44600
* comment out adios2+rocm due to kokkos rocm spack issue #44832
* comment out kokkos due to spack issue #44832
When we changed how to deal with errors in November,
we didn't realize that for an unconstrained choice
rule it is more important in the heuristic to guess
what is NOT in the answer set, since it will be the
majority of options.
Previously this was following automatically from what
was in the answer set, via `1 { ... } 1` cardinality
constraints.
Here we improve the heuristic and the solve time for specs.
* Patched CMakeLists.txt to find drm include at libdrm - archlinux
* Restricted usage patch to versions 34 onwards
* Update var/spack/repos/builtin/packages/rdma-core/libdrm.patch
---------
Co-authored-by: Mark Abraham <Mark.J.Abraham@gmail.com>
* Spglib: add version 2.4.0
* DLA-Future: fix +test option
* elsi
* elsi improvements
* [@spackbot] updating style on behalf of RMeli
* remove test variants
* fix ntpoly and and use externals as default
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* py-multiqc: updating to @1.21, adding new dependency py-pyyaml-env
* Correcting dependencies
* Drop matplotlib depends_on() for versions of multiqc not included in package.py
---------
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
* Allow zlib to find external installations.
* Apply suggestions from code review
* Fixed the determine_version function to loop over all of the potential
libraries that could be installed by zlib.
---------
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
* Update hohqmesh and feq-parse versions
Update feq-parse homepage to new documentation page
Update feq-parse license to be consistent with 2.2.2 release
---------
Co-authored-by: fluidnumerics-joe <fluidnumerics-joe@users.noreply.github.com>
This commit adds new versions of actsvg. I have also taken the liberty
of adding myself a maintainer as I work actively on the ACTS project and
its dependencies, and would like to start keeping the Spack specs more
up to date.
* e4s external rocm ci: bump rocm stack to v6.1.1
* comment out exago+rocm due to issue with raja@0.14.0 see spack issue #44593
* comment out adios2+rocm due to spack issue #44594
* comment out petsc+rocm due to spack issue #44600
* comment out sundials+rocm due to spack issue #44601
* comment out slepc+rocm due to petsc spack issue #44600
* comment out tau+rocm due to spack issue #44659
* comment out ecp-data-vis-sdk due to spack issue #44745
* packages: register rocm-core as external
* re-enable tau due to issue #44659 having been resolved
* use latest ci image: ecpe4s/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.1.1:2024.06.17
* comment out paraview due to spack issue #44745
* comment out ecp-data-vis-sdk +vtkm due to issue https://gitlab.spack.io/spack/spack/-/jobs/11632511
* Update for 2.0.1
* cloudpickle dependency is only 'run'
* Follow new formatting guidelines
* black wants trailing commas
* Simplified version ranges, as recommended by @tldahlgren
* vtk-m: Changed test method names and skipping non-applicable tests from old to new approach
-----
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* add cray detection taken from upcxx
* add CUDA/ROCm support
* add numerous pass-through options to Chapel build,
like gpu_mem_strategy, comm_substrate, etc.; all variants are
translated to analogous CHPL_* environment variables. As a side
effect, this defines a number of environment variables that are
not actually used by Chapel.
* Define LD_LIBRARY_PATH, LIBRARY_PATH, and PKG_CONFIG_PATH to
help programs built with Chapel properly locate needed runtime
dependencies
---------
Co-authored-by: bonachea <dobonachea@lbl.gov>
* Projsync needs curl
This fixes a proj compilation error from paraview
* tiff does not set TIFF_INCLUDE_DIR tested by proj
* TIFF patch exists for proj 9.2
* Add comments with location of proj patch
* Use TIFF patches for different cmake versions
* Use modules starting cmake 3.28 and variables before that
* Fix black style
* Simpler patches, add newer proj
* Remove duplicated flag
* Deprecate PROJ 4.8 and older
* Remove PRIVATE
* Deprecate PROJ 7.0.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Not sure why I had this here, as `/bin/sh` will find the first spack in `PATH` just like
`env`.
- [x] remove `/usr/bin/env` and avoid an extra process launch.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* omega-h: reformatted from old test method to new test method
* omega-h: added proper output checking
--------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The ``+cluster` variant requires there be an MPI family known in the
spec. When using externals it is easy to miss-configure this requirement
leading to a runtime exception which is not desirable. This converts the
exception to a package rule.
* xorg libs: new versions
* xorg pkgs: update homepage from cgit to gitlab
* xorgs pkgs: fix homepage since, yeah, I did edit those 122 files by hand...
* libxmu: don't need the .0 patch version here
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Fixed some issues with installation breaking due to mysql_client
* Remove debugging details
* Adding client_only to deps
* Adding client_only to variant and deps
* Undoingw client_only to variant and deps
* py-pyopengl: new simple package
* py-pyopengl: Fix typo in comment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pyopengl: add variants and dependencies
* py-pyopengl: build from source and improve variants
* py-pyopengl: use corrected freeglut libs
* py-pyopengl: update copyright
* py-pyopengl: remove duplicate conflict clause
* py-pyopengl: change dependency to link
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Core change: logic for extracting RPATHs from modules may return
`None`: filter this out of the set of RPATHs that is auto-generated
* Core change: `CachedCMakePackage` no longer adds ldflags to
`CMAKE_STATIC_LINKER_FLAGS`: generally these flags are not appropriate
for static linking (e.g. invocation of `ar`)
* [mfem] Add version 4.7
* [mfem] Add variant for precision (single/double). Enforce consistency
for precision amongst mfem and hypre/petsc/mumps dependencies
* [mfem] Add cxxstd (and related constraints preventing use of
old cxxstd values for newer versions of some dependencies)
* [hypre] In line with prior point, added support for specifying
precision
* [petsc] Add config option to avoid error when building against
`superlu-dist+rocm`
* [hiop] add proper `raja`/`umpire`/`camp` version constraints for
`hiop` versions 0.3.99-0.4.x; require `+raja` for `+rocm`, and
add dependency on `hiprand` for `+rocm`
* [butterflypack, mfem, strumpack, suite-sparse] Require
`CRAYLIBS_{target-family}` env var to be defined
* [suite-sparse] versions `@7.4:` changed install location of headers:
add symlink from old location to new location
* [zlib-ng] Fix error where shared libs were not successfully built for
`%cce@17` (the build did not fail, but the finished `zlib-ng%cce@17`
install did not have shared libs)
* suite-sparse library path can be lib or lib64
* fixing style check error
* Adding the missing import os
* Style check error fix
* Updating patch for 6.1.0 with correct lib path
* Preserve higher weight for CLI git ref versions
Currently the concretizer fails if you reuse a git ref version
that has already been installed but modify the spec at all.
See #38484 for futher diagnosis
The issue here is that since there is no established provenance for
these versions the highest weight they are currently assigned is
that of prior install. Re-use checks then fail because the weight of
the version is identical to the solver.
Ironically, these versions are given the highest weights possible when
specified on the CLI for the first time. They should only appear in a
DAG if they are an exact match or if the user specifies them at the CLI.
Therefore it makes sense to preserve their higher ordering.
Getting this right is critical to moving all branch based versions to a pinned
git-ref in the future.
* [@spackbot] updating style on behalf of psakievich
* Update lib/spack/spack/solver/asp.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Add provenance specific to git ref installs
* Sensitvity to name that I could not track down
* Add regression test
* Adjust test
* Add prefer standard unit-test
* Style
* Add required mock
* Format and mark
* Make unit-test case reproduce CLI investigation
* Remove unnecessary mock package
* [@spackbot] updating style on behalf of psakievich
* Use already developed fixture
* Add zlib-ng to mocks again
* Remove accidental adds
* Remove maintainer
* [@spackbot] updating style on behalf of psakievich
* Rename test file
* [@spackbot] updating style on behalf of psakievich
* Remove unused imports
* Update tests
* [@spackbot] updating style on behalf of psakievich
* Style
* Update lib/spack/spack/test/concretize.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Update solver rule
* Duplicate installed rules for installed_git_version
* Revert "Duplicate installed rules for installed_git_version"
This reverts commit 17223fc8d1.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
* libdistributed: renamed test to check_test
* libpressio-rmetric: renamed test method
* libpressio-tools: renamed test method
* Renamed build-time test methods for: libpressio-opt, libpressio-tthresh, opened
---------
Co-authored-by: Jackson Peter Lawrence <lawrence31@borax5.llnl.gov>
* Provide instructions for building a developmental version of lcov.
* style fix
* style fix
* Promote single version git URL to the package level.
* Formatting fix.
* verilator: Some environment varibles are no longer required
* Revert depends_on for flex back to the deafult case
* Use decorator, when, for setup_run_environment insdtead of if inside function
* libtiff: add v4.6.0 and default disable opengl (#44545)
* libtiff: Fix typo in CMake key
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: Broader description of OpenGL variant
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: reformat using spack style black recommendation
* libtiff: couple opengl flag with autotools
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There hasn't been a release in almost a decade, and build failures with
GCC 14 were reported. I don't think it makes sense to patch it, since
the project moved over to `util-linux`, and Spack's default provider is
that package. Better to just get rid of it in the next Spack release.
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/pcre-8.45-33dfhul
[+] /opt/apps/spack/libpciaccess-0.17-2qdxdjo
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/qhull-2020.2-klc7ewb
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/alsa-lib-1.2.3.2-a7icjdy
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/yasm-1.3.0-v4etmmm
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/unzip-6.0-mqftjtp
[+] /opt/apps/spack/libjpeg-turbo-3.0.0-vjvivme
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/libpng-1.6.39-kiuku4y
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/swig-4.0.2-fortran-z3sbnze
[+] /opt/apps/spack/binutils-2.42-vkjcvfr
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/hdf5-1.14.3-sbuiw6q
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/ffmpeg-6.1.1-2vhrbda
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/freetype-2.13.2-in4taxi
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/hwloc-2.9.1-fhoz6al
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/lua-5.3.6-47356ia
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/elfutils-0.190-uswzaiw
[+] /opt/apps/spack/py-pytz-2023.3-ojdlhrd
[+] /opt/apps/spack/py-cycler-0.11.0-b7mjvv7
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/llvm-14.0.6-3nosumn
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-six-1.16.0-iv6iv3q
[+] /opt/apps/spack/py-pybind11-2.12.0-5rvupjv
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-pyparsing-3.1.2-fq6imlt
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/py-llvmlite-0.41.1-qom3l5h
[+] /opt/apps/spack/py-astropy-4.0.1.post1-xbojixg
[+] /opt/apps/spack/py-python-dateutil-2.8.2-kzdfskc
[+] /opt/apps/spack/py-numexpr-2.8.4-fc5xc5s
[+] /opt/apps/spack/py-h5py-3.11.0-y6drk3j
[+] /opt/apps/spack/py-tifffile-2023.8.30-oof4try
[+] /opt/apps/spack/py-pygments-2.16.1-stgrccl
[+] /opt/apps/spack/py-mdurl-0.1.2-nqk43ry
[+] /opt/apps/spack/py-scipy-1.13.0-vxjgfov
[+] /opt/apps/spack/py-contourpy-1.0.7-jg5lhss
[+] /opt/apps/spack/py-pillow-10.3.0-ijh2cju
[+] /opt/apps/spack/py-kiwisolver-1.4.5-vdh5sq5
[+] /opt/apps/spack/py-fonttools-4.39.4-x5ydbox
[+] /opt/apps/spack/py-bottleneck-1.3.7-ztsm4lg
[+] /opt/apps/spack/py-lazy-loader-0.4-k7hgvka
[+] /opt/apps/spack/py-numba-0.58.1-hzvrjwj
[+] /opt/apps/spack/py-markdown-it-py-3.0.0-l4p4qv5
[+] /opt/apps/spack/py-imageio-2.34.0-z5hu4yc
[+] /opt/apps/spack/py-matplotlib-3.8.4-azq2fzm
[+] /opt/apps/spack/py-pandas-1.5.3-p3gnh6t
[+] /opt/apps/spack/py-rich-13.4.2-okhgwwz
[+] /opt/apps/spack/py-networkx-3.1-b54br7r
[+] /opt/apps/spack/py-scikit-image-0.23.2-cvyzh3t
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
#40773 introduced python-venv, which improved build isolation and avoids issues with,
e.g., `ubuntu`'s system python modifying `sysconfig` to include a (very unwanted)
`local` directory within the default install layout.
This addresses a few cases where #40773 removed functionality, without harming the
default cases where we use `python-venv`.
Traditionally, *every* view with `python` in it was essentially a virtual environment,
because we would copy the `python` interpreter and `os.py` into every view when linking.
We now rely on `python-venv` to do that, but only when it's used (i.e. new builds) and
only for packages that have an `extends("python")` directive.
This again makes every view with `python` in it a virtual environment, but only
if we're not already using a package like `python-venv`. This uses a different
mechanism from before -- instead of using the `virtualenv` trick of copying `python`
into the prefix, we instead create a `pyvenv.cfg` like `venv` (the more modern way
to do it).
This fixes two things:
1. If you already had an environment before Spack `v0.22` that worked, it would
stop working without a reconcretize and rebuild in `v0.22`, because we no longer
copy the python interpreter on link. Adding `pyvenv.cfg` fixes this in a more
modern way, so old views will keep working.
2. If you have an env that only includes python packages that use `depends_on("python")`
instead of `extends("python")`, those packages will now be importable as before,
though they won't have the same level of build isolation you'd get with `extends`
and `python-venv`.
* views: avoid making client code deal with link functions
Users of views and ViewDescriptors shouldn't have to deal with link functions -- they
should just say what type of linking they want.
- [x] views take a link_type, not a link function
- [x] views work out the link function from the link type
- [x] view descriptors and commands now just tell the view what they want.
* python: simplify logic for avoiding pyvenv.cfg in copy views
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
In #30882, we made Spack ignore `-Werror` calls so that it could more easily build
projects that inject `-Werror` into their builds. We did this by translating them to
`-Wno-error` in the compiler wrapper. However, some compilers (like `nvhpc`) do not
support `-Wno-error`. We need to exclude them from this feature until they do.
- [x] make a property on `PackageBase` for `keep_werror` that knows not to use it for
`nvhpc`.
- [x] update property so that it keeps only the specific `-Werror=...` args for newer nvhpc's,
which support `-Wno-error` but not `-Wno-error=...`
---------
Co-authored-by: William Mou <william.mou1024@gmail.com>
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
When Spack concretizes environments, it prints every (newly concretized) root spec
individually with all of its dependencies. For most reasonably sized environments, this
is too much output. This is true for three commands:
* `spack concretize` when concretizing an environment with newly added specs
* `spack install` when installing an environment with newly added specs
* `spack spec` with no arguments in an environment
The output dates back to before we had unified environments or nicer spec traversal
routines, and we can improve it.
This PR makes environment concretization output analogous to what we do for regular
specs. Just like `spack spec` for a single spec, we show all root specs with no
indentation, so you can easily see the specs you explicitly requested. Dependencies are
shown:
1. With indentation according to their depth in a breadth-first traversal starting at
the roots;
2. Only once if they appear on paths from multiple roots
So, the default is now consistent with `spack spec` for one spec--it's `--cover=nodes`.
i.e., if there are 100 specs in your environment, you'll get 100 lines of output.
If you want to see more details, you can do that with `spack spec` using the arguments
you're already familiar with. For example, if you wanted to see dependency types and
*all* dependencies, you could use `spack spec -l --cover=edges`. Or you could add
deptypes and namespaces with, e.g. `spack spec -ltN`.
With no arguments in an environment, `spack spec` concretizes (if necessary) and shows
the concretized environment. If you run `spack concretize` *first*, inspecting the
environment repeatedly with `spack spec` will be fast, as everything is already in the
`spack.lock` file.
- [x] factor most logic of `Spec.tree()` out of `Spec` class into `spack.spec.tree()`,
which can take multiple specs as roots.
- [x] make `Spec.tree()` call `spack.spec.tree()`
- [x] `spack.environment.display_specs()` now uses `spack.spec.tree()`
- [x] Update `spack concretize`
- [x] Update `spack install`
- [x] Update `spack spec` to call `spack.spec.tree()` for environments.
- [x] Continue to output specs individually for `spack spec` when using
`--yaml` or `--json`
* Apply patch from upstream to allow vtk compilation with %gcc 12 and 14.
* Fixes#44331
* fix soec usage
* fix compiler version range
* Finalize version range, switch to .diff file
---------
Co-authored-by: Chrismarsh <Chrismarsh@users.noreply.github.com>
* Now need to explicitly depend on libfabric for Cray EX
* Ensure build uses the selected CUDA and ROCm versions
* Correct spelling on `configure --with-ldflags`
* Patch a defect regarding `configure --with-ldflags`
* Default to Spack's copy of GASNet-EX, which is newer than embedded
e2fsprogs: adding version 1.47.0 and fuse2fs variant
fuse-overlayfs: adding version 1.13
squashfuse: adding version 0.5.1 and 0.5.2
e2fsprogs: fixing audit errors
apptainer: expanding deps to cover more versions, fix binary_path logic.
apptainer: add go version deps, fix e2fsprogs patch sums, rework dep paths
fuse-overlayfs: tightening up the libfuse dep
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* util-macros: ensure url_for_version only used for older versions
* util-macros: use url.substitute_version after xz -> bz2
* util-macros: mv url_for_version down the file
`BuildcacheBootstrapper` uses `Spec.intersects` to match specs needed
for bootstrapping against the binary cache. The specs were not
sufficiently-detailed to prevent matching e.g. cached binaries for
Mac OS on Windows; this commit adds the platform to each requested
bootstrap spec to prevent that.
* initial commit for RoCm-6.1.1 release
* fix style issues
* update the version for rocmlir and rocm-cmake
* restrict the patch for 6.0.0 release
* fix build failure for hipsolver-6.1.1 release
* update the hipblaslt for rocm-6.1.1 release
* add the patch for hipsparselt for 6.1.1 rel
Remove support for `cray` as a separate platform.
Any platform previously detected as `cray` is now detected as `linux`.
Users who still need platform=cray have to stick to Spack 0.22
Fixes a bug in the concretizer where specs depending on a host
incompatible libc would be used. This bug triggers when nothing is
built.
In the case where everything is reused, there is no libc provider from
the perspective of the solver, there is only compatible_libc. This
commit ensures that we require a host compatible libc on any reused
spec, additionally to requiring compat with the chosen libc provider.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Patch to add hsa prefix/include path:wq
* Style check error fix
* The patch need to be applied to 2024.01.stable. ROCm 6.0 requires hpctoolkit 2024.01.1 or later
* Fixing style chec error
* Adding a py4j variant that requires Java via spack to avoid situations where a system doesn't have Java and py4j expects it
* Adding new versions of py-pyspark
* Adding a new variant to require java (via py4j) and clean up dependency handling
* Adding myself as a maintainer for py-pyspark and py-py4j
* Fix overlooked version bump in py4j
* Version bump to meet py-spark expectations
* Version bump to add latest compatibile version with pyspark
* Matching py-grpcio bump
* Adding variants and dependents for pyspark
* Adding runtime deps
* Changing default java requirement. I'm not sure this is the right call
* Changing py4j with java dependency handling
* Fix style
* Update package.py fix unnecessary f-strinh
* Make +java the default for both
* Fix nested deps
* Revert styles after default change
* Added new versions and GCC 14 compatbility conflicts
* Added new versions and compatibility conflicts for gcc 14
* Added new versions paired to arrow (for gcc14 compat)
* Update py-protobuf compiler conflict
* Update depends to match
See https://github.com/grpc/grpc/blob/master/src/python/grpcio_status/setup.py
* Updating dependencies and conflicts for py-googleapis-common-protos. Added new version to avoid future issues
* Remove upper bound version on py-protobuf and add default_args
* Adding new versions and updating dependencies back to versions 1.35.0
* Updating oldest numpy deps
* Fixing merge
* bit more cleaniness for var/spack/repos/builtin/packages/py-googleapis-common-protos/package.py
* Adding latest matching version of py-grpcio and py-grpcio-status
* Update package.py
https://github.com/spack/spack/pull/44263#discussion_r1612317943
* Update dependencies
* Adding additional versions for dependent packages. Deprecated two versions: 1.16 is old, built for python ~3.6, and does not build for 3.8. 1.52.0 was removed from pypi
* Revert py-grpcio-tools changes. Will include in separate PR
* Adding patches and constraints to get 1.48 to build as it's a dependency that is called out for some other packages
* Updating to account for yanked packages for dependencies
* Fix style
* Update sha256 for py-grpcio v0.16.0 to reflect change
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* py-astropy: Add version 6.1
Added build info for version 6.1 in py-astropy. Requires a new additional package, astropy-iers-data which has been included as py-astropy-iers-data to match with spack's general naming conventions.
Below is the output of the spack install showing successful build fro version 6.1 and the new py-astropy-iers-data package
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-pyerfa-2.0.0.1-kyfazhs
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/git-2.45.1-tuc5jnb
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
==> Installing py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj [52/55]
==> No binary for py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj found: installing from source
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj
Stage: 1.74s. Install: 0.93s. Post-install: 0.52s. Total: 3.36s
[+] /opt/apps/spack/py-astropy-iers-data-main-ukchsfz
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-qdhxchg
==> Installing py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo [55/55]
==> No binary for py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> Warning: Module file /opt/modulefiles/spack/Core/py-astropy/6.1.0.lua exists and will not be overwritten
==> py-astropy: Successfully installed py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo
Stage: 1.29s. Install: 1m 5.77s. Post-install: 0.60s. Total: 1m 7.94s
* Removed extra-whitespace
A blank line had white space, removed the white space
* Additional formatting changes for black
* Additional package updates
Based on previous recommendations updated py-astropy and py-astropy-iers-data packages. Also added a new version to py-pyerfa package to match the 6.1.0 dependencies better
Of importance in these updates, I did add pypi and version info to py-astropy-iers-data. Originally I had argued that this package updates quite frequently (on a weekly basis) and so it did not make sense to include pypi/versions and we should instead use the non-version git-repo structure based on the master branch. However, when I tried to build the package py-setuptools-scm errored out when trying to build the package:
/opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro/lib/python3.10/site-packages/setuptools_scm/git.py:163: UserWarning: "/tmp/root/spack-stage/spack-stage-py-astropy-iers-data-main-iw2mdzlukb37mkcbcozjjefjoefw2eyp/spack-src" is shallow and may cause errors
I believe this is due to the download file/stage directory not containing the version and instead including the branch. I changed the package to use versions instead and it worked just fine as shown below. In addition, when I had done some preliminary testing, the package installed fine using the non-version master branch. When I checked, that installation it used py-setuptools-scm@7.1 while in this installation run it used a much more recent 8.0.4 so it is possible that somewhere between scm7.1 and 8.0.04 something changed that caused this error to show up. Since the setuptools-scm package has something to do extracting package versions, I imagine it's some kind of mismatch issue
Output from build:
spack install py-astropy
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/git-2.45.1-zu6qkoc
[+] /opt/apps/spack/py-markupsafe-2.1.3-isgtki6
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
==> Installing py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a [49/57]
==> No binary for py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a found: installing from source
==> Fetching ac8a6fe91c.tar.gz
==> No patches needed for py-extension-helpers
==> py-extension-helpers: Executing phase: 'install'
==> py-extension-helpers: Successfully installed py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a
Stage: 0.37s. Install: 0.88s. Post-install: 0.52s. Total: 1.89s
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
==> Installing py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m [50/57]
==> No binary for py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m found: installing from source
==> Fetching 350b18f967.tar.gz
==> No patches needed for py-cython
==> py-cython: Executing phase: 'install'
==> py-cython: Successfully installed py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m
Stage: 1.07s. Install: 2m 49.80s. Post-install: 0.40s. Total: 2m 51.37s
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
[+] /opt/apps/spack/py-jinja2-3.1.2-wacpq7j
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro
==> Installing py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 [55/57]
==> No binary for py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 found: installing from source
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/7f/7fff3d3404ae8560533ac0e685db7acc02c4d8984faa4ac3d607096879fba2d1.tar.gz
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6
Stage: 0.07s. Install: 1.21s. Post-install: 0.39s. Total: 1.88s
[+] /opt/apps/spack/py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6
==> Installing py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp [56/57]
==> No binary for py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/pyerfa/pyerfa-2.0.1.1.tar.gz
==> No patches needed for py-pyerfa
==> py-pyerfa: Executing phase: 'install'
==> py-pyerfa: Successfully installed py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp
Stage: 0.93s. Install: 17.72s. Post-install: 0.33s. Total: 19.15s
[+] /opt/apps/spack/py-pyerfa-2.0.1.1-pkokp6u
==> Installing py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs [57/57]
==> No binary for py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> py-astropy: Successfully installed py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs
Stage: 1.16s. Install: 1m 3.34s. Post-install: 1.05s. Total: 1m 5.77s
[+] /opt/apps/spack/py-astropy-6.1.0-5brbkjn
* Updated to match with black formatting
Change the installer to take `([pkg], args)` in the constructor instead
of `[(pkg, args)]`. The reason is that certain arguments are global
settings, and the new API ensures that those arguments cannot be
different across different "build requests".
The `explicit` install arg is now a list of hashes, and the installer is
no longer responsible for determining what package is installed
explicitly. This way environment installs can simply pass the list of
environment roots, without them necessarily being explicit build
requests. For example an env with two roots [a, b], where b depends on
a, would not always cause spack install to mark b as explicit.
Notice that `overwrite` already took a list of hashes, this makes
`explicit` consistent.
`package.do_install(explicit=True)` continues to take a boolean.
* legion: add pip dependency for build
pip is needed for the build when the Legion Python binding is enabled
* legion: add variants for gc logging and system OpenMP use
This also removes the `cmake_cxx_flags` variable from `cmake_args()`
because that variable had no effect.
* [@spackbot] updating style on behalf of mpokorny
* legion: use spec.satisfies() in cmake_args()
e.g, replace use of '"+foo" in spec' with 'spec.satisfies("+foo")'
* legion: avoid configuring with multiple "-DLegion_REDOP_COMPLEX=ON" arguments
In the previous version, when both '+redop_complex' and '+bindings'
was set, two instances of "-DLegion_REDOP_COMPLEX=ON" arguments were
generated for cmake configuration.
* legion: fix value of "Legion_OUTPUT_LEVEL" for configuration
the previous version had no effect on setting the configuration value
---------
Co-authored-by: mpokorny <mpokorny@users.noreply.github.com>
The windows wrappers for basic functions like `os.symlink`,
`os.readlink` and `os.path.islink` in the `llnl.util.symlink` module
have bugs, and trigger more file system operations on non-windows than
they should.
This commit just binds `llnl.util.symlink.symlink = os.symlink` etc so
built-in functions are used on non-windows
`spack clean <spec>` will now resolve specs based on the active environment if one is active.
If an env is active but no matching spec is found, this will fall back on fully concretizing.
Before this PR, if Spack could see a possibility to reuse a spec that
doesn't match a strong preference, it would do so. After the PR, a
strong preference would take precedence.
`glibc` and `musl` provide a basic implementation of `iconv` (`iconv`,
`iconv_open`, `iconv_close`), but in practice the installation may be
missing the character encoding methods to make them usable. On Fedora
for example, users need to
```yum install glibc-gconv-extra```
to get the character encodings that `gettext` requires during configure,
namely EUC-JP. Users may not have permissions to install the missing
parts of glibc.
Since Spack can install `libiconv`, it is simpler to use that by
default.
* Spglib: add version 2.4.0
* dla-future-fortran: new package version 0.1.0
* [@spackbot] updating style on behalf of RMeli
* apply suggestion and add maintainer
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* [emacs] When installing on Mac OS, build and install the native
Emacs.app along with the standard executables.
* [emacs] Make the GUI build on Mac optional by adding "gui" variant
* Apply reviewer suggestion
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Add emacs version 29.3
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
avoid calling `spec.target` when None.
When an external compiler package has an `os` set but no `target` set, Spack
currently falls into a codepath that calls `spec.target` (which itself calls
`spec.architecture.target.Microarchitecture`) when `spec.architecture.target`
is None, throwing an error.
e.g.
```
packages:
gcc:
externals:
- spec: gcc@12.3.1 os=rhel7
prefix: /usr
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* glew: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-demos: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-glu: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* paraview: fix dependency on glew
* mesa: group dependency on when("+glx")
* Add missing dependency on libxml2
* paraview: remove the "osmesa" and "egl" variant
Instead, enforce consistency using the "gl" virtual that allows
only one provider.
* visit: remove osmesa variant
* Disable paraview in the aws-isc stacks
* data-vis-sdk: rework constrains to enforce front-ends
* e4s-power: remove redundant paraview
* Pipelines: update osmesa variants
* trilinos-catalyst-ioss-adapter: make gl a run dependency
Starting from 24.04, armpl-gcc will only have three packages files: dev, rpm, and macOS.
Only one version for gcc is provided, so the changes in the code reflect that the tar provided
for gcc is only one rather than many.
* Remove mesa18 and libosmesa
mesa18 was introduced in #19528 as a way to maintain the old
autotools build of mesa separate from the new meson build.
We could add a second build system to mesa, but since mesa18 has
been deprecated for a long time, we'll just remove it.
libosmesa was used to multiplex the gl provider between mesa18
and mesa, and is thus unecessary. Remove it to reduce complexity
in the graphical stack.
* Remove references to mesa18 and libosmesa
* vtk: rework dependency on gl and osmesa
* memsurfer: rework dependency on vtk
* visit: minimal fix to avoid having both osmesa and glx
* Use hdf5::hdf5 on Windows from Paraview CMake
This patch is already applied on VTK 9 or greater.
* Add comments stating that vtk and paraview patches are the same and should be modified in concert.
This fixes a bug occurring when two root specs need to select
old versions, and these versions have the same penalty in the
optimization. This sometimes caused an older version to be
preferred to a more recent one.
The issue was the omission of `PackageNode` in the optimization
tuple.
* rayleigh: new package
* Update var/spack/repos/builtin/packages/rayleigh/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* split edit into three methods
* add comments to clarify use of configure
* rayleigh: copyright year
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Use correct path separator in get_all_package_diffs for all platforms.
Ensures correct package change computation on Windows when pruning unchanged specs in Gitlab CI
* Update, fix serenity & serenity-libint
Version 1.6.1 of `serenity` is added, some dependencies are made more
explicit, some options are improved or fixed.
The url of `serenity-libint` is fixed. The old url is not reachable
anymore.
* Use upstream patch, modify cmake patch
* Update var/spack/repos/builtin/packages/serenity/package.py
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* heimdall: Astronomy software package. Requires dedisp and psrdada (included as part of this commit)
* Updated packages to align with Spack's style
Minor updates based on wdconinc's comments regarding Spack's style guide
* [@spackbot] updating style on behalf of aweaver1fandm
* Minor edits to fix copyright year and dedisp install
Fixed copyright year to be 2024 instead of 2023
Removed the overridden version of install and created a preinstall function to create the missing lib and inc directories, therefore allowing the default install to run
Here is output from the spack install of heimdall showing successful build with cuda. If necessary I do a spack clean and freshly install it
./spack install heimdall +cuda cuda_arch=86
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libiconv-1.17-enpmbhsi3kztebwmpclpub2afhlbr3gy
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/xz-5.4.1-pte76kujkezxb3laqse3o4sctlbygsaw
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zlib-1.2.13-utlfo5ltxz5v5bckirn5v3amtbxjdvwh
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/bzip2-1.0.8-x6navz7ucgfnb5xq7aelqmgd4zxsz5bs
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libmd-1.0.4-ncomhrodpdul4dm64o6b7426fhmc2u64
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/ncurses-6.4-c77h34rooycbzapxjvc27sg5td5jiwyb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zstd-1.5.5-eporpybumydxveg5rwtfzysrsu4eqzcv
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libffi-3.4.4-vskntokaojclfqxjfzkbyirkeogddbpx
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxcrypt-4.4.33-gtpn32p6mxztul3c3dxzqj7gvcyh555j
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/util-linux-uuid-2.38.1-g322a5peqjaad6gl5q64cdu4qo7kvw6o
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxml2-2.10.3-pza3kz2mtbncbbeim6rejfqkgftnf4rz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/openssl-1.1.1t-adbquvgg4qpc3vq6jynf44qzq3gfwrv5
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/pigz-2.7-e7mxj4ya2u4a6zb4hu64g7docujmkxeb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libbsd-0.11.7-wh4xleivbe7wndiqt5nsehzlfrccnjcg
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/readline-8.2-hhb647bwmbcj7iwpmtetbylninfm5rxf
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/cuda-11.7.1-osue2sx5rv7dgzhsmaemydpwhyribxng
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/tar-1.34-35f5gki2ycxmy5zd7zs5tsvp3xoszxum
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/expat-2.5.0-4gizyhhqklciuyrbyinq2tdggt73gds4
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gdbm-1.23-w3uzihtubj2iwv6es55fis6nt2q5zwlr
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/sqlite-3.40.1-m4ntzvuupsnbtkdfbz7oqpbjdlaffp2a
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/dedisp-1.0-pq6r3jnyxq6jzoygz3fp2e6jc2ojpvap
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gettext-0.21.1-hglzdeadmgkzjb76bmemt6dnulfkrpha
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/python-3.10.10-khu36qq4p2te7jf475ewr2h7egidekfl
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/psrdada-master-by4w6mrpcfnoihlhos2jcfo2roiyagaz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/heimdall-1.0-ohtdnltuhhejysshcert25h6nmuvluqp
* [@spackbot] updating style on behalf of aweaver1fandm
Restrict the hpcrun-fmt.txt patch to :2022. It's fixed in the code
after that, and in recent develop, some code paths have moved causing
this patch to fail.
This fixes an issue where ghcr, gitlab and possibly other container registries paginate tags by default, which violates the OCI spec v1.0, but is common practice (the spec was broken itself). After this commit, you can create build cache indices of > 100 specs on ghcr.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* pythia8: prefer 8.310
* [@spackbot] updating style on behalf of wdconinc
* pythia8: filter_file to remove sed n
* Revert "[@spackbot] updating style on behalf of wdconinc"
This reverts commit e2a3decaffbd3f464d1bd992025e1812df49f088.
* Revert "pythia8: prefer 8.310"
This reverts commit 568cb056b87129085e245d9dbef1732ee1c6c0aa.
* [@spackbot] updating style on behalf of wdconinc
* pythia8: comment for fix
* pythia8: fix style
* pythia8: filter_file with raw string because of escaped pipe
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
The release tarballs are not available anymore which means autoconf, automake and libtool are always needed.
The NVHPC specific patches don't make sense anymore being that the patched files are not distributed in the new tar files.
Add support for Gitlab CI on Windows
This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows
infrastructure).
* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Symlinks on Windows can use longpath prefixes (\\?\); these are fine
in the context of win32 API interactions but break numerous facets of
Spack behavior that rely on string parsing/matching (archiving,
binary distributions, tarball extraction, view regen, etc).
Spack's internal readlink method (llnl.util.symlink.readlink)
gracefully handles this by removing the prefix and otherwise behaving
exactly as os.readlink does, so we should prefer that in all cases.
* py-matplotlib: qualify when to do a post install
Older versions of py-matplotlib don't seem to have some of the
files that the post install step is trying to install.
Looks like the files first appeared in 3.6.0 and later.
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
* Change install paths for older matplotlib
---------
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add maintainer and fix linting
* allow for fewer deps in archive
* use meson for archive packages
* Fix version spec and f-string
* fix blas dependency links
* Add new release to spack
* Fix checksums for latest release
* Update version of iperf3 from 3.6 to 3.16
Spack currently only explicitly has version 3.6 of the iPerf3 package
(out of ESnet / LBNL). This makes the default the latest version of 3.16,
and adds some other versions (found in some Linux distros, for possible
compatibility purposes).
* iperf3: update to 3.17; update 3.6 hash for new url
* protobuf: update hash for patch needed when="@3.4:3.21"
* Revert "protobuf: update hash for patch needed when="@3.4:3.21""
This reverts commit 4d168d0b27.
* initial commit to enable tensorflow-2.14 for spack built ROCm
* fix style errors
* modify hipcc patch
* updates for rocm 6.1
* updates for tf-rocm-enhanced 2.16
* fix styling
* fix styling
* add patch for 2.16
* add patch for 2.16
* Update var/spack/repos/builtin/packages/py-tensorflow/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* update rocm enhanced version names
* changes for rocm-enhanced version name change
* fix styling
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix gromacs installation with SVE. Issue 44062
* [@spackbot] updating style on behalf of paolotricerri
* Remove `neoverse_n2` target
We have removed the neoverse_n2 target as its detection is more involved
compared to neoverse_v*.
Set MPI_C_COMPILER etc. env vars when building with fujitsu-mpi, which
override CMake logic. Without using these variables to explicitly
request the Fujitsu MPI wrappers, the builtin CMake logic is unwilling
to use these wrappers unless the Fujitsu compiler is used, but they
should be used for %clang as well.
This avoids patching CMake module files like in #16864, primarily to
avoid the possibility of altering behavior for specs that do not use
%fj or ^fujitsu-mpi.
* Add openfoam to aws-pcluster-neoverse_v1 stack
* Add more apps to aws-pcluster-x86_64_v4 stack
* Remove WRF while hdf5 cannot build in buildcache at the moment
* Update comment
* Add more apps for aws-pcluster-neoverse_v1 stack
* Remove apps that currently do not build
* Disable those packages that won't build
* Modify syntax such that correct cflags are used
* Changing syntax again to what works with other packages
* Fix overriding packages.yaml entry for gettext
* Use new `prefer` and `require:when` clauses to clarify intent
* Use newer spack version to install intel compiler
This removes the need for patches and makes sure the `prefer` directives in
`package.yaml` are understood.
* `prefer` not strong enough, need to set compilers
* Revert "Use newer spack version to install intel compiler"
This reverts commit ecb25a192c.
Cannot update the spack version to install intel compiler as this changes the
compiler hash but not the version. This leads to incompatible compiler paths. If
we update this spack version in the future make sure the compiler version also updates.
Tested-by: Stephen Sachs <stesachs@amazon.com>
* Remove `prefer` clause as it is not strong enough for our needs
This way we can safely go back to installing the intel compiler with an older
spack version.
* Prefer gcc or oneapi to build gettext
* Pin gettext version compatible with system glibc-headers
* relax gettext version requirement to enable later versions
* oneapi cannot build older gettext version
* octave: use pcre2 for @8:
* Add 'pcre2' variant to octave to control pcre vs. pcre2
* Update var/spack/repos/builtin/packages/octave/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alex Richert <alexander.richert@noaa.gov>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Apparently urllib can throw a range of different exceptions:
1. HTTPError
2. URLError with e.reason set to the actual exception
3. TimeoutError from getresponse, which is not wrapped
* New package: qpdf
* Update var/spack/repos/builtin/packages/qpdf/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Fix format of cmake_args
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* archive: relative links only
Ensure all links written into tarfiles generated from Spack prefixes do not contain symlinks pointing outside the prefix
* binary_distribution: limit extraction to prefix
Ensure files extracted from spackballs are not links pointing outside of the prefix
* Ensure rpaths are properly set on Windows
* hard error on extraction of absolute links
* refactor for non link-modifying approach
* Restore tarball extraction to original impl
* use custom readlink
* cleanup symlink module
* make lstrip
* flux-sched: set the version if the ver file is missing
problem: flux-sched needs a version, it normally gets this from a
release tarball or from git tags, but if using a source archive or a git
clone without tags the version is missing
solution: set the version through cmake based on the version spack sees
when the version file is missing
* Update var/spack/repos/builtin/packages/flux-sched/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Problem: the libsodium download endpoints are not reliable,
they fail multiple times a day for several of our automated
pipelines.
Solution: use the GitHub releases and branches.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
GCC C++ headers like cstdlib use `#include_next <stdlib.h>` to wrap libc
headers. We're using `-isystem` for libc, which puts those headers too
early in the search path. `-idirafter` fixes this so `include_next`
works.
* add darshan-3.4.5 packages, update URLs
* py-setuptools version switches for py-darshan
* more py-darshan test dependencies
* try a conditional importlib_resources dependency
Add the ability to include any number of (potentially nested) concrete environments, e.g.:
```yaml
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
or, from the CLI:
```console
$ spack env create myenv
$ spack -e myenv add python
$ spack -e myenv concretize
$ spack env create --include-concrete myenv included_env
```
The contents of included concrete environments' spack.lock files are
included in the environment's lock file at creation time. Any changes
to included concrete environments are only reflected after the environment
is re-concretized from the re-concretized included environments.
- [x] Concretize included envs
- [x] Save concrete specs in memory by hash
- [x] Add included envs to combined env's lock file
- [x] Add test
- [x] Update documentation
Co-authored-by: Kayla Butler <<butler59@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co
m>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Currently SPACK_COLOR=always is not respected in the build process on
macOS, because the global `_force_color` is re-evaluated in global scope
during module setup, where it is always `None`.
So, move global init bits from main.py to the module itself.
Some specs which were excluded from reuse,
are currently added back to the solve when
we traverse dependencies of other reusable
specs.
This fixes the issue by keeping track of what
we can explicitly reuse.
This commit adds a layer of indirection to improve build isolation with
and without external Python, as well as usability of environment views.
It adds `python-venv` as a dependency to all packages that `extends("python")`,
which has the following advantages:
1. Build isolation: only `PYTHONPATH` is considered in builds, not
user / system packages
2. Stable install layout: fixes the problem on Debian, RHEL and Fedora where
external / system python produces `bin/local` subdirs in Spack install prefixes.
3. Environment views are Python virtual environments (and if you add
`py-pip` things like `pip list` work)
Views work whether they're symlink, hardlink or copy type.
This commit additionally makes `spec["python"].command` return
`spec["python-venv"].command`. The rationale is that packages in repos we do
not own do not pass the underlying python to the build system, which could still
result in incorrectly computed install layouts.
Other attributes like `libs`, `headers` should be on `python` anyways and need no change.
Currently bootstrapping from source fails because clingo requires gnupg
requires clingo.
This commit stops eager bootstrapping. We don't need `patchelf` nor `gnupg`
generally. They're bootstrapped when needed.
This creates shared infrastructure for compiler packages to implement the
detailed search capabilities from the `spack compiler find` command for the
`spack external find` command.
After this commit, `spack compiler find` can be replaced with
`spack external find --tag compiler`, with the exception of mixed toolchains.
A named env cannot contain `.` and `/`.
So when a user runs `spack env create ./here` do not error but treat it
as `spack env create -d ./here`.
Also fix help string of `spack env create`, which seems to have been
copied from `activate` incorrectly.
Since reuse is the default now, `--reuse-deps` can be confusing, as it
technically does not imply roots are fresh.
So add `--fresh-roots`, which is also easier to discover when running
`spack concretize --fre<tab>`
* WIP NWChem with ARMCI-MPI and TCE optional
* rename armci-mpi to armcimpi
* add version for git
* add ARMCI-MPI support to NWChem package
EXTERNAL_ARMCI_PATH needs to be set to the ARMCI-MPI package install prefix.
I could not get it from spec["armcimpi"] but it worked to use the dependent build environment method to export a generic environmental variable to use instead.
* i suppose i can maintain NWChem as well
* check rejects option that dozens of packages use
i do not have time to fight with this nonsense
Run . share/spack/setup-env.sh
==> Error: armcimpi version 'master' has extra arguments: 'branch'
Valid arguments for a url fetcher are:
'url', 'sha256', 'md5', 'sha1', 'sha224', 'sha384', 'sha512', and 'checksum'
Error: Process completed with exit code 1.
* style
* ARMCI-MPI needs depends_on; the rest seems fixed
* fix ARMCI selection per review feedback from @zzzoom
https://github.com/spack/spack/pull/43883#discussion_r1585014147
* address reviewer feedback from @yizeyi18
https://github.com/spack/spack/pull/43883#discussion_r1587228084
elaborate on what +extratce does, in terms of the NWChem build environment variables,
some of which are documented on https://nwchemgit.github.io/TCE.html.
* style
* Update var/spack/repos/builtin/packages/nwchem/package.py
---------
Signed-off-by: Jeff Hammond <jeff.science@gmail.com>
Co-authored-by: Carlos Bederián <4043375+zzzoom@users.noreply.github.com>
* [TAU package] Update with +rocprofv2. Updated some tests.
pdated with +rocprofv2 flag. Only works with rocm-core >= 6.0.0
Can be tested with (Omnia):
spack install tau@master +rocm+rocprofv2 %gcc@11
Needs the last commit in our local repository, can be done by modifying the "git = " line, or waiting until the public one is updated.
In the case that tests cause issues when building TAU, there is the flag:
disable_tests = False
The rocm test is disabled by default, as the PR regarding tests loading dependencies is not solved (PR#43682).
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
* py-tensorflow: fix aarch64 build
* [@spackbot] updating style on behalf of aweits
* patch format
* change patch strategy, actually get the logic correct in
the patch
* only patch 2.16.1 onwards for aarch64
* __CUDACC__ not __NVCC__
* !(defined(__NVCC__) && defined(__CUDACC__))
---------
Co-authored-by: aweits <aweits@users.noreply.github.com>
We recently switched to using the new ReadTheDocs with "addons". That includes its own
analytics, which is nice, but we also want to continue using our GA4 analytics.
Adding GA4 is no longer supported by RTD, so we have to add it manually.
- [x] re-add the gtag to all pages, manually
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
PR #43701 is broken, preventing scorep from being installed. As explained in #43700 the issue is internal to scorep and can not be fixed in the package, at least by the method being attempted here.
Adds a pre-concretization check for the Windows SDK and WGL (Windows
GL) packages as non-buildable externals.
This is a redo of https://github.com/spack/spack/pull/43459, but makes
sure to modify the configuration scope outside of the bootstrap scope:
whichever is highest-precedence in the user's environment at the time
the concretization runs, which should either be an env scope or the
~ scope.
Adds pytest fixture mocking the check for WGL and WSDK as if they were
present.
This PR gives users finer control over which specs are reused during concretization.
The value of the `concretizer:reuse` config option now can take an object with the following properties:
- `roots`: true if reusing roots, false if reusing just dependencies
- `exclude`: list of constraints used to select reusable specs
- `include`: list of constraints used to select reusable specs
- `from`: allows to select the sources of reused specs
### Examples
#### Reuse only specs compiled with GCC
```yaml
concretizer:
reuse:
roots: true
include:
- "%gcc"
```
#### `openmpi` must be used from externals, and it must be the only external used
```yaml
concretizer:
reuse:
roots: true
from:
- type: local
exclude:
- "openmpi"
- type: buildcache
exclude:
- "openmpi"
- type: external
include:
- "openmpi"
```
With v2.10+, ADIOS added a campaign manager. This is auto-enabled
if SQLite3 is found.
Add explicit control for it now and disables it by default, to avoid
picking up system dependencies or bloating by default the ADIOS2
dependencies. Also, not yet fully mature to be used by default:
https://github.com/ornladios/ADIOS2/issues/4148
* pypi build of py-simpy
* [py-simpy] toml explicitly called out
* py-simpy: New version
* py-setuptools-scm: Added new version
* py-setuptools-scm: add url_for_version
Because versions @:7 have an underscore (setuptools_scm) in the URL, and
newer versions have a dash (setuptools-scm).
* py-setuptools-scm: fix flake8 line too long
* py-simpy: Fix hash
---------
Co-authored-by: Sid Pendelberry <sid@rit.edu>
Co-authored-by: Jen Herting <jen@herting.cc>
* PackageStillNeededError: add pkg that needs spec to exception msg
* PackageStillNeededError: f-string with short fmt and hash
* PackageStillNeededError: split long string
The old concretizer creates a cyclic graph when expanding virtuals for
`iconv`, which is a bug. This hack drops glibc and musl as possible
providers for `iconv` in the old concretizer to work around it.
* Added package to build Ollama
* Update package.py
Add license and documentation
* [@spackbot] updating style on behalf of teaguesterling
* We can now use OVERRIDE_GIT_DESCRIBE to set the version in DuckDB
* Update duckdb/package.py
- use f-string
- fix version specs to address inconsistencies
* Update package.py
Fix spec definitions further
* [@spackbot] updating style on behalf of teaguesterling
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The "DOTNET_CLI_TELEMETRY_OPTOUT" environment variable should be defined when using the product, not when installing it (the installation phase is just extract the files anyway).
Also use `~` instead of `-` to check for the variant and fix the second argument for `env.set` which should also be a string.
Add debug log for external detection tests. The debug log
is used to print which test is being executed.
Skip version audit on Windows where appropriate
* Update package.py to osg 1.119 and igtf 1.128
* Remove unnecessary white space
* Add missing comma
* change url to use git and use commit hash from repository instead of ssh256sum of tar.gz
* [@spackbot] updating style on behalf of rahmans1
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
* * Add initial tests-sos package
* Remove failing call of missing setup_compiler_environment from sos
package
* Add several variants for sos package
* [@spackbot] updating style on behalf of jack-morrison
We run `extend spack_flags_list SPACK_LDFLAGS` for `$mode in ld|ccld`.
That's problematic, cause `ccld` needs `-Wl,--flag` whereas `ld` needs
`--flag` directly. Only `-L` and `-l` are common to compiler & linker.
In all build systems `LDFLAGS` is for the compiler not the linker, cause
any linker flag `-x` can be passed as a compiler flag `-Wl,-x`, and there
are many compiler flags that affect the linker invocation, like `-fopenmp`,
`-fuse-ld=`, `-fsanitize=` etc.
So don't pass `LDFLAGS` to the linker directly.
This way users can set `ldflags: -Wl,--allow-shlib-undefined` in compilers.yaml
to work around an issue where the linker tries to resolve the `libcuda.so.1`
stub lib which cannot be located by design in `cuda`.
if [ "${{ needs.prechecks.result }}" == "failure" ] || [ "${{ needs.prechecks.result }}" == "canceled" ]; then
echo "Unit tests failed."
exit 1
else
exit 0
fi
coverage:
needs:[unit-tests, prechecks ]
uses:./.github/workflows/coverage.yml
secrets:inherit
all:
needs:[unit-tests, coverage, bootstrap ]
if:${{ always() }}
runs-on:ubuntu-latest
# See https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#needs-context
steps:
- name:Status summary
run:|
if [ "${{ needs.unit-tests.result }}" == "failure" ] || [ "${{ needs.unit-tests.result }}" == "canceled" ]; then
.. Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
.. Copyright Spack Project Developers. See COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -865,7 +864,7 @@ There are several different ways to use Spack packages once you have
installed them. As you've seen, spack packages are installed into long
paths with hashes, and you need a way to get them into your path. The
easiest way is to use :ref:`spack load <cmd-spack-load>`, which is
described in the next section.
described in this section.
Some more advanced ways to use Spack packages include:
@@ -959,7 +958,86 @@ use ``spack find --loaded``.
You can also use ``spack load --list`` to get the same output, but it
does not have the full set of query options that ``spack find`` offers.
We'll learn more about Spack's spec syntax in the next section.
We'll learn more about Spack's spec syntax in :ref:`a later section <sec-specs>`.
.._extensions:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python packages and virtual environments
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Spack can install a large number of Python packages. Their names are
typically prefixed with ``py-``. Installing and using them is no
different from any other package:
..code-block::console
$ spack install py-numpy
$ spack load py-numpy
$ python3
>>> import numpy
The ``spack load`` command sets the ``PATH`` variable so that the right Python
executable is used, and makes sure that ``numpy`` and its dependencies can be
located in the ``PYTHONPATH``.
Spack is different from other Python package managers in that it installs
every package into its *own* prefix. This is in contrast to ``pip``, which
installs all packages into the same prefix, be it in a virtual environment
or not.
For many users, **virtual environments** are more convenient than repeated
``spack load`` commands, particularly when working with multiple Python
packages. Fortunately Spack supports environments itself, which together
with a view are no different from Python virtual environments.
The recommended way of working with Python extensions such as ``py-numpy``
is through :ref:`Environments <environments>`. The following example creates
a Spack environment with ``numpy`` in the current working directory. It also
puts a filesystem view in ``./view``, which is a more traditional combined
prefix for all packages in the environment.
..code-block::console
$ spack env create --with-view view --dir .
$ spack -e . add py-numpy
$ spack -e . concretize
$ spack -e . install
Now you can activate the environment and start using the packages:
..code-block::console
$ spack env activate .
$ python3
>>> import numpy
The environment view is also a virtual environment, which is useful if you are
sharing the environment with others who are unfamiliar with Spack. They can
either use the Python executable directly:
..code-block::console
$ ./view/bin/python3
>>> import numpy
or use the activation script:
..code-block::console
$ source ./view/bin/activate
$ python3
>>> import numpy
In general, there should not be much difference between ``spack env activate``
and using the virtual environment. The main advantage of ``spack env activate``
is that it knows about more packages than just Python packages, and it may set
additional runtime variables that are not covered by the virtual environment
activation script.
See :ref:`environments` for a more in-depth description of Spack
environments and customizations to views.
.._sec-specs:
@@ -1096,6 +1174,17 @@ unspecified version, but packages can depend on other packages with
could depend on ``mpich@1.2:`` if it can only build with version
``1.2`` or higher of ``mpich``.
..note:: Windows Spec Syntax Caveats
Windows has a few idiosyncrasies when it comes to the Spack spec syntax and the use of certain shells
Spack's spec dependency syntax uses the carat (``^``) character, however this is an escape string in CMD
so it must be escaped with an additional carat (i.e. ``^^``).
CMD also will attempt to interpret strings with ``=`` characters in them. Any spec including this symbol
must double quote the string.
Note: All of these issues are unique to CMD, they can be avoided by using Powershell.
For more context on these caveats see the related issues: `carat <https://github.com/spack/spack/issues/42833>`_ and `equals <https://github.com/spack/spack/issues/43348>`_
Below are more details about the specifiers that you can add to specs.
.._version-specifier:
@@ -1269,6 +1358,10 @@ For example, for the ``stackstart`` variant:
mpileaks stackstart==4# variant will be propagated to dependencies
mpileaks stackstart=4# only mpileaks will have this variant value
Spack also allows variants to be propagated from a package that does
not have that variant.
^^^^^^^^^^^^^^
Compiler Flags
^^^^^^^^^^^^^^
@@ -1354,22 +1447,12 @@ the reserved keywords ``platform``, ``os`` and ``target``:
$ spack install libelf os=ubuntu18.04
$ spack install libelf target=broadwell
or together by using the reserved keyword ``arch``:
..code-block::console
$ spack install libelf arch=cray-CNL10-haswell
Normally users don't have to bother specifying the architecture if they
are installing software for their current host, as in that case the
values will be detected automatically. If you need fine-grained control
over which packages use which targets (or over *all* packages' default
target), see :ref:`package-preferences`.
..admonition:: Cray machines
The situation is a little bit different for Cray machines and a detailed
explanation on how the architecture can be set on them can be found at :ref:`cray-support`
.._support-for-microarchitectures:
@@ -1678,19 +1761,24 @@ Verifying installations
The ``spack verify`` command can be used to verify the validity of
Spack-installed packages any time after installation.
^^^^^^^^^^^^^^^^^^^^^^^^^
``spack verify manifest``
^^^^^^^^^^^^^^^^^^^^^^^^^
At installation time, Spack creates a manifest of every file in the
installation prefix. For links, Spack tracks the mode, ownership, and
destination. For directories, Spack tracks the mode, and
ownership. For files, Spack tracks the mode, ownership, modification
time, hash, and size. The Spack verify command will check, for every
file in each package, whether any of those attributes have changed. It
will also check for newly added files or deleted files from the
installation prefix. Spack can either check all installed packages
time, hash, and size. The ``spack verify manifest`` command will check,
for every file in each package, whether any of those attributes have
changed. It will also check for newly added files or deleted files from
the installation prefix. Spack can either check all installed packages
using the `-a,--all` or accept specs listed on the command line to
verify.
The ``spack verify`` command can also verify for individual files that
they haven't been altered since installation time. If the given file
The ``spack verify manifest`` command can also verify for individual files
that they haven't been altered since installation time. If the given file
is not in a Spack installation prefix, Spack will report that it is
not owned by any package. To check individual files instead of specs,
use the ``-f,--files`` option.
@@ -1705,164 +1793,21 @@ check only local packages (as opposed to those used transparently from
``upstream`` spack instances) and the ``-j,--json`` option to output
machine-readable json data for any errors.
^^^^^^^^^^^^^^^^^^^^^^^^^^
``spack verify libraries``
^^^^^^^^^^^^^^^^^^^^^^^^^^
.._extensions:
The ``spack verify libraries`` command can be used to verify that packages
do not have accidental system dependencies. This command scans the install
prefixes of packages for executables and shared libraries, and resolves
their needed libraries in their RPATHs. When needed libraries cannot be
located, an error is reported. This typically indicates that a package
was linked against a system library, instead of a library provided by
a Spack package.
---------------------------
Extensions & Python support
---------------------------
Spack's installation model assumes that each package will live in its
own install prefix. However, certain packages are typically installed
*within* the directory hierarchy of other packages. For example,
`Python <https://www.python.org>`_ packages are typically installed in the
The first step to contribute new runners is to open an issue in the `spack infrastructure <https://github.com/spack/spack-infrastructure/issues/new?assignees=&labels=runner-registration&projects=&template=runner_registration.yml>`_
project. This will be reported to the spack infrastructure team who will guide users through the process
of registering new runners for Spack CI.
The information needed to register a runner is the motivation for the new resources, a semi-detailed description of
the runner, and finallly the point of contact for maintaining the software on the runner.
The point of contact will then work with the infrastruture team to obtain runner registration token(s) for interacting with
with Spack's GitLab instance. Once the runner is active, this point of contact will also be responsible for updating the
GitLab runner software to keep pace with Spack's Gitlab.
Tagging
~~~~~~~
In the initial stages of runner registration it is important to **exclude** the special tag ``spack``. This will prevent
the new runner(s) from being picked up for production CI jobs while it is configured and evaluated. Once it is determined
that the runner is ready for production use the ``spack`` tag will be added.
Because gitlab has no concept of tag exclustion, runners that provide specialized resource also require specialized tags.
For example, a basic CPU only x86_64 runner may have a tag ``x86_64`` associated with it. However, a runner containing an
CUDA capable GPU may have the tag ``x86_64-cuda`` to denote that it should only be used for packages that will benefit from
a CUDA capable resource.
OIDC
~~~~
Spack runners use OIDC authentication for connecting to the appropriate AWS bucket
which is used for coordinating the communication of binaries between build jobs. In
order to configure OIDC authentication, Spack CI runners use a python script with minimal
dependencies. This script can be configured for runners as seen here using the ``pre_build_script``.
.. Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
.. Copyright Spack Project Developers. See COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
.._environments:
=========================
Environments (spack.yaml)
=========================
=====================================
Environments (spack.yaml, spack.lock)
=====================================
An environment is used to group together a set of specs for the
purpose of building, rebuilding and deploying in a coherent fashion.
Environments provide a number of advantages over the *à la carte*
approach of building and loading individual Spack modules:
An environment is used to group a set of specs intended for some purpose
to be built, rebuilt, and deployed in a coherent fashion. Environments
define aspects of the installation of the software, such as:
#.Environments separate the steps of (a) choosing what to
install, (b) concretizing, and (c) installing. This allows
Environments to remain stable and repeatable, even if Spack packages
are upgraded: specs are only re-concretized when the user
explicitly asks for it. It is even possible to reliably
transport environments between different computers running
different versions of Spack!
#. Environments allow several specs to be built at once; a more robust
solution than ad-hoc scripts making multiple calls to ``spack
install``.
#. An Environment that is built as a whole can be loaded as a whole
into the user environment. An Environment can be built to maintain
a filesystem view of its packages, and the environment can load
that view into the user environment at activation time. Spack can
also generate a script to load all modules related to an
environment.
#.*which* specs to install;
#.*how* those specs are configured; and
#.*where* the concretized software will be installed.
Aggregating this information into an environment for processing has advantages
over the *à la carte* approach of building and loading individual Spack modules.
With environments, you concretize, install, or load (activate) all of the
specs with a single command. Concretization fully configures the specs
and dependencies of the environment in preparation for installing the
software. This is a more robust solution than ad-hoc installation scripts.
And you can share an environment or even re-use it on a different computer.
Environment definitions, especially *how* specs are configured, allow the
software to remain stable and repeatable even when Spack packages are upgraded. Changes are only picked up when the environment is explicitly re-concretized.
Defining *where* specs are installed supports a filesystem view of the
environment. Yet Spack maintains a single installation of the software that
can be re-used across multiple environments.
Activating an environment determines *when* all of the associated (and
installed) specs are loaded so limits the software loaded to those specs
actually needed by the environment. Spack can even generate a script to
load all modules related to an environment.
Other packaging systems also provide environments that are similar in
some ways to Spack environments; for example, `Conda environments
<https://conda.io/docs/user-guide/tasks/manage-environments.html>`_ or
.. Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
.. Copyright Spack Project Developers. See COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -476,9 +475,3 @@ implemented using Python's built-in `sys.path
:py:mod:`spack.repo` module implements a custom `Python importer
<https://docs.python.org/2/library/imp.html>`_.
..warning::
The mechanism for extending packages is not yet extensively tested,
and extending packages across repositories imposes inter-repo
dependencies, which may be hard to manage. Use this feature at your
own risk, but let us know if you have a use case for it.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
