* Trilinos: Add AMD to SuiteSparse TPL list
When Trilinos is built with SuiteSparse support, it should enable AMD as
one of the TPLs. It was previously enabling only UMFPACK. The Xyce
package uses AMD (but not UMFPACK).
* Xyce: Add 7.8 release and various improvements
In addition to adding the latest Xyce release (7.8), all the earlier
releases were deprecated, with the exception of 7.7.
The Trilinos specification was updated to remove unneeded packages,
explicitly enable all needed packages, and specify additional
do-not-build packages.
The serial build is now the default, with MPI still being an option.
I also epanded the explanation for one of the patches; and, finally, I
took the opportunity to update the Xyce description to better match the
current Xyce README description.
Fixes a bug where Spack did not generate module files of non-roots during
spack install with an active environment.
The reason being that Environment.new_installs only contained roots.
This PR:
Drops special casing of automatic module generation in post-install hooks
When `use_view`, compute environment variable modifications like always, and
applies a view projection to them afterwards, like we do for spack env activate.
This ensures we don't have to delay module generation until after the view is
created.
Fixes another bug in use_view where prefixes of dependencies would not be
projected -- previously Spack would only temporarily set the current spec's prefix.
Removes the one and only use of the post_env_write hook (but doesn't drop it to
make this backportable w/o changes)
Previously `std_args` was called on non-roots in a build context, which is redundant, and also leads to issues when `std_args` expects build deps of the `pkg` to be installed.
HPCToolkit `develop` now can optionally be built via Meson. This PR adds the `build_system=(autotools|meson)` variant and splits the build-system-dependent pieces into `AutotoolsBuilder` and `MesonBuilder`. The default is to build with `autotools`.
As of writing, the Meson is simply a wrapper around the original Autotools build, hence the build requires a native file listing install prefixes of all dependencies (which are internally mapped to `--with-{pkg}={prefix}` arguments for `./configure`). This is an unconventional but temporary state of affairs until the build system is fully ported to Meson and conventional dependency acquisition techniques like `pkg-config` and `cmake` are practically available.
* Environments: fix environment config
* Don't change the lockfile manifest path
* Update activate's comments to tie the manifest to the configuration
* Add spec_list override method
* Remove type checking from 'activate' since already have built-in check
* Refactor global methods tied to the manifest to be in its class
* Restore Environment's 'env_subdir_path' and pass its config_stage_dir to EnvironmentManifestFile
* Restore global env_subdir_path for use by Environment and EnvironmentManifestFile
The CUDA target should be specified at build time, otherwise
by default `autodock-gpu` will be built for compute capabilities
52, 60, 61, 70, which may cause errors on unsopported cards.
Currently requirements allow to express "strong preferences" and "conflicts" from
configuration using a convoluted syntax:
```yaml
packages:
zlib-ng:
require:
# conflict on %clang
- one_of: ["%clang", "@:"]
# Strong preference for +shared
- any_of: ["+shared", "@:"]
```
This PR adds syntactic sugar so that the same can be written as:
```yaml
packages:
zlib-ng:
conflict:
- "%clang"
prefer:
- "+shared"
```
Preferences written in this way are "stronger" that the ones documented at:
- https://spack.readthedocs.io/en/latest/packages_yaml.html#package-preferences
`spack install` early exit behavior was sometimes convenient, except
that it had and has bugs:
1. prior bug: we didn't mark env roots of already installed specs as
explicit in the db
2. current bug: `spack install --overwrite` is ignored
So this PR simplifies by letting the installer do its thing even if
everything is supposedly installed.
* Bump up the version for ROCm-6.0.0
* Adding patch files
* Style check failure fix
* Style check fixes
* Style check error fixes
* Patch to remove hipblas client file installation in 6.0
* Patch need to be applied on all 5.7 relases
* 6.0 update for math libs and other packages, new github url etc
* Correct package-audit failures
* Correcting shasum for rocfft patch and limiting patch in rocblas
* Reverting updates in rocprofiler-dev due to ci-gitlab failure
* Fixes for ci-gitlab failure due to disabling hip backward compatibilit
* Adding patch file to Change HIP_PLATFORM from HCC to AMD and NVCC to NVIDIA
* Use the gcnArchName inplace of gcnArch as gcnArch is deprecated from rocm-6.0.0
* Patches to fix magma and blaspp build error with rocm 6.0.0
* Patch for mfem and arborx for rocm 6.0
* Style check error fix
* Correcting style check errors
* Uodating dependent version
* Update for petsc to build with rocm 6.0
Need reverting-operator-mixup-fix-for-slate.patch for rocm 6.0
* Reverting the change in url for 2.7.4-rocm-enhanced
* hip-tensor 6.0.0 update
This commit ensures that CMake packages that also have Python as a build/link dep get a couple defines for the Python path so that CMake's builtin `FindPython3`, `FindPython`, `FindPythonInterp` modules can locate Python correctly.
The main problem with those CMake modules is that they first search for Python versions known at the time of release, meaning that old CMake maybe find older system Python 3.8 even though Python 3.11 comes first in `CMAKE_PREFIX_PATH` and `PATH`.
Package maintainers can opt out of this by overriding the `find_python_hints = False` attribute in the package class.
* Add nglview package
* Use slightly older version
* py-nglview: Correct py-versioneer version
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-nglview: Correct version of py-jupyter-packaging dependency
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add py-versioneer-518 package
* py-versioneer-518: Correct version
* py-nglview: Numpy is needed during build for the tests
* py-nglview: dependency needed for tests
* py-nglview: Correct dependency types
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* e4s ci: use latest intel/hpckit 2024 based image
* use latest container image: ecpe4s/ubuntu22.04-runner-amd64-oneapi-2024.0.0:2023.12.01
* comment out failing specs
* update to use patched container
* remove generalized package preference for intel-oneapi-mkl@2023
* change packages commented out
* VecGeom: new version 1.2.7 and fix URLs
* vecgeom: remove deprecated ancient RC version with incorrect hash
* geant4: remove support for @10.3+vecgeom
The piece of code that is removed in this PR predates environment views.
Spack would symlink build logs in `<env>/.spack-env/logs/*`, but this is
redundant because:
1. Views already add `<prefix>/.spack` (and there's logic there to avoid
clashes)
2. The code was broken anyways: it would only symlink the logs of
environment roots, not their deps, even if they were just built.
If users disable views, I'm pretty sure they're not waiting for
`.spack-env/logs` either. So, imo we can delete this code, and it was
probably overlooked in the past.
For a requirement like
```
packages:
foo:
require:
- "+debug"
```
(not `one_of:`, `any_of:`, or `spec:`)
`spack config change` would ignore the string. This was particularly evident if toggling a variant for a previously unmentioned package:
```
$ spack config change packages:foo:require:+debug
$ spack config change packages:foo:require:~debug
```
This fixes that and adds a test for it.
```
$ curl 'https://thepeg.hepforge.org/downloads/?f=ThePEG-2.3.0.tar.bz2' | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1904k 0 1904k 0 0 1075k 0 --:--:-- 0:00:01 --:--:-- 1074k
ac35979ae89c29608ca92c156a49ff68aace7a5a12a0c92f0a01a833d2d34572 -
```
* Reduce the size on disk for logs
This PR does two things:
1. Store a compressed `spack-build-out.txt.gz`
2. Get rid of phase logs, as they are duplicates of
`spack-build-out.txt`
The logs are not compressed in the stage dir, so on build failure the
workflow for users is no different.
It's just that on install the logs are rarely used, and if needed, users
can easily `gzip -d` or `zgrep` them.
In the case of GCC installs, the compressed logs are <5% of the original
size, which is typically dozens of MBs.
* get rid of "backwards compat" of file names in stage dirs
Sbangs don't exist on Native Windows, and the hook is causing errors
due to the file comparison + behavior of os.rename on Windows. Skip
the hook on Windows.
New patch 7.5.3p5, new bugfix 7.7.2, new minor 7.8.0.
Only possible impact on spack is the potential addition of a variant to select the memory manager in 7.8.0, see [diff](https://github.com/Open-Cascade-SAS/OCCT/compare/V7_7_2...V7_8_0). Not adding a variant at this time.
Like `spack change` for specs in environments, this can e.g. replace `examplespec+debug` with `examplespec~debug` in a `require:` section.
Example behavior for a config like:
```
packages:
foo:
require:
- spec: +debug
```
* `spack config change packages:foo:require:~debug` replaces `+debug` with `~debug`
* `spack config change packages:foo:require:@1.1` adds a requirement to the list
* `spack config change packages:bar:require:~debug` adds a requirement
As observed in #40944, when using `spack config add <path>`, the `path` might
contain keys that are enclosed in quotes.
This was broken in https://github.com/spack/spack/pull/39831, which assumed that
only the value (if present, the final element of the path) would use quotes.
This preserves the primary intended behavior of #39931 (allowing ":" in values when
using `spack config add`) while also allowing quotes on keys.
This has complicated the function `process_config_path`, but:
* It is not used outside of `config.py`
* The docstring has been updated to account for this
* Created an object to formalize the DSL, added a test for that, and
refactored parsing to make use of regular expressions as well.
* Updated the parsing and also updated the `config_path_dsl` test with an explicit check.
At a higher level, split the parsing to check if something is either a key or not:
* in the first case, it is covered by a regex
* in the second, it may be a YAML value, but in that case it would have to be the last
entry of x:y:z, so in that case I attempt to use the YAML handling logic to parse it as such
- Use MakefilePackage and simplified package.py
- Deprecate old versions - they did not build for me with OCaml 4.13.1
that is currently in Spack. Also, the changes from the previous
versions seem to be quite significant.
Spack packages may not have a public download option, and can implement
`download_instr` to inform users how to obtain the artifacts needed to
build. `spack checksum` however did not account for this and would print
out a confusing error message when invoked on such packages ("Could not
find any remote versions").
This PR updates the error message to output the manual download instructions
if `spack checksum` is invoked on a package with `manual_download = True`.
Currently when you repeatedly create a bootstrap mirror that includes
`clingo-bootstrap@spack` you get different tarballs every time.
This is a general problem with mirroring checkouts from version control
as tarballs. I think it's best to create tarballs ourselves, since that way we
have more control over its contents.
This PR ensures normalized tarballs like we do for build caches:
- normalize file permissions (in fact that was already inspired by git, so
should be good)
- normalized file creation/modification time (timestamp 0)
- uid / guid = 0, no usernames
- normalized gzip header
- dir entries are ordered by `(is_dir, name)` where strings are not locale aware ;)
- POSIX says st_mode of symlinks is unspecified, so work around it and
force mode to `0o755`
R embeds an absolute path to the `which` executable in the sources for
`Sys.which`. This gets ultimately stored as serialized byte code in some
custom database format, which uses compression for entries.
As a result, Spack cannot relocate `<prefix which>/bin/which` when
installing from a build cache.
The patch works around this by making R create a symlink to `which` in
its own prefix, have the R sources call that, so that relocation works
again.
See https://github.com/r-devel/r-svn/pull/151
Explicitly requested namespaces are annotated during
the setup phase, and used to retrieve the correct package
class.
An attribute for the namespace has been added for each node.
Currently, a single namespace per package is allowed
during concretization.
* add new versions of py-altair
* fix year
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* reorder dependencies
* remove rc
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add py-nanoplot and py-nanostat
* Add myself as spack package maintainer
* Remove python version requirement
* Remove python dependency
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py: remove python dependency
* Set dependency types
* Update py-nanomath package.py
* Update py-nanoplot package.py
* Update py-nanostat package.py
* Add missing py-python-deprecated dependency
* Make kaleido a source package
* Fix py-nanoget deps
* Kaleido lint
* Nanoget lint
* Nanomath lint
* Nanoplot lint
* Nanostat lint
* Another kaleido lint I missed
* py-nanoplot missed lint
* py-nanostat missed lint
* py-kaleido even more missed lint
* The linter really can't make up its mind
* The linter REALLY can't make up its mind
* Add py-python-deprecated package
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* perl-class-accessor-lvalue: New package
Adds Class::Accessor::Lvalue
* perl-date-utils and deps: New packages
This adds:
- perl-date-utils and its dependencies:
- perl-date-exception
- perl-term-ansicolor-markup
The current `mkdir {{ paths.environment }}` will generate an error if:
* `{{ paths.environment }}` already exists, or
* `{{ paths.environment }}` is nested in non-existing dirs.
Adding `-p` to the command will make this robust to both possibilities.
Set noclobber bash option when writing manifest.
* Add GASNet-EX release 2023.9.0
Add level_zero variant
Deprecate old versions that are no longer supported
Add version enforcement to accelerator variants
* Add UPC++ release 2023.9.0
Deprecate old versions that are no longer supported
Add version enforcement to accelerator variants
* perl-rose-datetime: New package plus updates
- perl-rose-datetime: New package
- perl-datetime: New version, updated dependencies to enable build
time tests and added a runtime test
* Remove copyright line
* New year
* bigdft: convert the use of format strings to fstrings everywhere
* Fix formatting
* reformat with black
* Update compiler flags in bigdft-core
* Revert "Update compiler flags in bigdft-core"
This reverts commit f7524ed784.
* Bump the build cache layout version from 1 to 2
* Version to lists parent directories of the prefix in the tarball too, which is required from some container runtimes
* Move in vs. satisfies to a note and mention special cases of in
* Address feedback: oveoverlap -> intersect
* Re-word the satisfies versus in note.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This fixes an issue where pkg.stage throws because a patch cannot be found,
but the patch is redundant because the spec is reused from a build cache and
will be installed from existing binaries.
Part 4 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ provided_spec: {when_spec, ...} }
to this:
{ when_spec: {provided_spec, ...} }
`make_when_spec()` was being used in the solver, but it has semantics that are specific
to parsing when specs from `package.py`. In particular, it returns `None` when the
`when` spec is `False`, and directives are responsible for ignoring that case and not
adding requirements, deps, etc. when there's an actual `False` passed in from
`package.py`.
In `asp.py`, we know that there won't ever be a raw boolean when spec or constraint, so
we know we can parse them without any of the special boolean handling. However, we
should report where in the file the error happened on error, so this adds some parsing
logic to extract the `mark` from YAML and alert the user where the bad parse is.
- [x] refactor `config.py` so that basic `spack_yaml` mark info is in its own method
- [x] refactor `asp.py` so that it uses the smarter YAML parsing routine
- [x] refactor `asp.py` so that YAML input validation for requirements is done up front
Part 3 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ (requirement_spec, ...): [(when_spec, policy, msg)] }
to this:
{ when_spec: [((requirement_spec, ...), policy, msg), ...] }
Part 2 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ conflict_spec: [(when_spec, msg), ...] }
to this:
{ when_spec: [(conflict_spec, msg), ...] }
Also attempts to consistently name the variables used to iterate over conflict
dictionaries.
Part 1 of making all package metadata indexed by `when` condition. This
will allow us to handle all the dictionaries on `PackageBase` consistently.
Convert the current dependency dictionary structure from this:
{ name: { when_spec: [Dependency ...] } }
to this:
{ when_spec: { name: [Dependency ...] } }
On an M1 mac, this actually shaves 5% off the time it takes to load all
packages, I think because we're able to trade off lookups by spec key
for more lookups by name.
* Boost: add version 1.84.0
* Conflict with 98/03
* Set C++11 as default
Starting with 1.84.0, the minimum required is c++11. It has been a very
long time since 98/03 has been required. It's time to bump the minimum.
* Add OpenMPI 5.0.0/5.0.1 release
* Fix a problem with dlopen syms with 5.0.0
* Crank up lex buffer to 1MB so that Open MPI's compiler wrapper can parse the enormously long lines present in, for example, mpicc-wrapper-data.txt when the spack install is utilizing Spack's path padding feature.
* Disable romio by default for 5.0.0 and beyond owing to problems compiling the romio package when using the Intel OneAPI compiler.
* Patch for addiing cuda lib location in case of non-standard location of libcuda.so
* build accel components as DSOs. It appears from looking at some of the spack CI that it implicitly assumes that Open MPI is built with components as DSOs. The default behavior for Open MPI was changed between the 4.1.x release stream and the 5.0.x release stream changed and this premise is now incorrect.
Turns out that starting with Open MPI 5.0.0 building static
does not work when using a now very important variant, namely cuda.
In older versions of Open MPI the libcuda.so was dlopened at
run time when needed, but now libcuda is linked in to the cuda
components of openmpi directly. This works when using Open MPI's
dynamically loadable component option, but doesn't work now for
a lot of the Spack CI pipelines because they don't include libcuda.so
in LD_LIBRARY_PATH of packages that dont think they are using
cuda themselves.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Co-authored-by: Jack Morrison <jack.morrison@cornelisnetworks.com>
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Needed for #40326, which can changes the iteration order over package dependencies during concretization.
While clingo doesn't have this problem, the original concretizer (which we still use for bootstrapping) can be sensitive to iteration order when evaluating dependency constraints in `when` conditions. This can cause it to ignore conditional dependencies unless the dependencies in the condition are listed first in the package.
The issue was in the way the original concretizer would disconnect specs *every* time `normalize()` ran. When specs were disconnected, `^dependency` constraints wouldn't see the dependency in the dependency condition loop.
We now only only disconnect *all* dependencies at the start of `concretize()` and `normalize()`, and we disconnect any leftover dependents from replaced externals at the *end* of `normalize()`. This trims stale connections while keeping the ones that are needed to trigger dependency conditions.
- [x] refactor `flat_dependencies()` to not disconnect the spec by default.
- [x] `flat_dependencies()` is never called with `copy=True` -- remove the `copy` kwarg.
- [x] disconnect only once at the beginning of `normalize()` or `concretize()`.
- [x] add a test that perturbs dependency iteration order to ensure this doesn't regress.
- [x] disconnect unused dependents at end of `normalize()`
* setup-env: Fix back and forth between two instances
* setup-env.csh: Fix SPACK_ROOT when switch to a different instance
i.e. Always look for the current SPACK_ROOT
* setup-env: Update comments
This adds options to `spack list` that allow you to list only packages from specific
repositories/namespaces, e.g.:
```console
spack list -r builtin
```
only lists packages from the `builtin` repo, while:
```console
spack list -r myrepo -r myrepo2
```
would list packages from `myrepo` and `myrepo2`, but not from `builtin`. Note that you
can use the same argument multiple times.
You can use either `-r` / `--repo` or `-N` / `--namespace`. `-N` is there to match the
corresponding option on `spack find`.
- [x] add `-r` / `--repo` / `-N` / `--namespace` argument
- [x] add test
* Add rust-analyzer as variant to rust build
* Expose cargo module only when +cargo
* rust: add v1.74.0 and v1.75.0 and remove variants in favor of +dev
* [@spackbot] updating style on behalf of alecbcs
* Fix variant typo
---------
Co-authored-by: alecbcs <alecbcs@users.noreply.github.com>
This method is vestigial; the only arg we ever used was `ignore=`, and that was
eliminated in #29317 and #35588.
The `kwargs` field of the extensions dictionary is actually completely unused now. Add a
note for future removal.
Literal compiler config in `test_requires_directive` specifically lists `target:
x86_64`, but it doesn't need to, and the unnecessary target makes the test fail on
non-`x86_64` machines.
- [x] Remove target from config yaml in `test_requires_directive`
* py-torch: set env OpenBLAS_HOME
Because [`FindOpenBLAS.cmake`](https://github.com/pytorch/pytorch/blob/main/cmake/Modules/FindOpenBLAS.cmake) uses a hardcoded list of search paths for includes and libraries, we have to pass the `OpenBLAS_HOME` environment variable.
* py-torch: patch for ${OpenBLAS_HOME}/include/openblas
The context of this patch is unchanged since v0.4.0.
* py-torch: move patch before def patch
* py-torch: also set Atlas_ROOT_DIR and BLIS_HOME
* py-torch: fix openblas patch range to @:2.1
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Xorg pkgs: updated version to current latest
This updates the versions of multiple Xorg packages to their current
latest version. Verified the requirements and version dependencies, and
updated where needed. Modified one homepage (xkbcomp) in the interest of
conformity with the other packages.
Summary of dependency changes:
- libsm:
- depends_on("libice@1.1.0:", when="@1.2.4:")
- libx11:
- depends_on("libxcb@1.11.1:", when="@1.6.4:")
- libxcomposite:
- depends_on("xproto@7.0.22:", when="@0.4.6")
- libxfixes:
- depends_on("fixesproto@5.0:", when="@5")
- depends_on("fixesproto@6.0:", when="@6")
- libxi:
- depends_on("inputproto@2.2.99.1:", when="@1.7:")
- depends_on("inputproto@2.3.99.1:", when="@1.8:")
* xcb-proto, libxcb: new version 1.15
* xorg libs: additional new versions
New minor version upgrades:
- libXcursor (no changed needed)
- libXres
- depends_on("resourceproto@1.0:", when="@1.0")
- depends_on("resourceproto@1.2:", when="@1.2")
* libxpm: ... depends_on ncompress only when 3.5.15
* Xorg libs: add maintainer
* xtrans: new version 1.5.0
* xcb-proto: new version 1.16.0
* libxt: new version 1.3.0
* libxrandr: new version 1.5.4
* libxpm: new versions 3.5.16, 3.5.17
* libxi: new version 1.8.1
* libxft: new version 2.3.8
* libxfixes: new version 6.0.1
* libxcb: new version 1.16
* libx11: new version 1.8.5, 1.8.6, 1.8.7
* libxfixes: comment out problematic fixesproto versions
* libxi: comment out problematic inputproto versions
* libxfixes, libxi: add reference to issue that blocks updates
* patch on variant fftw3; fix for #41577
* a line of doubtful blas/fftw +openmp. Are they needed with nwchem+openmp?
* Update var/spack/repos/builtin/packages/nwchem/package.py
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
---------
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
* gaudi: add a patch for catch2
* Fix indentation
* Add a diff at the end of the path
* gaudi: canonicalize patch url
* gaudi: canonicalize patch url for gitlab diff
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
PAPI CI checks a `spack install` of `papi@master`, and the open range here breaks their
CI with the fix because the patch is no longer needed (see #26784, #27625 for why it's
difficult to avoid this).
The patch issue is going to be fixed in PAPI upstream with whatever release is after
`7.1.0`, so we can restrict the patch to `7.1.0` and avoid this issue.
* shell: fix zsh color formatting for PS1 in environments
The `colorize` function in `llnl.util.tty.color` only applies proper formatting for Bash
ANSI and for console output, but this is not what zsh expects for environment variables.
In particular, when using `zsh`, `spack env activate -p` produces a `PS1` prompt that
looks like this:
```
\[\033[0;92m\][ENVIRONMENT]\[\033[0m\]
```
For zsh the formatting should be:
```
\e[0;92m[ENVIRONMENT]\e0;m
```
- [x] Add a `zsh` option to `colorize()` to enable zsh color formatting
- [x] Add conditional to choose the right `PS1` for `zsh`, `bash`, and `sh`
- [x] Don't use color escapes for `sh`, as they don't print properly
* convert lots of += lines to triple quotes
No online release notes or repository, but the new tarball has the following in `NEWS`:
* ThePEG-2.3.0 release: 2023-12-11
** gcc-12/c++17/c++20 compatibility added
** hepmc3 compatibility added
** rivet interface improved
** inforstructure for dark interaction added
PAPI 7.1.0 unconditionally adds `FFLAGS = -ffree-form` in the sysdetect tests,
regardless of the compiler.
This was added in https://github.com/icl-utk-edu/papi/pull/108 to make a build with
`armflang` work, but it breaks CCE (and our `develop` pipeline).
- [x] Add a patch that fixes both problems
- [x] Patch PAPI when at 7.1.0 or higher
Add a "checked_by" field to the `license()` directive so that we can track who verified
the license for a project. also check the license of 18 or so projects and mark them
checked.
This patch adds license information for about 5,300 packages from automated sources.
The license information was obtained from Alpine Linux and PyPI and processed
using tooling available in https://github.com/boomanaiden154/spack-license-utils.
The license field was added in after all other directives in an automated fashion.
Note that while this license information is probably fairly accurate, it is not
guaranteed to be accurate. In addition some of the license strings from Alpine Linux
might not be valid SPDX license strings. Invalid SPDX identifiers can be picked up
and fixed once we have validation/parsing infrastructure in place for the solver,
and issues can be fixed as they come up.
This adds a few options to `spack gc`.
One to give you a little more control over dependencies:
* `-b` / `--keep-build-dependencies`: By default, `spack gc` considers build dependencies to be "no longer needed" once their dependents are installed. With this option, we'll keep build dependencies of needed installations as well.
And two more to make working with environments easier:
* `-E` / `--except-any-environment`: Garbage collect anything NOT needed by an environment. `spack gc -E` and `spack gc -bE` are now easy ways to get rid of everytihng not used by some environment.
* `-e` / `--except-environment` `ENV`: Instead of considering all environments, garbage collect everything not needed by a *specific* environment. Note that you can use this with `-E` to add directory environments to the list of considered envs, e.g.:
spack gc -E -e /path/to/direnv1 -e /path/to/direnv2 #...
- [x] rework `unused_specs()` method on DB to add options for roots and deptypes
- [x] add `all_hashes()` method on DB
- [x] rework `spack gc` command to add 3 more options
- [x] tests
* Add all versions back to 0.20, add more depends_on (flex, bison, libffi and ccache), add the ability to enable or disable both abc and ccache, abc is enabled by default, ccache is disabled by default
* Fixed style with black
* Removed unused f-string setups
* Fixed style with black (again)
* Add enum34, numdifftools, and updated pyomo packages
* Syntax error
* Apply black style
* Trying to get around Python spec issue
* All SHAs were somehow wrong
* Change enum version
* Change optional dependencies to be on run, not build
* Add Pyomo 6.7.0
* Update SHA and version mismatch
* Remove py-enum34
* Add three new packages to address comments
* Fix linting errors; move casadi to py-casadi
* Update license; add in dependency
* Update setuptools version
* Update class name to python class
* Remove other boielerplate stuff
* Update homepage addresses; update py-casadi
* Add PAPI 7.0.1
* Add comment about skipping PAPI 7.0.0
* Add patch to avoid adding Intel ifort/ifx flag on Cray ftn
* Modify patch to include Cray-specific flags
* Adjust recipe to always apply patch for 7.0.1
* Expand Cray compiler checks in patch
* Forgot to update recipe
* Adjust recipe so it looks for hipcc in the correct path
* Revert "Adjust recipe so it looks for hipcc in the correct path"
This reverts commit 0db3df4fe2.
* Patch HIP_PATH to work with Spack-built HIP
* Patch LDFLAGS with llvm-amdgpu path
* Forgot the depends_on line
* libomptarget only builds with clang
* Try a self-consistent build of llvm-amdgpu
* Try making llvm-amdgpu depend on llvm for llvmoffloadarch library
* Update prereq to use rocm-openmp-extras instead
* Refactor llvm-amdgpu to use a version dict
* Fix typo
* Hack to exclude older versions without matching rocm-openmp-extras
* Add PAPI 7.1.0
* Revert changes to llvm-amdgpu
* Fix PAPI 7.1.0 checksum
For now, this only includes packages that I personally maintain.
Notable removals:
* Anaconda 2
* Catalyst
* Ancient numpy/scipy
* Ancient PyTorch
* Ancient Bazel/TF
To work properly, Spack requires a few directories from its repository to be added to
`sys.path`. Previously these were buried in `spack_installable.main.main()`, but it's
sometimes useful to get the paths separately, e.g., if you want to set up your own
functioning spack environment.
With this change, adding the paths is much simpler:
```python
import spack_installable
sys.path[:0] = get_spack_sys_paths(spack_prefix)
```
- [x] Add `get_spack_sys_paths()` method with extra paths in order.
- [x] Refactor `spack_installable.main.main()` to use it.
With an improper/incomplete/broken installation of Clingo, it can be
importable but not have any of the expected attributes
Improve error reporting in this case
* Restore PackageBase class, and modify only ASP
This prevents a noticeable slowdown in concretization
due to the number of directives involved.
* Fix issue with 'clang' being preferred to 'gcc',
due to runtime version weights
* Constraints on runtimes are declared by compilers
The declaration of available runtime versions, and of
their compatibility constraints are in the associated
compiler class.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
The gcc-runtime package adds a separate node for gcc's dynamic runtime
libraries.
This should help with:
1. binary caches where rpaths for compiler support libs cannot be
relocated because the compiler is missing on the target system
2. creating "minimal" container images
The package is versioned like `gcc` (in principle it could be
unversioned, but Spack doesn't always guarantee not mixing compilers)
If you are calling Spack from the python API, you might have written something like this
before #41529:
```
find = SpackCommand("find")
find('--format={name}', 'saxpy@1.0.0', '+rocm', 'amdgpu_target="gfx90a"')
```
But with the breaking change in #41529, you should write:
```
find = SpackCommand("find")
find('--format={name}', 'gromacs', '+rocm', 'amdgpu_target=gfx90a')
```
Note that we don't need quotes in Python strings, and that this is what would come in
via argv if you typed a quoted variant on the CLI.
The error messages for strings like this are not great -- you get something like this:
```
==> No package matches the query: gromacs+rocm amdgpu_target="gfx90a"
```
Which doesn't indicate that the issue might be your quoting. This is because we were
simply outputting the argv we got, instead of using spec.format() to output the error
message. This PR fixes such errors to use `spec.format()` and to look like this:
```
==> No package matches the query: gromacs+rocm amdgpu_target='"gfx90a"'
```
So users should have an easier time understanding that Spack considers the variant value
to contain quotes here.
- [x] update ConstraintAction to store parsed Specs
- [x] refactor commands to display formatted parsed Specs instead of raw input
Users expect that changes to the externals sections in packages.yaml config apply immediately, but reuse concretization caused this not to be the case. With this commit, the concretizer is only allowed to reuse externals previously imported from config if identical config exists.
This PR adds a flag `--tag/-t` to `buildcache push`, which you can use like
```
$ spack mirror add my-oci-registry oci://example.com/hello/world
$ spack -e my_env buildcache push --base-image ubuntu:22.04 --tag my_custom_tag my-oci-registry
```
and lets users ship a full, installed environment as a minimal container image where each image layer is one Spack package, on top of a base image of choice. The image can then be used as
```
$ docker run -it --rm example.com/hello/world:my_custom_tag
```
Apart from environments, users can also pick arbitrary installed spec from their database, for instance:
```
$ spack buildcache push --base-image ubuntu:22.04 --tag some_specs my-oci-registry gcc@12 cmake
$ docker run -it --rm example.com/hello/world:some_specs
```
It has many advantages over `spack containerize`:
1. No external tools required (`docker`, `buildah`, ...)
2. Creates images from locally installed Spack packages (No need to rebuild inside `docker build`, where troubleshooting build failures is notoriously hard)
3. No need for multistage builds (Spack just tarballs existing installations of runtime deps)
4. Reduced storage size / composability: when pushing multiple environments with common specs, container image layers are shared.
5. Automatic build cache: later `spack install` of the env elsewhere speeds up since the containerized environment is a build cache
These packages were written before the "requires" directive,
and so they are conflicting with all compilers but Fujitsu
to express they _require_ `%fj`
* Ensure a spack libproj is used instead of a system libproj when libproj < 8.
spack/spack/issues/41299
* Fix style as per ci-bot
* Fix style as per ci-bot
* Ensure 3.5:3.8.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add trim function to `Spec` and `--ignore` option to 'spack diff'
Allows user to compare two specs while ignoring the sub-DAG of a particular dependency, e.g.
spack diff --ignore=mpi --ignore=zlib trilinos/abcdef trilinos/fedcba
to focus on differences closer to the root of the software stack
Sometimes env variables computed in `setup_run_environment` depend on tests
w.r.t. files in `spec.prefix`, but Spack temporarily projects `spec.prefix` to
the view.
This is problematic for two reasons:
1. Some packages iterate over `<prefix>/bin`: they expect only the current
package's executables, but find all linked in the view, leading to false
positives.
2. Some packages test for `os.path.islink(...)`, which is always true in a view
`gcc` is an example that does both.
This PR lets Spack compute the environment modifications using the original
prefix, and projects to the view afterwards
Currently, a virtual spec is composed of just a name and a version. When a virtual spec contains other components, such as variants, Spack won't emit warnings or errors but will silently drop them - which is unexpected by users.
This adds three new versions in the 31.* series. Release notes of 31.0.0 at https://github.com/acts-project/acts/releases/tag/v31.0.0. No changes to the CMakeLists.txt files that need addressing in the package recipe.
The only new feature I'm a bit concerned about is https://github.com/acts-project/acts/pull/2626, which replaces testing for C++20 concepts support by the feature-testing macro `__cpp_concepts`, which is also a C++20 feature. So technically we now should require `cxxstd=20` even though Acts itself still allows (and defaults to) 17. Judging by https://en.cppreference.com/w/cpp/compiler_support/20, the support for feature-testing macros was added very early by most compilers.
This PR changes the default behavior of `spack config get` and `spack config blame`
to print a flattened version of the entire spack configuration, including any active
environment, if the commands are invoked with no section arguments.
The new behavior is used in Gitlab CI to help debug CI configuration, but it can also
be useful when asking for more information in issues, or when simply debugging Spack.
Convert the 'develop' section of an environment to a dedicated configuration section.
This means for example that instead of having to define `develop` specs in the
`spack.yaml`, the environment can `include:` another `develop.yaml` configuration
which specifies which specs should be developed in the environment.
This change is not expected to be disruptive given that existing environment `spack.yaml`
files will conform to the new schema.
(Update 11/28/2023) I have implemented the `develop`/`undevelop` commands in terms
of more-generic modification functions added to the `config` module: `change_or_add`
and `update_all`. It is assumed that the semantics added here (described in 11/18 update)
would be desirable to extend to other config update actions (e.g. adding compilers,
changing package requirements, adding mirrors).
(Update 11/18/2023) I have updated this such that `spack develop`, and
`spack undevelop` to potentially modify all writable scopes, like
https://github.com/spack/spack/pull/41147. https://github.com/spack/spack/pull/35307
will be useful for modifying included scopes, but generally speaking specifying a
`--scope` will not be required for `spack develop`: `spack develop` will add new
develop specs to whatever scope already has develop specs defined, or to the
highest-priority writable scope (which should be the env scope).
TODOs:
- [x] If you `spack undevelop` a package which is mentioned at multiple layers of
configuration, then currently this would only modify one of them. That's not
technically a new issue (has always existed for configuration modification), but
may be confusing to users when presented via an interface other than `spack config set`
- [x] Need to add (or confirm) the ability to modify individual config files by providing
a path (rather than using a scope identifier as a key to retrieve associated config).
- [x] `spack develop` adds new develop specs to the scope that defines them
(potentially skipping higher priority scopes to e.g. augment included scope files)
---------
Co-authored-by: scheibelp <scheibelp@users.noreply.github.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* py-plum-dispatch: add new package
* Update var/spack/repos/builtin/packages/py-plum-dispatch/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-htgettoken: use os.environ, avoid AttributeError
This avoids the following error:
```
Warning: could not load runtime environment due to AttributeError: 'EnvironmentModifications' object has no attribute 'get'
```
* py-htgettoken: allow for undefined variables
* py-htgettoken: use dict get()
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* mpifileutils: add DAOS variant
* mpifileutils: Add daos dep when +daos
Add dependency on DAOS when +daos
Pass DAOS prefix to ensure correct DAOS is found by during configuration
* Change in to satisfies for boolean variants
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* perl-datetime-format-strptime: New package
Adds package:
- perl-datetime-format-strptime
And adds these because they are test dependencies:
- perl-test-file-sharedir
- perl-test2-plugin-nowarnings
- perl-test2-suite
And modifies these to enable build time tests:
- perl-b-hooks-endofscope
- perl-class-singleton
- perl-datetime-locale
- perl-datetime-timezone
- perl-file-sharedir
- perl-namespace-autoclean
- perl-namespace-clean
- perl-params-validationcompiler
- perl-specio
* Add myself as maintainer
* add new cpp compiler version
* empty ftn for 2023.2.3
* OLD ftn in 2023.2.3 version
* tolerate missing fortran compiler
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* geant4: new version 11.2.0
* geant4: depends_on geant4-data@11.2:
* geant4-data: new version 11.2.0
* g4abla: new version 3.3
* g4emlow: new version 8.4
* g4incl: new version 1.1
* geant4: depends_on vecgeom@1.2.6:
* geant4: depends_on qt@5.9: when @11.2: +qt
* vecgeom: new version 1.2.6
message: "If you are referencing Spack in a publication, please cite the paper below."
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
abstract: >-
Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools.
Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible.
However, managing many configurations is difficult because the configuration space is combinatorial in size.
We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages and dependencies.
It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment.
We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.
Large HPC centers spend considerable time supporting software for thousands of users, but the
complexity of HPC software is quickly outpacing the capabilities of existing software management
tools. Scientific applications require specific versions of compilers, MPI, and other dependency
libraries, so using a single, standard software stack is infeasible. However, managing many
configurations is difficult because the configuration space is combinatorial in size. We
introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages
and dependencies. It allows any number of builds to coexist on the same system, and it ensures
that installed packages can find their dependencies, regardless of the environment. We show
through real-world use cases that Spack supports diverse and demanding applications, bringing
order to HPC software chaos.
preferred-citation:
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
type: conference-paper
@@ -71,7 +75,7 @@ preferred-citation:
type: doi
value: 10.1145/2807591.2807623
- description: "The DOE Document Release Number of the work"
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
