* VecGeom: new version 1.2.7 and fix URLs
* vecgeom: remove deprecated ancient RC version with incorrect hash
* geant4: remove support for @10.3+vecgeom
The piece of code that is removed in this PR predates environment views.
Spack would symlink build logs in `<env>/.spack-env/logs/*`, but this is
redundant because:
1. Views already add `<prefix>/.spack` (and there's logic there to avoid
clashes)
2. The code was broken anyways: it would only symlink the logs of
environment roots, not their deps, even if they were just built.
If users disable views, I'm pretty sure they're not waiting for
`.spack-env/logs` either. So, imo we can delete this code, and it was
probably overlooked in the past.
For a requirement like
```
packages:
foo:
require:
- "+debug"
```
(not `one_of:`, `any_of:`, or `spec:`)
`spack config change` would ignore the string. This was particularly evident if toggling a variant for a previously unmentioned package:
```
$ spack config change packages:foo:require:+debug
$ spack config change packages:foo:require:~debug
```
This fixes that and adds a test for it.
```
$ curl 'https://thepeg.hepforge.org/downloads/?f=ThePEG-2.3.0.tar.bz2' | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1904k 0 1904k 0 0 1075k 0 --:--:-- 0:00:01 --:--:-- 1074k
ac35979ae89c29608ca92c156a49ff68aace7a5a12a0c92f0a01a833d2d34572 -
```
* Reduce the size on disk for logs
This PR does two things:
1. Store a compressed `spack-build-out.txt.gz`
2. Get rid of phase logs, as they are duplicates of
`spack-build-out.txt`
The logs are not compressed in the stage dir, so on build failure the
workflow for users is no different.
It's just that on install the logs are rarely used, and if needed, users
can easily `gzip -d` or `zgrep` them.
In the case of GCC installs, the compressed logs are <5% of the original
size, which is typically dozens of MBs.
* get rid of "backwards compat" of file names in stage dirs
Sbangs don't exist on Native Windows, and the hook is causing errors
due to the file comparison + behavior of os.rename on Windows. Skip
the hook on Windows.
New patch 7.5.3p5, new bugfix 7.7.2, new minor 7.8.0.
Only possible impact on spack is the potential addition of a variant to select the memory manager in 7.8.0, see [diff](https://github.com/Open-Cascade-SAS/OCCT/compare/V7_7_2...V7_8_0). Not adding a variant at this time.
Like `spack change` for specs in environments, this can e.g. replace `examplespec+debug` with `examplespec~debug` in a `require:` section.
Example behavior for a config like:
```
packages:
foo:
require:
- spec: +debug
```
* `spack config change packages:foo:require:~debug` replaces `+debug` with `~debug`
* `spack config change packages:foo:require:@1.1` adds a requirement to the list
* `spack config change packages:bar:require:~debug` adds a requirement
As observed in #40944, when using `spack config add <path>`, the `path` might
contain keys that are enclosed in quotes.
This was broken in https://github.com/spack/spack/pull/39831, which assumed that
only the value (if present, the final element of the path) would use quotes.
This preserves the primary intended behavior of #39931 (allowing ":" in values when
using `spack config add`) while also allowing quotes on keys.
This has complicated the function `process_config_path`, but:
* It is not used outside of `config.py`
* The docstring has been updated to account for this
* Created an object to formalize the DSL, added a test for that, and
refactored parsing to make use of regular expressions as well.
* Updated the parsing and also updated the `config_path_dsl` test with an explicit check.
At a higher level, split the parsing to check if something is either a key or not:
* in the first case, it is covered by a regex
* in the second, it may be a YAML value, but in that case it would have to be the last
entry of x:y:z, so in that case I attempt to use the YAML handling logic to parse it as such
- Use MakefilePackage and simplified package.py
- Deprecate old versions - they did not build for me with OCaml 4.13.1
that is currently in Spack. Also, the changes from the previous
versions seem to be quite significant.
Spack packages may not have a public download option, and can implement
`download_instr` to inform users how to obtain the artifacts needed to
build. `spack checksum` however did not account for this and would print
out a confusing error message when invoked on such packages ("Could not
find any remote versions").
This PR updates the error message to output the manual download instructions
if `spack checksum` is invoked on a package with `manual_download = True`.
Currently when you repeatedly create a bootstrap mirror that includes
`clingo-bootstrap@spack` you get different tarballs every time.
This is a general problem with mirroring checkouts from version control
as tarballs. I think it's best to create tarballs ourselves, since that way we
have more control over its contents.
This PR ensures normalized tarballs like we do for build caches:
- normalize file permissions (in fact that was already inspired by git, so
should be good)
- normalized file creation/modification time (timestamp 0)
- uid / guid = 0, no usernames
- normalized gzip header
- dir entries are ordered by `(is_dir, name)` where strings are not locale aware ;)
- POSIX says st_mode of symlinks is unspecified, so work around it and
force mode to `0o755`
R embeds an absolute path to the `which` executable in the sources for
`Sys.which`. This gets ultimately stored as serialized byte code in some
custom database format, which uses compression for entries.
As a result, Spack cannot relocate `<prefix which>/bin/which` when
installing from a build cache.
The patch works around this by making R create a symlink to `which` in
its own prefix, have the R sources call that, so that relocation works
again.
See https://github.com/r-devel/r-svn/pull/151
Explicitly requested namespaces are annotated during
the setup phase, and used to retrieve the correct package
class.
An attribute for the namespace has been added for each node.
Currently, a single namespace per package is allowed
during concretization.
* add new versions of py-altair
* fix year
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* reorder dependencies
* remove rc
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add py-nanoplot and py-nanostat
* Add myself as spack package maintainer
* Remove python version requirement
* Remove python dependency
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py: remove python dependency
* Set dependency types
* Update py-nanomath package.py
* Update py-nanoplot package.py
* Update py-nanostat package.py
* Add missing py-python-deprecated dependency
* Make kaleido a source package
* Fix py-nanoget deps
* Kaleido lint
* Nanoget lint
* Nanomath lint
* Nanoplot lint
* Nanostat lint
* Another kaleido lint I missed
* py-nanoplot missed lint
* py-nanostat missed lint
* py-kaleido even more missed lint
* The linter really can't make up its mind
* The linter REALLY can't make up its mind
* Add py-python-deprecated package
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* perl-class-accessor-lvalue: New package
Adds Class::Accessor::Lvalue
* perl-date-utils and deps: New packages
This adds:
- perl-date-utils and its dependencies:
- perl-date-exception
- perl-term-ansicolor-markup
The current `mkdir {{ paths.environment }}` will generate an error if:
* `{{ paths.environment }}` already exists, or
* `{{ paths.environment }}` is nested in non-existing dirs.
Adding `-p` to the command will make this robust to both possibilities.
Set noclobber bash option when writing manifest.
* Add GASNet-EX release 2023.9.0
Add level_zero variant
Deprecate old versions that are no longer supported
Add version enforcement to accelerator variants
* Add UPC++ release 2023.9.0
Deprecate old versions that are no longer supported
Add version enforcement to accelerator variants
* perl-rose-datetime: New package plus updates
- perl-rose-datetime: New package
- perl-datetime: New version, updated dependencies to enable build
time tests and added a runtime test
* Remove copyright line
* New year
* bigdft: convert the use of format strings to fstrings everywhere
* Fix formatting
* reformat with black
* Update compiler flags in bigdft-core
* Revert "Update compiler flags in bigdft-core"
This reverts commit f7524ed784.
* Bump the build cache layout version from 1 to 2
* Version to lists parent directories of the prefix in the tarball too, which is required from some container runtimes
* Move in vs. satisfies to a note and mention special cases of in
* Address feedback: oveoverlap -> intersect
* Re-word the satisfies versus in note.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This fixes an issue where pkg.stage throws because a patch cannot be found,
but the patch is redundant because the spec is reused from a build cache and
will be installed from existing binaries.
Part 4 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ provided_spec: {when_spec, ...} }
to this:
{ when_spec: {provided_spec, ...} }
`make_when_spec()` was being used in the solver, but it has semantics that are specific
to parsing when specs from `package.py`. In particular, it returns `None` when the
`when` spec is `False`, and directives are responsible for ignoring that case and not
adding requirements, deps, etc. when there's an actual `False` passed in from
`package.py`.
In `asp.py`, we know that there won't ever be a raw boolean when spec or constraint, so
we know we can parse them without any of the special boolean handling. However, we
should report where in the file the error happened on error, so this adds some parsing
logic to extract the `mark` from YAML and alert the user where the bad parse is.
- [x] refactor `config.py` so that basic `spack_yaml` mark info is in its own method
- [x] refactor `asp.py` so that it uses the smarter YAML parsing routine
- [x] refactor `asp.py` so that YAML input validation for requirements is done up front
Part 3 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ (requirement_spec, ...): [(when_spec, policy, msg)] }
to this:
{ when_spec: [((requirement_spec, ...), policy, msg), ...] }
Part 2 of reworking all package metadata to key by `when` conditions.
Changes conflict dictionary structure from this:
{ conflict_spec: [(when_spec, msg), ...] }
to this:
{ when_spec: [(conflict_spec, msg), ...] }
Also attempts to consistently name the variables used to iterate over conflict
dictionaries.
Part 1 of making all package metadata indexed by `when` condition. This
will allow us to handle all the dictionaries on `PackageBase` consistently.
Convert the current dependency dictionary structure from this:
{ name: { when_spec: [Dependency ...] } }
to this:
{ when_spec: { name: [Dependency ...] } }
On an M1 mac, this actually shaves 5% off the time it takes to load all
packages, I think because we're able to trade off lookups by spec key
for more lookups by name.
* Boost: add version 1.84.0
* Conflict with 98/03
* Set C++11 as default
Starting with 1.84.0, the minimum required is c++11. It has been a very
long time since 98/03 has been required. It's time to bump the minimum.
* Add OpenMPI 5.0.0/5.0.1 release
* Fix a problem with dlopen syms with 5.0.0
* Crank up lex buffer to 1MB so that Open MPI's compiler wrapper can parse the enormously long lines present in, for example, mpicc-wrapper-data.txt when the spack install is utilizing Spack's path padding feature.
* Disable romio by default for 5.0.0 and beyond owing to problems compiling the romio package when using the Intel OneAPI compiler.
* Patch for addiing cuda lib location in case of non-standard location of libcuda.so
* build accel components as DSOs. It appears from looking at some of the spack CI that it implicitly assumes that Open MPI is built with components as DSOs. The default behavior for Open MPI was changed between the 4.1.x release stream and the 5.0.x release stream changed and this premise is now incorrect.
Turns out that starting with Open MPI 5.0.0 building static
does not work when using a now very important variant, namely cuda.
In older versions of Open MPI the libcuda.so was dlopened at
run time when needed, but now libcuda is linked in to the cuda
components of openmpi directly. This works when using Open MPI's
dynamically loadable component option, but doesn't work now for
a lot of the Spack CI pipelines because they don't include libcuda.so
in LD_LIBRARY_PATH of packages that dont think they are using
cuda themselves.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Co-authored-by: Jack Morrison <jack.morrison@cornelisnetworks.com>
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Needed for #40326, which can changes the iteration order over package dependencies during concretization.
While clingo doesn't have this problem, the original concretizer (which we still use for bootstrapping) can be sensitive to iteration order when evaluating dependency constraints in `when` conditions. This can cause it to ignore conditional dependencies unless the dependencies in the condition are listed first in the package.
The issue was in the way the original concretizer would disconnect specs *every* time `normalize()` ran. When specs were disconnected, `^dependency` constraints wouldn't see the dependency in the dependency condition loop.
We now only only disconnect *all* dependencies at the start of `concretize()` and `normalize()`, and we disconnect any leftover dependents from replaced externals at the *end* of `normalize()`. This trims stale connections while keeping the ones that are needed to trigger dependency conditions.
- [x] refactor `flat_dependencies()` to not disconnect the spec by default.
- [x] `flat_dependencies()` is never called with `copy=True` -- remove the `copy` kwarg.
- [x] disconnect only once at the beginning of `normalize()` or `concretize()`.
- [x] add a test that perturbs dependency iteration order to ensure this doesn't regress.
- [x] disconnect unused dependents at end of `normalize()`
* setup-env: Fix back and forth between two instances
* setup-env.csh: Fix SPACK_ROOT when switch to a different instance
i.e. Always look for the current SPACK_ROOT
* setup-env: Update comments
This adds options to `spack list` that allow you to list only packages from specific
repositories/namespaces, e.g.:
```console
spack list -r builtin
```
only lists packages from the `builtin` repo, while:
```console
spack list -r myrepo -r myrepo2
```
would list packages from `myrepo` and `myrepo2`, but not from `builtin`. Note that you
can use the same argument multiple times.
You can use either `-r` / `--repo` or `-N` / `--namespace`. `-N` is there to match the
corresponding option on `spack find`.
- [x] add `-r` / `--repo` / `-N` / `--namespace` argument
- [x] add test
* Add rust-analyzer as variant to rust build
* Expose cargo module only when +cargo
* rust: add v1.74.0 and v1.75.0 and remove variants in favor of +dev
* [@spackbot] updating style on behalf of alecbcs
* Fix variant typo
---------
Co-authored-by: alecbcs <alecbcs@users.noreply.github.com>
This method is vestigial; the only arg we ever used was `ignore=`, and that was
eliminated in #29317 and #35588.
The `kwargs` field of the extensions dictionary is actually completely unused now. Add a
note for future removal.
Literal compiler config in `test_requires_directive` specifically lists `target:
x86_64`, but it doesn't need to, and the unnecessary target makes the test fail on
non-`x86_64` machines.
- [x] Remove target from config yaml in `test_requires_directive`
* py-torch: set env OpenBLAS_HOME
Because [`FindOpenBLAS.cmake`](https://github.com/pytorch/pytorch/blob/main/cmake/Modules/FindOpenBLAS.cmake) uses a hardcoded list of search paths for includes and libraries, we have to pass the `OpenBLAS_HOME` environment variable.
* py-torch: patch for ${OpenBLAS_HOME}/include/openblas
The context of this patch is unchanged since v0.4.0.
* py-torch: move patch before def patch
* py-torch: also set Atlas_ROOT_DIR and BLIS_HOME
* py-torch: fix openblas patch range to @:2.1
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Xorg pkgs: updated version to current latest
This updates the versions of multiple Xorg packages to their current
latest version. Verified the requirements and version dependencies, and
updated where needed. Modified one homepage (xkbcomp) in the interest of
conformity with the other packages.
Summary of dependency changes:
- libsm:
- depends_on("libice@1.1.0:", when="@1.2.4:")
- libx11:
- depends_on("libxcb@1.11.1:", when="@1.6.4:")
- libxcomposite:
- depends_on("xproto@7.0.22:", when="@0.4.6")
- libxfixes:
- depends_on("fixesproto@5.0:", when="@5")
- depends_on("fixesproto@6.0:", when="@6")
- libxi:
- depends_on("inputproto@2.2.99.1:", when="@1.7:")
- depends_on("inputproto@2.3.99.1:", when="@1.8:")
* xcb-proto, libxcb: new version 1.15
* xorg libs: additional new versions
New minor version upgrades:
- libXcursor (no changed needed)
- libXres
- depends_on("resourceproto@1.0:", when="@1.0")
- depends_on("resourceproto@1.2:", when="@1.2")
* libxpm: ... depends_on ncompress only when 3.5.15
* Xorg libs: add maintainer
* xtrans: new version 1.5.0
* xcb-proto: new version 1.16.0
* libxt: new version 1.3.0
* libxrandr: new version 1.5.4
* libxpm: new versions 3.5.16, 3.5.17
* libxi: new version 1.8.1
* libxft: new version 2.3.8
* libxfixes: new version 6.0.1
* libxcb: new version 1.16
* libx11: new version 1.8.5, 1.8.6, 1.8.7
* libxfixes: comment out problematic fixesproto versions
* libxi: comment out problematic inputproto versions
* libxfixes, libxi: add reference to issue that blocks updates
* patch on variant fftw3; fix for #41577
* a line of doubtful blas/fftw +openmp. Are they needed with nwchem+openmp?
* Update var/spack/repos/builtin/packages/nwchem/package.py
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
---------
Co-authored-by: Rocco Meli <r.meli@bluemail.ch>
* gaudi: add a patch for catch2
* Fix indentation
* Add a diff at the end of the path
* gaudi: canonicalize patch url
* gaudi: canonicalize patch url for gitlab diff
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
PAPI CI checks a `spack install` of `papi@master`, and the open range here breaks their
CI with the fix because the patch is no longer needed (see #26784, #27625 for why it's
difficult to avoid this).
The patch issue is going to be fixed in PAPI upstream with whatever release is after
`7.1.0`, so we can restrict the patch to `7.1.0` and avoid this issue.
* shell: fix zsh color formatting for PS1 in environments
The `colorize` function in `llnl.util.tty.color` only applies proper formatting for Bash
ANSI and for console output, but this is not what zsh expects for environment variables.
In particular, when using `zsh`, `spack env activate -p` produces a `PS1` prompt that
looks like this:
```
\[\033[0;92m\][ENVIRONMENT]\[\033[0m\]
```
For zsh the formatting should be:
```
\e[0;92m[ENVIRONMENT]\e0;m
```
- [x] Add a `zsh` option to `colorize()` to enable zsh color formatting
- [x] Add conditional to choose the right `PS1` for `zsh`, `bash`, and `sh`
- [x] Don't use color escapes for `sh`, as they don't print properly
* convert lots of += lines to triple quotes
No online release notes or repository, but the new tarball has the following in `NEWS`:
* ThePEG-2.3.0 release: 2023-12-11
** gcc-12/c++17/c++20 compatibility added
** hepmc3 compatibility added
** rivet interface improved
** inforstructure for dark interaction added
PAPI 7.1.0 unconditionally adds `FFLAGS = -ffree-form` in the sysdetect tests,
regardless of the compiler.
This was added in https://github.com/icl-utk-edu/papi/pull/108 to make a build with
`armflang` work, but it breaks CCE (and our `develop` pipeline).
- [x] Add a patch that fixes both problems
- [x] Patch PAPI when at 7.1.0 or higher
Add a "checked_by" field to the `license()` directive so that we can track who verified
the license for a project. also check the license of 18 or so projects and mark them
checked.
This patch adds license information for about 5,300 packages from automated sources.
The license information was obtained from Alpine Linux and PyPI and processed
using tooling available in https://github.com/boomanaiden154/spack-license-utils.
The license field was added in after all other directives in an automated fashion.
Note that while this license information is probably fairly accurate, it is not
guaranteed to be accurate. In addition some of the license strings from Alpine Linux
might not be valid SPDX license strings. Invalid SPDX identifiers can be picked up
and fixed once we have validation/parsing infrastructure in place for the solver,
and issues can be fixed as they come up.
This adds a few options to `spack gc`.
One to give you a little more control over dependencies:
* `-b` / `--keep-build-dependencies`: By default, `spack gc` considers build dependencies to be "no longer needed" once their dependents are installed. With this option, we'll keep build dependencies of needed installations as well.
And two more to make working with environments easier:
* `-E` / `--except-any-environment`: Garbage collect anything NOT needed by an environment. `spack gc -E` and `spack gc -bE` are now easy ways to get rid of everytihng not used by some environment.
* `-e` / `--except-environment` `ENV`: Instead of considering all environments, garbage collect everything not needed by a *specific* environment. Note that you can use this with `-E` to add directory environments to the list of considered envs, e.g.:
spack gc -E -e /path/to/direnv1 -e /path/to/direnv2 #...
- [x] rework `unused_specs()` method on DB to add options for roots and deptypes
- [x] add `all_hashes()` method on DB
- [x] rework `spack gc` command to add 3 more options
- [x] tests
* Add all versions back to 0.20, add more depends_on (flex, bison, libffi and ccache), add the ability to enable or disable both abc and ccache, abc is enabled by default, ccache is disabled by default
* Fixed style with black
* Removed unused f-string setups
* Fixed style with black (again)
* Add enum34, numdifftools, and updated pyomo packages
* Syntax error
* Apply black style
* Trying to get around Python spec issue
* All SHAs were somehow wrong
* Change enum version
* Change optional dependencies to be on run, not build
* Add Pyomo 6.7.0
* Update SHA and version mismatch
* Remove py-enum34
* Add three new packages to address comments
* Fix linting errors; move casadi to py-casadi
* Update license; add in dependency
* Update setuptools version
* Update class name to python class
* Remove other boielerplate stuff
* Update homepage addresses; update py-casadi
* Add PAPI 7.0.1
* Add comment about skipping PAPI 7.0.0
* Add patch to avoid adding Intel ifort/ifx flag on Cray ftn
* Modify patch to include Cray-specific flags
* Adjust recipe to always apply patch for 7.0.1
* Expand Cray compiler checks in patch
* Forgot to update recipe
* Adjust recipe so it looks for hipcc in the correct path
* Revert "Adjust recipe so it looks for hipcc in the correct path"
This reverts commit 0db3df4fe2.
* Patch HIP_PATH to work with Spack-built HIP
* Patch LDFLAGS with llvm-amdgpu path
* Forgot the depends_on line
* libomptarget only builds with clang
* Try a self-consistent build of llvm-amdgpu
* Try making llvm-amdgpu depend on llvm for llvmoffloadarch library
* Update prereq to use rocm-openmp-extras instead
* Refactor llvm-amdgpu to use a version dict
* Fix typo
* Hack to exclude older versions without matching rocm-openmp-extras
* Add PAPI 7.1.0
* Revert changes to llvm-amdgpu
* Fix PAPI 7.1.0 checksum
For now, this only includes packages that I personally maintain.
Notable removals:
* Anaconda 2
* Catalyst
* Ancient numpy/scipy
* Ancient PyTorch
* Ancient Bazel/TF
To work properly, Spack requires a few directories from its repository to be added to
`sys.path`. Previously these were buried in `spack_installable.main.main()`, but it's
sometimes useful to get the paths separately, e.g., if you want to set up your own
functioning spack environment.
With this change, adding the paths is much simpler:
```python
import spack_installable
sys.path[:0] = get_spack_sys_paths(spack_prefix)
```
- [x] Add `get_spack_sys_paths()` method with extra paths in order.
- [x] Refactor `spack_installable.main.main()` to use it.
With an improper/incomplete/broken installation of Clingo, it can be
importable but not have any of the expected attributes
Improve error reporting in this case
* Restore PackageBase class, and modify only ASP
This prevents a noticeable slowdown in concretization
due to the number of directives involved.
* Fix issue with 'clang' being preferred to 'gcc',
due to runtime version weights
* Constraints on runtimes are declared by compilers
The declaration of available runtime versions, and of
their compatibility constraints are in the associated
compiler class.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
The gcc-runtime package adds a separate node for gcc's dynamic runtime
libraries.
This should help with:
1. binary caches where rpaths for compiler support libs cannot be
relocated because the compiler is missing on the target system
2. creating "minimal" container images
The package is versioned like `gcc` (in principle it could be
unversioned, but Spack doesn't always guarantee not mixing compilers)
If you are calling Spack from the python API, you might have written something like this
before #41529:
```
find = SpackCommand("find")
find('--format={name}', 'saxpy@1.0.0', '+rocm', 'amdgpu_target="gfx90a"')
```
But with the breaking change in #41529, you should write:
```
find = SpackCommand("find")
find('--format={name}', 'gromacs', '+rocm', 'amdgpu_target=gfx90a')
```
Note that we don't need quotes in Python strings, and that this is what would come in
via argv if you typed a quoted variant on the CLI.
The error messages for strings like this are not great -- you get something like this:
```
==> No package matches the query: gromacs+rocm amdgpu_target="gfx90a"
```
Which doesn't indicate that the issue might be your quoting. This is because we were
simply outputting the argv we got, instead of using spec.format() to output the error
message. This PR fixes such errors to use `spec.format()` and to look like this:
```
==> No package matches the query: gromacs+rocm amdgpu_target='"gfx90a"'
```
So users should have an easier time understanding that Spack considers the variant value
to contain quotes here.
- [x] update ConstraintAction to store parsed Specs
- [x] refactor commands to display formatted parsed Specs instead of raw input
Users expect that changes to the externals sections in packages.yaml config apply immediately, but reuse concretization caused this not to be the case. With this commit, the concretizer is only allowed to reuse externals previously imported from config if identical config exists.
This PR adds a flag `--tag/-t` to `buildcache push`, which you can use like
```
$ spack mirror add my-oci-registry oci://example.com/hello/world
$ spack -e my_env buildcache push --base-image ubuntu:22.04 --tag my_custom_tag my-oci-registry
```
and lets users ship a full, installed environment as a minimal container image where each image layer is one Spack package, on top of a base image of choice. The image can then be used as
```
$ docker run -it --rm example.com/hello/world:my_custom_tag
```
Apart from environments, users can also pick arbitrary installed spec from their database, for instance:
```
$ spack buildcache push --base-image ubuntu:22.04 --tag some_specs my-oci-registry gcc@12 cmake
$ docker run -it --rm example.com/hello/world:some_specs
```
It has many advantages over `spack containerize`:
1. No external tools required (`docker`, `buildah`, ...)
2. Creates images from locally installed Spack packages (No need to rebuild inside `docker build`, where troubleshooting build failures is notoriously hard)
3. No need for multistage builds (Spack just tarballs existing installations of runtime deps)
4. Reduced storage size / composability: when pushing multiple environments with common specs, container image layers are shared.
5. Automatic build cache: later `spack install` of the env elsewhere speeds up since the containerized environment is a build cache
These packages were written before the "requires" directive,
and so they are conflicting with all compilers but Fujitsu
to express they _require_ `%fj`
* Ensure a spack libproj is used instead of a system libproj when libproj < 8.
spack/spack/issues/41299
* Fix style as per ci-bot
* Fix style as per ci-bot
* Ensure 3.5:3.8.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add trim function to `Spec` and `--ignore` option to 'spack diff'
Allows user to compare two specs while ignoring the sub-DAG of a particular dependency, e.g.
spack diff --ignore=mpi --ignore=zlib trilinos/abcdef trilinos/fedcba
to focus on differences closer to the root of the software stack
Sometimes env variables computed in `setup_run_environment` depend on tests
w.r.t. files in `spec.prefix`, but Spack temporarily projects `spec.prefix` to
the view.
This is problematic for two reasons:
1. Some packages iterate over `<prefix>/bin`: they expect only the current
package's executables, but find all linked in the view, leading to false
positives.
2. Some packages test for `os.path.islink(...)`, which is always true in a view
`gcc` is an example that does both.
This PR lets Spack compute the environment modifications using the original
prefix, and projects to the view afterwards
Currently, a virtual spec is composed of just a name and a version. When a virtual spec contains other components, such as variants, Spack won't emit warnings or errors but will silently drop them - which is unexpected by users.
This adds three new versions in the 31.* series. Release notes of 31.0.0 at https://github.com/acts-project/acts/releases/tag/v31.0.0. No changes to the CMakeLists.txt files that need addressing in the package recipe.
The only new feature I'm a bit concerned about is https://github.com/acts-project/acts/pull/2626, which replaces testing for C++20 concepts support by the feature-testing macro `__cpp_concepts`, which is also a C++20 feature. So technically we now should require `cxxstd=20` even though Acts itself still allows (and defaults to) 17. Judging by https://en.cppreference.com/w/cpp/compiler_support/20, the support for feature-testing macros was added very early by most compilers.
This PR changes the default behavior of `spack config get` and `spack config blame`
to print a flattened version of the entire spack configuration, including any active
environment, if the commands are invoked with no section arguments.
The new behavior is used in Gitlab CI to help debug CI configuration, but it can also
be useful when asking for more information in issues, or when simply debugging Spack.
Convert the 'develop' section of an environment to a dedicated configuration section.
This means for example that instead of having to define `develop` specs in the
`spack.yaml`, the environment can `include:` another `develop.yaml` configuration
which specifies which specs should be developed in the environment.
This change is not expected to be disruptive given that existing environment `spack.yaml`
files will conform to the new schema.
(Update 11/28/2023) I have implemented the `develop`/`undevelop` commands in terms
of more-generic modification functions added to the `config` module: `change_or_add`
and `update_all`. It is assumed that the semantics added here (described in 11/18 update)
would be desirable to extend to other config update actions (e.g. adding compilers,
changing package requirements, adding mirrors).
(Update 11/18/2023) I have updated this such that `spack develop`, and
`spack undevelop` to potentially modify all writable scopes, like
https://github.com/spack/spack/pull/41147. https://github.com/spack/spack/pull/35307
will be useful for modifying included scopes, but generally speaking specifying a
`--scope` will not be required for `spack develop`: `spack develop` will add new
develop specs to whatever scope already has develop specs defined, or to the
highest-priority writable scope (which should be the env scope).
TODOs:
- [x] If you `spack undevelop` a package which is mentioned at multiple layers of
configuration, then currently this would only modify one of them. That's not
technically a new issue (has always existed for configuration modification), but
may be confusing to users when presented via an interface other than `spack config set`
- [x] Need to add (or confirm) the ability to modify individual config files by providing
a path (rather than using a scope identifier as a key to retrieve associated config).
- [x] `spack develop` adds new develop specs to the scope that defines them
(potentially skipping higher priority scopes to e.g. augment included scope files)
---------
Co-authored-by: scheibelp <scheibelp@users.noreply.github.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* py-plum-dispatch: add new package
* Update var/spack/repos/builtin/packages/py-plum-dispatch/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-htgettoken: use os.environ, avoid AttributeError
This avoids the following error:
```
Warning: could not load runtime environment due to AttributeError: 'EnvironmentModifications' object has no attribute 'get'
```
* py-htgettoken: allow for undefined variables
* py-htgettoken: use dict get()
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* mpifileutils: add DAOS variant
* mpifileutils: Add daos dep when +daos
Add dependency on DAOS when +daos
Pass DAOS prefix to ensure correct DAOS is found by during configuration
* Change in to satisfies for boolean variants
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* perl-datetime-format-strptime: New package
Adds package:
- perl-datetime-format-strptime
And adds these because they are test dependencies:
- perl-test-file-sharedir
- perl-test2-plugin-nowarnings
- perl-test2-suite
And modifies these to enable build time tests:
- perl-b-hooks-endofscope
- perl-class-singleton
- perl-datetime-locale
- perl-datetime-timezone
- perl-file-sharedir
- perl-namespace-autoclean
- perl-namespace-clean
- perl-params-validationcompiler
- perl-specio
* Add myself as maintainer
* add new cpp compiler version
* empty ftn for 2023.2.3
* OLD ftn in 2023.2.3 version
* tolerate missing fortran compiler
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* geant4: new version 11.2.0
* geant4: depends_on geant4-data@11.2:
* geant4-data: new version 11.2.0
* g4abla: new version 3.3
* g4emlow: new version 8.4
* g4incl: new version 1.1
* geant4: depends_on vecgeom@1.2.6:
* geant4: depends_on qt@5.9: when @11.2: +qt
* vecgeom: new version 1.2.6
This PR does several things:
- [x] Allow any character to appear in the quoted values of variants and flags.
- [x] Allow easier passing of quoted flags on the command line, e.g. `cflags="-O2 -g"`.
- [x] Handle quoting better in spec output, using single quotes around double
quotes and vice versa.
- [x] Disallow spaces around `=` and `==` when parsing variants and flags.
## Motivation
This PR is motivated by the issues above and by ORNL's
[tips for launching at scale on Frontier](https://docs.olcf.ornl.gov/systems/frontier_user_guide.html#tips-for-launching-at-scale).
ORNL recommends using `sbcast --send-libs` to broadcast executables and their
libraries to compute nodes when running large jobs (e.g., 80k ranks). For an
executable named `exe`, `sbcast --send-libs` stores the needed libraries in a
directory alongside the executable called `exe_libs`. ORNL recommends pointing
`LD_LIBRARY_PATH` at that directory so that `exe` will find the local libraries and
not overwhelm the filesystem.
There are other ways to mitigate this problem:
* You could build with `RUNPATH` using `spack config add config:shared_linking:type:runpath`,
which would make `LD_LIBRARY_PATH` take precedence over Spack's `RUNPATHs`.
I don't recommend this one because `RUNPATH` can cause many other things to go wrong.
* You could use `spack config add config:shared_linking:bind:true`, added in #31948, which
will greatly reduce the filesystem load for large jobs by pointing `DT_NEEDED` entries in
ELF *directly* at the needed `.so` files instead of relying on `RPATH` search via soname.
I have not experimented with this at 80,000 ranks, but it should help quite a bit.
* You could use [Spindle](https://github.com/hpc/Spindle) (as LLNL does on its machines)
which should transparently fix this without any changes to your executable and without
any need to use `sbcast` or other tools.
But we want to support the `sbcast` use case as well.
## `sbcast` and Spack
Spack's `RPATHs` break the `sbcast` fix because they're considered with higher precedence
than `LD_LIBRARY_PATH`. So Spack applications will still end up hitting the shared filesystem
when searching for libraries. We can avoid this by injecting some `ldflags` in to the build, e.g.,
if were were going to launch, say, `LAMMPS` at scale, we could add another `RPATH`
specifically for use with `sbcast`:
spack install lammps ldflags='-Wl,-rpath=$ORIGIN/lmp_libs'
This will put the `lmp_libs` directory alongside `LAMMPS`'s `lmp` executable first in the
`RPATH`, so it will be searched before any directories on the shared filesystem.
## Issues with quoting
Before this PR, the command above would've errored out for two reasons:
1. `$` wasn't an allowed character in our spec parser.
2. You would've had to double quote the flags to get them to pass through correctly:
spack install lammps ldflags='"-Wl,-rpath=$ORIGIN/lmp_libs"'
This is ugly and I don't think many users will easily figure it out. The behavior was added in
#29282, and it improved parsing of specs passed as a single string, e.g.:
spack install 'lammps ldflags="-Wl,-rpath=$ORIGIN/lmp_libs"'
but a lot of users are naturally going to try to quote arguments *directly* on the command
line, without quoting their entire spec. #29282 used a heuristic to detect unquoted flags
and warn the user, but the warning could be confusing. In particular, if you wrote
`cflags="-O2 -g"` on the command line, it would break the flags up, warn, and tell you
that you could fix the issue by writing `cflags="-O2 -g"` even though you just wrote
that. It's telling you to *quote* that value, but the user has to know to double quote.
## New heuristic for quoted arguments from the CLI
There are only two places where we allow arbitrary quoted strings in specs: flags and
variant values, so this PR adds a simpler heuristic to the CLI parser: if an argument in
`sys.argv` starts with `name=...`, then we assume the whole argument is quoted.
This means you can write:
spack install bzip2 cflags="-O2 -g"
directly on the command line, without multiple levels of quoting. This also works:
spack install 'bzip2 cflags="-O2 -g"'
The only place where this heuristic runs into ambiguity is if you attempt to pass
anonymous specs that start with `name=...` as one large string. e.g., this will be
interpreted as one large flag value:
spack find 'cflags="-O2 -g" ~bar +baz'
This sets `cflags` to `"-O2 -g" ~bar +baz`, which is likely not what you wanted. You
can fix this easily by either removing the quotes:
spack find cflags="-O2 -g" ~bar +baz
Or by adding a space at the start, which has the same effect:
spack find ' cflags="-O2 -g" ~bar +baz'
You may wonder why we don't just look for quotes inside of flag arguments, and the
reason is that you *might* want them there. If you are passing arguments like:
spack install zlib cppflags="-D DEBUG_MSG1='quick fox' -D DEBUG_MSG2='lazy dog'"
You *need* the quotes there. So we've opted for one potentially confusing, but easily
fixed outcome vs. limiting what you can put in your quoted strings.
## Quotes in formatted spec output
In addition to being more lenient about characters accepted in quoted strings, this PR fixes
up spec formatting a bit. We now format quoted strings in specs with single quotes, unless
the string has a single quote in it, in which case we JSON-escape the string (i.e., we add
`\` before `"` and `\`).
zlib cflags='-D FOO="bar"'
zlib cflags="-D FOO='bar'"
zlib cflags="-D FOO='bar' BAR=\"baz\""
* adding necessary headers, to fix https://github.com/spack/spack/issues/41398
* deleting something imported by accident
* [@spackbot] updating style on behalf of yizeyi18
* undo commit 7688fed according to suggestion from @msimberg
* patching camp@:2022.10.1 for compatibility with gcc-13
* adding the patch
* fixing paths in the patch
* [@spackbot] updating style on behalf of yizeyi18
* Update camp patch using LLNL/camp@05e1c35
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
* changing patch name
---------
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
`setup_dependent_package` is not a build phase, it should just set
globals for a package.
It's called during setup of runtime environment of packages, and there
have been reports of it actually failing due to a read only file system
(not sure under what exact conditions that is possible).
This license annotation is currently invalid as it specifies a URL
rather than an SPDX expression. Remove it for now until we have a
consensus on how to represent this case.
* New package Model Coupling Toolkit (MCT)
* Remove ~mpi variant from mct, build is not working correctly
* Remove boilerplate stuff from var/spack/repos/builtin/packages/mct/package.py
MySQL was performing a core API call to `Spec.flat_dependencies`
when setting up the build environment. This function is an
implementation detail of the old concretizer, where multiple nodes
from the same package are not allowed.
This PR uses a more idiomatic way to check if "python" is
in the DAG.
For reference, see #11356 to check why the call was introduced.
* [protobuf] New versions, explicit cxxstd variant
* New versions 3.15.8, 3.25.0, 3.25.1.
* New explicit variant `cxxstd` with support for older Protobuf
versions.
* Support testing.
* Use Protobuf's `protobuf_BUILD_SHARED_LIBS` instead of
`BUILD_SHARED_LIBS`.
* Support building with LLVM/Clang's `libc++`.
* Address audit issue
* Variant default does not honor `when` clause
* Use `self.spec.satisfies()` instead of `with when()`
* Fix silliness; improve consistency
* Today was apparently a go-back-to-bed day
* initial commit for rocm-5.7.0 and 5.7.1 releases
* bump up ther version for 5.7.0 and 5.7.1 releases
* update recipes to support 5.7.0 and 5.7.1 releases
* bump up the version for ROCm 5.7.0 and ROCm-5.7.1 releases
* bump up the version for composable-kernel amd miopen-hip
* fix style errors
* fix style errors in hip etc
* renaming composable-kernel recipe
* changes for composable_kernel
* Revert "renaming composable-kernel recipe"
This reverts commit 0cf6c6debf.
* Revert "changes for composable_kernel"
This reverts commit 05272a10a7.
* bump up the version for hiprand
* using the checksum for hiprand-5.7.1
* bump up the version for 5.7.0 and 5.7.1 releases
* fix style errors
* fix merge conflicts with the develop.
* temp workaround for the error seen with rocm-5.7.0 when trying
to generate the dependency file for runtime/legion/legion_redop.cu
* fix build issue(work around) with legion
* add patch for migraphx package to turn off ck
* update to hip recipe
* fix hip-path detection inside llvm clang driver
* update llvm-amdgpu and rocm-validation-suite recipes
* fix style errors
* bump up the version for amdsmi for rocm-5.7.0 release
* add support for gfx941,gfx942 for rocm-5.7.0 release onwards
* revert changes to rocm.py file
* added gfx941 and gfx942 to rocm.py and add the gfx942 to kokkos and new checksum
the new version seem to support gfx942
* bump up the version for rccl for 5.7.1
* update the patch for rocm-openmp-extras for 5.7.0
* update mivisionx recipe for 5.7.0 release
* add new dependencies for rocfft tests
* port the fix for avx build, the start address of values_ buffer in KernelParameters is not
correct as it is computed based on 16-byte alignment
* set HIP_PATH=ROCM_PATH for 5.7.0 onwards
* address review comments
* revert adding xnack- and xnack+ to gfx940,gfx941,gfx942 as the prechecks were failing
* Add `signed` property to mirror config
* make unsigned a tri-state: true/false overrides mirror config, none takes mirror config
* test commands
* Document this
* add a test
The new feq-parse version includes fixes for ifort and ifx compilers
Additionally, evaluation of parser objects with multidimension arrays is
now supported.
* samtools/htslib add latest version
* Given I work at the same institute as the authors I think it fair I am willing to review changes, if it's complex I can ask them over tea.
---------
Co-authored-by: James Beal <jb23@sanger.ac.uk>
* New variants:
- `tmvz-cpu`
- `tmvz-gpu`
- `tmvz-pymva`
- `tmvz-sofie`
* Improve X-related dependencies.
* Improve TMVA-related dependencies with more specificity.
* Patch possible missing standard header include in Eve7.
* Patch Protobuf handling to support new Protobuf-provided CMake config
files required to handle transitive `abseil-cpp` dependence.
* Add missing terminal newline to `webgui` patch to remove patch
warning.
* Handle deprecated/removed build options.
* geant4/geant4-data: add builtin_clhep variant and v10.0.4.
* geant4: revert addition of builtin_clhep variant.
* geant4: fix vecgeom variant only being available for v10.3 and above.
Fix filer_compiler_wrapper for cases where the compiler returned in None, this happens on some installed gcc systems that do not have fortran built into them as standard, e.g. gcc@11.4.0 on ubuntu 22.04
Before (hard to read, doesn't fit on small terminals):
:
```console
-I, --install-status show install status of packages
packages can be: installed [+], missing and needed by an installed package [-], installed in an upstream instance [^], or not installed (no annotation)
```
After (fits in 80 columns):
```console
-I, --install-status show install status of packages
[+] installed [^] installed in an upstream
- not installed [-] missing dep of installed package
```
* Fix cdash reporter time stamps (#38818).
The cdash reporter is created before packages are installed so save the
starttime then instead of the endtime.
* Use endtime instead of starttime for the endtime of update
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Participation in the venerable Spack google group has dwindled, though we still have
540+ subscribers there. I've made the mailing list announcement-only, and I've given
a few maintainers posting privileges.
This PR adds some notes to the README indicating that the mailing list is only for
announcements.
* Ensure that additional environment variables are set when a module
file is generated.
* Fixed the detection of the opal_prefix / MPI_ROOT field to use ompi_info.
---------
Co-authored-by: Greg Becker <becker33@llnl.gov>
* depend_on python
There is an ill-named variant "python" that enables the pytrilinos1
variant. This made it through our testing but broke on our actual
CI test machines.
* adjust "python" variant based on Trilinos version
For Trilinos <= 14, enable PyTrilinos(1). For later versions
of Trilinos, enable PyTrilinos2.
We still support directly enabling PyTrilinos2 via the "pytrilinos2"
variant.
* remove pytrilinos2 variant
* correct depends_on constraints
- we don't have a fallback if make is not installed
- we assume file system locking works
- we don't verify that make is gnu make (bootstrapping fails on FreeBSD as a result)
- there are some weird race conditions in writing spack.yaml on concurrent spack install
- the view is updated after every package install instead of post environment install.
Forbid nested dependencies in depends_on declarations, by running an audit in CI.
Fix the packages not passing the new audit:
- amd-aocl
- exago
- palace
- shapemapper
- xsdk-examples
ginkgo: add a commit sha to v1.5.0.glu_experimental
* New variants:
- `tmvz-cpu`
- `tmvz-gpu`
- `tmvz-pymva`
- `tmvz-sofie`
* Improve X-related dependencies.
* Improve TMVA-related dependencies with more specificity.
* Patch possible missing standard header include in Eve7.
* Patch Protobuf handling to support new Protobuf-provided CMake config
files required to handle transitive `abseil-cpp` dependence.
* Add missing terminal newline to `webgui` patch to remove patch
warning.
* Handle deprecated/removed build options.
* Handle unwanted system paths in various `PATH`-like environment
variables.
The condition on swig can be interpreted as "true if true,
false if false" and gives clingo the option to add swig
or not.
If not other optimization criteria break the tie, then
the concretization is non-deterministic.
* added external libxc/elpa choice
* fixed formatting issues and 1 unused variant found by reviewer
* try to fix a string formatting issue
* try to fix some other string formatting issues
* fixed 1 flake8 style issue
* use explicit fftw-api@3
* Add `url_list` to facilitate finding new versions.
* `cxxstd` is not meaningful when `@:2.99.99` as it was a header-only
package before v3.
* Support C++20/23, remove C++14 support.
* Add @greenc-FNAL to maintainers.
* Add CMake arguments to support testing, build of extras and examples.
* Only build tests for proj package if required
Even if tests are not explictly required to be built, proj build them
anyway and tries to download Google Test.
* proj: fix name of test activation flag
* proj: Always set test activation flag
* proj: Patch test activation logic for versions 5.x
* py-python-pptx: new package
* py-python-pptx: use pil instead of pillow, remove version constraint on python
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* py-tldextract: new package
* py-tldextract: add version 5.1.1
* py-tldextract: fix version constraint on py-setuptools-scm
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* Add a new version of ruff
* Add a comment about where the dependency can be found
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
* Update py-werkzeug version dependency for py-graphene-tornado@2.6.1
* Add note on diverging version requirements for py-werkzeug in py-graphene-tornado
* legion: correct cuda dependency for cr version
* Update var/spack/repos/builtin/packages/legion/package.py
---------
Co-authored-by: Davis Herring <herring@lanl.gov>
* py-gidgethub: add new package
* Add main branch version and scope flit/flit-core dependency
* Update var/spack/repos/builtin/packages/py-gidgethub/package.py
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* Add optional dependencies as variants of package
* Add git url for main version
* Fix variant and dependency ordering
---------
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* gidgetlab: add new package
* Convert both cachetools and aiohttp to optional deps with variants
* Fix forgotten variant conditional on cachetools dependency
* Add git url and main version for dev workflows
* Fix variant and dependency ordering
* Remove cachetools variant and merge dependency with aiohttp variant
* grep WM_PROJECT_VERSION from etc/bashrc
This fixes the problem when building from a manually checked out repo
which might have a different version wrt the one defined in the spack
package (e.g. anything later than 5.0 is known as 5.x by the build
system)
* patch applies to just 5.0, in newer versions it is already addressed
In `5.20171030` there's a commit
c66fba323c
very similar (almost identical) to what the patch `50-etc.patch` does.
So the patch should not be applied to other than `5.0` otherwise it errors.
References:
- https://github.com/OpenFOAM/OpenFOAM-5.x/commits/20171030/etc/bashrc
- 197d9d3bf2/etc/bashrc (L45-L47)
This was missed while backporting the new `spack info` command from #40326.
Variants should be sorted by name when invoking `spack info --variants-by-name`.
Intel made an incompatible change in XED in 2023.08.21 that breaks
hpctoolkit (at run time). Hpctoolkit develop can adapt (soon will),
but older versions must use xed :2023.07.09.
* Updating the LBANN, Hydrogen, and DiHydrogen recipes for both new
variants and to make sure that RPATHs are properly setup.
Co-authored-by: bvanessen <bvanessen@users.noreply.github.com>
Deprecating intel package, which contains intel classic compilers. This package has not been updated in 3 years. Please use intel-oneapi-compilers instead.
* gipaw.x installed by cmake if version >= 5c4a4ce.
gipaw.x will only be installed with cmake if the qe-gipaw version
is >= 5c4a4ce. Currently, QE source uses the older f5823521 one.
Here a patch to the submodule_commit_hash_records to use a newer
qe-gipaw version.
* Update package.py
* Delete var/spack/repos/builtin/packages/quantum-espresso/gipaw-eccee44.patch
* Update package.py
* Restoring gipaw-eccee44 patch
* Update package.py
* Add fox variant in quantum-espresso
* Fix an issue introduced in #36484. Patches are 7.1 only.
* Change plugin handling.
* formatting.
* Typo correction
* Refine conflict
---------
Co-authored-by: S. Alexis Paz <alexis.paz@gmail.com>
* Add EGL support to ParaView and Glew
add a package for egl that provides GL but also adds
EGL libs and headers for projects that need them
Fix a header problem with the opengl package
Format files using black
* better description for egl variant description
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* better check/setup of non egl variant dependencies
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* Add biddisco as maintainer
* Fix unused var style warning
* Add egl conflicts for other gl providers
---------
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* py-pyglet: version bump
* py-pyglet: use zip instead of whl, update dependencies
* py-pyglet: 2.0.9 and 2.0.10 zips should be downloaded from github
* py-pyglet: style
* py-pyglet: use virtual packages in dependencies
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* py-pyglet: doesn't depend on py-future any more
* py-pyglet: remove glx dependency
* py-pyglet: back to the pypi zipfiles with patch instead
---------
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* Add initial version of verible to spack
* Update to use explict url path for each release, as the release tagh includes extra data, also added the bottom most point of gcc, gcc9
* py-pycma: new package
* rename py-pycma in py-cma; py-cma: use pypi instead of github sources
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* lammps: add new stable version 20230802.1
* lammps: add missing potential download for +mesont
* lammps: fix python package install
* Update var/spack/repos/builtin/packages/lammps/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* lammps: py-numpy and py-mpi4py should be build and run deps
* lammps: add new 20231121 release
- MPIIO package has been removed -> disable mpiio variant
- LAMMPS_EXCEPTIONS is now always on -> disable exceptions variant
- CMake 3.16+ is now required
- Kokkos 4.1.0 is now supported
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add missing runtime dependency on py-colorama to py-ansimarkup
* Add py-metomi-isodatetime@3.1.0
* New package py-graphql-relay
* Update py-cylc-flowi, add version 8.2.3
* Fix merge conflict
* Revert mistake in var/spack/repos/builtin/packages/py-cylc-flow/package.py
* Update py-metomi-isodatetime dependencies for py-cylc-flow
* Add 'climbfuji' to list of maintainers for py-cylc-flow
* py-beartype: new package with version 0.15.0
* Update var/spack/repos/builtin/packages/py-beartype/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-beartype: depend on python 3.8 or higher
* py-beartype: add new version 0.16.2
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-mpi4py: fix build with Apple Clang
* [@spackbot] updating style on behalf of adamjstewart
---------
Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
* mochi-thallium: added a few newer versions
* mochi-thallium: added constraint on the version of margo required
* removed thallium 0.12 which needs updating
* mochi-thallium: fixed hash for version 0.12.0
* removed thallium 0.12 which needs updating (again)
When using `spack external find acfl`, we get the full version string
with 4 components in `packages.yaml`. This PR truncates the version
nubmer when finding the `armpl` component to be able to run without
intervention.
* developer tools stack try 2
This version is actually in use locally and has largely stabilized, at
least on x86. Some packages are still a challenge on ppc64le, but maybe
worth keeping this working as a set.
* add packages, try to get container with newer gcc
* remove reuse: true
* try to get cmake to build on medium, 25 minutes is too long
* add lsd package and add to dev tools stack
* clean up fzf dependency and sorting
* Update share/spack/gitlab/cloud_pipelines/stacks/developer_tools/spack.yaml
* cuda: add 12.3.0 (#40827)
* Switch to dashes
* yet more underscores
---------
Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
This looks to me like the best compromise regarding externals in a
build cache. I wouldn't want `spack install` on my machine to install
specs that were marked external on another. At the same time there are
centers that control the target systems on which spack is used, and
would want to use external in buildcaches.
As a solution, reuse concretization will now consider those externals
used in buildcaches that match a locally configured external in
packages.yaml.
So for example person A installs and pushes specs with this config:
```yaml
packages:
ncurses:
externals:
- spec: ncurses@6.0.12345 +feature
prefix: /usr
```
and person B concretizes and installs using that buildcache with the
following config:
```yaml
packages:
ncurses:
externals:
- spec: ncurses@6
prefix: /usr
```
the spec will be reused (or rather, will be considered for reuse...)
* gipaw.x installed by cmake if version >= 5c4a4ce.
gipaw.x will only be installed with cmake if the qe-gipaw version
is >= 5c4a4ce. Currently, QE source uses the older f5823521 one.
Here a patch to the submodule_commit_hash_records to use a newer
qe-gipaw version.
* initial commit to update hipblas rocalution, rocsolver, rocsparse to new syntax
* add rocblas test changes and fixes for hipblas and rocsolver tests
* fix styling
* remove updates for rocblas
* solver: use a unique counter for condition, triggers and effects
* Do not reset counters when re-running setup
What we need is just a unique ID, it doesn't need
to start from zero every time.
* Update SST packages to 13.1.0
* Allow mismatch between sst-core dependency and current macro version
* SST does not work with Python 3.12 yet
* Sanity check install binaries for sst-core
* Elements compiles with OTF2 but not OTF
* Version bounds in specs are inclusive
* Remove not-strictly-necessary file check
* oneapi 2024.0.0 release
* oneapi v2 directory support and some cleanups
* sycl abi change requires 2024 compilers for packages that use sycl
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* update spack recipe
* [@spackbot] updating style on behalf of toxa81
* change from @develop to @7.5.0
* return dependency on boost_filesystem
* return dependency on boost_filesystem
* remove boost filesystem as agreed by @RMeli and @simonpintarelli
---------
Co-authored-by: toxa81 <toxa81@users.noreply.github.com>
* Update to latest version
* Add dependency
* revert
* address PR comments
* Correct dependencies for 0.7 to 0.8 transition
* Fix cmake line.
* Update nanobind dep
---------
Co-authored-by: Matt Archer <ma595@cam.ac.uk>
Co-authored-by: Jack S. Hale <mail@jackhale.co.uk>
Co-authored-by: Garth N. Wells <gnw20@cam.ac.uk>
* add 0.12.0
* remove whitespace
* update deps
* Update var/spack/repos/builtin/packages/py-neo/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add dep for python 3.8+
* add dep for python 3.8+ with 0.12.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
PR #40929 reverted the argument parsing to make `spack --verbose
install` work again. It looks like `--verbose` is the only instance
where this kind of argument inheritance is used since all other commands
override arguments with the same name instead. For instance, `spack
--bootstrap clean` does not invoke `spack clean --bootstrap`.
Therefore, fix multi-line aliases again by parsing the resolved
arguments and instead explicitly pass down `args.verbose` to commands.
This commit discards type mismatches or failures to validate a package preference during concretization. The values discarded are logged as debug level messages. It also adds a config audit to help users spot misconfigurations in packages.yaml preferences.
This roughly restores the order of operation from Spack 0.20,
where where `AutotoolsPackage.setup_build_environment` would
override the env variable set in `setup_platform_environment` on
macOS.
When improving the error message, we started #showing in the
answer set a lot more symbols - but we forgot to suppress the
debug messages warning about UNKNOWN SYMBOLs
* onnxruntime: fix the call to as_string() operator
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-onnxruntime: rm now-unused stringpiece_1_10.patch
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* packages/hub: add new version, update to module
Hub now uses a go module to build, needs different env vars, and we're
on a very, very old version before that. Deprecate the old ones so we
can clean out that old build once we pass a spack version.
* cleanup suggested by @adamjstewart
* Update to latest version
* Fix linebreak
* Make suggested changes
* bumped to 0.6.1
* Update var/spack/repos/builtin/packages/py-scikit-build-core/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Chris Richardson <cnr12@cam.ac.uk>
Co-authored-by: Matt Archer <ma595@cam.ac.uk>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
In practice, one can only compiler for the Intel Data Center Max GPU
via a SYCL build and the oneAPI compiler. This is unlikely to change,
so we can be explicit about that.
* elbencho add new version and git master branch
* Update var/spack/repos/builtin/packages/elbencho/package.py
Co-authored-by: Alec Scott <alec@bcs.sh>
* formatting fix requested by @alecbcs
* remove whitespace added in blank line by github auto resolve
---------
Co-authored-by: Ethan W <mail@ethanwilliams.xyz>
Co-authored-by: Alec Scott <alec@bcs.sh>
* gromacs: Add new variants and clarify existing ones
Add new variants that reflect existing capabilities and defaults in
the upstream build system. Add other existing constraints that were
not yet specified.
* conform to style
* Fix missing hyphens
* Correct cmake variable names
* add elephant version v0.12.0 and 0.13.0
* update copyright
* reformat according to black format errors
* restore maintainers directive
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add dependency python 3.8+
* sorted dependencies
* sort dependencies from newest to oldest
* add deps for @master
* removed dependency for master, since it is included in 0.12.0:
* removed dependency for python 3.7+ , since 3.7+ is the lowest supported version anyway
* removed specific deps for master, since master is always newer than all stable releases
* updated numpy dependency for Elephant 0.12.0:
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* removed upper bounds for py-quantities, omitting v0.14.0
* add elephant v0.14.0
* update required quantities version
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: add versions 2.0.0 2.0.1; add maintainers
* py-cleo: add forgotten dependence
* py-cleo: update from review: remove preferred version, remove a dependence, fix py-rapidfuzz version
* py-cleo: deprecated version 1.0.0a5; add version 1.0.0; update dependences
* py-cleo: add version 2.1.0; update version range of dependences
* py-crashtest: add version 0.4.1, dependence of py-cleo
* py-cleo: update dependence
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: update dependence py-clikit
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: update dependence py-rapidfuzz
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-rapidfuzz: add version 2.2.0 dependence of py-cleo@2
* py-cleo: fix version range of py-crashtest
* py-rapidfuzz: fix dependences; add py-rapidfuzz-capi and py-jarowinkler
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Using Python 3.12 in a freshly cloned Spack repository results in
warnings such as this:
```
==> Warning: invalid escape sequence '\$'
==> Warning: invalid escape sequence '\('
==> Warning: invalid escape sequence '\.'
==> Warning: invalid escape sequence '\.'
```
These will turn into errors in 3.13, so fix them. All of them actually
do not need to be regexes, so convert them into normal strings.
* add version 0.14.1
* formatting
* style checks
* fix style errors
* remove old versions
* fix typo
* style
* update maintainers directive
* sort dependencies from newest to oldest
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* sort dependencies from newest to oldest
* removed upper bounds for python version
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* remove dependency on Python 3.7 +, since 3.7 is the lowest supported version anyway
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Added recent versions to ecflow/package.py, as well as added a cxxstd variant that is
needed to set BOOST_NO_CXX98_FUNCTION_BASE appropriately when building with C++17 standard.
* Fixed pep8 style error in the ecflow package.py script.
* Remov
* Removed cxxstd variant since the ecflow cmake configuration was already specifying
to use the c++17 standard for newer versions. The use of the BOOST_NO_CXX98_FUNCTION_BASE
define is now triggered by the ecflow version.
* Permit packages that depend on Intel oneAPI packages to access sdk
* Implement and use IntelOneapiLibraryPackageWithSdk
* Restore libs property to IntelOneapiLibraryPackage
* Conform to style
* Provide new class to infrastructure
* Treat sdk/include as the main include
* root: new version 6.30.00
There is a new release of ROOT, v6.30.00, with release notes at https://root.cern/doc/v630/release-notes.html.
In addition to some deprecations of build options, this updates the C++ standard to 17 or higher (well, 20), and increases the vc minimum version.
* vc: new version 1.4.4
* [@spackbot] updating style on behalf of wdconinc
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Improves the warning for deprecated preferences, and adds a configuration
audit to get files:lines details of the issues.
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* package/lemon: improve
* fix bug
* final improvements
* use f strings for boolean options, add soplex as TODO
* leave +coin as TODO
* depends on bzip2 when +coin
* tidy
---------
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
Co-authored-by: Sinan81 <Sinan81@users.noreply.github.com>
* initial commit to enable rocAL and add MIVisionX tests
* fix styling
* updated checksum for libjpeg patches
* update for 5.6
* use satisfies for checking spec version
xsdk: add +sycl variant - with amrex, arborx, ginkgo, petsc, sundials
xsdk: add +pflotran variant
xsdk: enable hypre+rocm
xsdk: enable superlu-dist for GPU - but use trilinos~superlu-dist [as that breaks builds]
xsdk: dealii: disable oce as it can cause intel-tbb-2017.6 to be picked up for some builds (for ex: gcc=13) and result in subsequent build failures
Some environments may have `dd4hep` as a concretized package without having it installed (yet). For those environments, `dd4hep` has property `libs` that is an empty list. Nevertheless, it can be added to a run environment (for example in case `dd4hep` is part of an environment). This results in an IndexError:
```
==> Warning: couldn't load runtime environment due to IndexError: list index out of range
```
To avoid the IndexError, only prepend the `dd4hep` libs if there are actually libs found.
Tests didn't cover the new `--variants-by-name` parameter in #40998.
Add some parameterization to hit that.
This changeset makes me think that the main section-printing loop in `spack info` isn't
factored so well. It makes it difficult to pass different arguments to different helper
functions. I could break it out into if statements if folks think that would be cleaner.
* dealii: 9.5.0
* kokkos+cuda_lambda
* dealii ^kokkos@3.7: require +cuda +cuda_lambda +wrapper
* Added 9.5.1, try ~cgal when +cuda
* Forward Cuda architecture request
* Remove workaround
* Try not enforcing the Kokkos compiler
* Enforce using nvcc_wrapper with Trilinos+Cuda
* Don't define CMAKE_*_COMPILER to point to MPI wrappers
* Use the same compiler as Trilinos/Kokkos
* Only check for Trilinos compiler
* Disable Trilinos+Cuda
* Disable Cuda support
* Try CUDA build without ninja
* Combined examples and examples_compile
* Use f-string for cuda_arch
* p -> _package
* Indentation
* Fix up f-string
---------
Co-authored-by: Luca Heltai <luca.heltai@sissa.it>
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
We have two ways to concretize now:
* `spack concretize` concretizes only the root specs that are not concrete in the environment.
* `spack concretize -f` eliminates all cached concretization data and reconcretizes the *entire* environment.
This PR adds `spack deconcretize`, which eliminates cached concretization data for a spec. This allows
users greater control over what is preserved from their `spack.lock` file and what is reused when not
using `spack concretize -f`. If you want to update a spec installed in your environment, you can call
`spack deconcretize` on it, and that spec and any relevant dependents will be removed from the lock file.
`spack concretize` has two options:
* `--root`: limits deconcretized specs to *specific* roots in the environment. You can use this to
deconcretize exactly one root in a `unify: false` environment. i.e., if `foo` root is a dependent
of `bar`, both roots, `spack deconcretize bar` will *not* deconcretize `foo`.
* `--all`: deconcretize *all* specs that match the input spec. By default `spack deconcretize`
will complain about multiple matches, like `spack uninstall`.
The ^mkl pattern was used to refer to three packages
even though none of software using it was depending
on "mkl".
This pattern, which follows Hyrum's law, is now being
removed in favor of a more explicit one.
In this PR gromacs, abinit, lammps, and quantum-espresso
are modified.
Intel packages are also modified to provide "lapack"
and "blas" together.
And improve the error message (load vs unload).
Of course you could have some uninstalled dependency too, but as long as
it doesn't implement `setup_run_environment` etc, I don't think it hurts
to attempt to load the root anyways, given that failure to do so is a
warning, not a fatal error.
This changes variant display to use a much more legible format, and to use screen space
much better (particularly on narrow terminals). It also adds color the variant display
to match other parts of `spack info`.
Descriptions and variant value lists that were frequently squished into a tiny column
before now have closer to the full terminal width.
This change also preserves any whitespace formatting present in `package.py`, so package
maintainers can make easer-to-read descriptions of variant values if they want. For
example, `gasnet` has had a nice description of the `conduits` variant for a while, but
it was wrapped and made illegible by `spack info`. That is now fixed and the original
newlines are kept.
Conditional variants are grouped by their when clauses by default, but if you do not
like the grouping, you can display all the variants in order with `--variants-by-name`.
I'm not sure when people will prefer this, but it makes it easier to tell that a
particular variant is/isn't there. I do think grouping by `when` is the better default.
* [lcov] Add build and runtime deps necessary for lcov@2.0.0:
+ Many additional Perl package dependecies are required for the new version of lcov.
+ Some of the new dependencies were not known to spack until now.
* Style fix
This commit improves forward compatibility of Spack with newer build cache metadata formats.
Before this commit, invalid or unrecognized metadata would be fatal errors, now they just cause
a mirror to be skipped.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Libgit2 requires python as build dependency. I was getting an error because it was falling back to system Python which is compiled with Intel compilers and thus, `libgit2` was failing because it couldn't find `libimf.so` (which doesn't make sense).
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
The libevent release tarballs ship with a `configure` script generated by an old `libtool`. The `libtool` generated by `configure` is not compatible with `MACOSX_DEPLOYMENT_VERSION` > 10. Regeneration of the `configure` scripts fixes build on macOS.
Original configure contains:
```
case $host_os in
rhapsody* | darwin1.[012])
_lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
darwin1.*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
darwin*) # darwin 5.x on
# if running on 10.5 or later, the deployment target defaults
# to the OS version, if on x86, and 10.4, the deployment
# target defaults to 10.4. Don't you love it?
case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
10.0,*86*-darwin8*|10.0,*-darwin[91]*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
10.[012][,.]*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
10.*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
esac
```
After re-running `autogen.sh`:
```
case $host_os in
rhapsody* | darwin1.[012])
_lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
darwin1.*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
darwin*)
case $MACOSX_DEPLOYMENT_TARGET,$host in
10.[012],*|,*powerpc*-darwin[5-8]*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
esac
```
* lcov: add version2, perl dep at build and runtime
* lcov: add runtime deps
* namespace-autoclean: new perl package
* datetime: dep on autoclean
* formatting
* abinit: add v9.10.3
Changed configure arguments for specfying how to use Wannier90 for versions
after 9.8.
When the mpi variant is requested, set the F90 environment variable to point
to the MPI Fortran wrapper when building versions after 9.8 instead of FC.
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 (#40676)
* exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 due to pybind error with py 3.11
* hiop@:1.0 +cuda: constrain to cuda@:11.9
* fixes syntax of maintainers
---------
Co-authored-by: eugeneswalker <38933153+eugeneswalker@users.noreply.github.com>
* intel-xed: fix git hash for mbuild, add version 2023.10.11
Fixes#40912
* Fix the git commit hash for mbuild 2022.04.17. This was broken in
commit eef9939c21 by mixing up the hashes for xed versus mbuild.
* Add versions 2023.08.21 and 2023.10.11.
* fix style
Before this PR, variant were not propagated to leaf nodes that could accept
the propagated value, if some intermediate node couldn't accept it.
This PR fixes that issue by marking nodes as "candidate" for propagation
and by setting the variant only if it can be accepted by the node.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Problem: the current configure arguments are added lists to a list,
and this needs to be adding strings to the same list.
Solution: ensure we add each item (string) separately.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Co-authored-by: vsoch <vsoch@users.noreply.github.com>
* Use `major.minor.patch`, `major.minor`, `major` in tags
* Ensure `latest` is the semver largest version, and not "latest in time"
* Remove Ubuntu 18.04 from the list of images
Modify the packages.yaml schema so that soft-preferences on targets,
compilers and providers can only be specified under the "all" attribute.
This makes them effectively global preferences.
Version preferences instead can only be specified under a package
specific section.
If a preference attribute is found in a section where it should
not be, it will be ignored and a warning is printed to screen.
Most queries will end up calling `spec.satisfies(query)` on everything in the DB, which
will cause Spack to ask whether the query spec is virtual if its name doesn't match the
target spec's. This can be expensive, because it can cause Spack to check if any new
virtuals showed up in *all* the packages it knows about. That can currently trigger
thousands of `stat()` calls.
We can avoid the virtual check for most successful queries if we consider that if there
*is* a match by name, the query spec *can't* be virtual. This PR adds an optimization to
the query loop to save any comparisons that would trigger a virtual check for last.
- [x] Add a `deferred` list to the `query()` loop.
- [x] First run through the `query()` loop *only* checks for name matches.
- [x] Query loop now returns early if there's a name match, skipping most `satisfies()` calls.
- [x] Second run through the `deferred()` list only runs if query spec is virtual.
- [x] Fix up handling of concrete specs.
- [x] Add test for querying virtuals in DB.
- [x] Avoid allocating deferred if not necessary.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Currently there's some hacky logic in the AppleClang compiler that makes
it also accept `gfortran` as a fortran compiler if `flang` is not found.
This is guarded by `if sys.platform` checks s.t. it only applies to
Darwin.
But on Linux the feature of detecting mixed toolchains is highly
requested too, cause it's rather annoying to run into a failed build of
`openblas` after dozens of minutes of compiling its dependencies, just
because clang doesn't have a fortran compiler.
In particular in CI where the system compilers may change during system
updates, it's typically impossible to fix compilers in a hand-written
compilers.yaml config file: the config will almost certainly be outdated
sooner or later, and maintaining one config file per target machine and
writing logic to select the correct config is rather undesirable too.
---
This PR introduces a flag `spack compiler find --mixed-toolchain` that
fills out missing `fc` and `f77` entries in `clang` / `apple-clang` by
picking the best matching `gcc`.
It is enabled by default on macOS, but not on Linux, matching current
behavior of `spack compiler find`.
The "best matching gcc" logic and compiler path updates are identical to
how compiler path dictionaries are currently flattened "horizontally"
(per compiler id). This just adds logic to do the same "vertically"
(across different compiler ids).
So, with this change on Ubuntu 22.04:
```
$ spack compiler find --mixed-toolchain
==> Added 6 new compilers to /home/harmen/.spack/linux/compilers.yaml
gcc@13.1.0 gcc@12.3.0 gcc@11.4.0 gcc@10.5.0 clang@16.0.0 clang@15.0.7
==> Compilers are defined in the following files:
/home/harmen/.spack/linux/compilers.yaml
```
you finally get:
```
compilers:
- compiler:
spec: clang@=15.0.7
paths:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
flags: {}
operating_system: ubuntu23.04
target: x86_64
modules: []
environment: {}
extra_rpaths: []
- compiler:
spec: clang@=16.0.0
paths:
cc: /usr/bin/clang-16
cxx: /usr/bin/clang++-16
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
flags: {}
operating_system: ubuntu23.04
target: x86_64
modules: []
environment: {}
extra_rpaths: []
```
The "best gcc" is automatically default system gcc, since it has no
suffixes / prefixes.
Add a new config section: `config:aliases`, which is a dictionary mapping aliases
to commands.
For instance:
```yaml
config:
aliases:
sp: spec -I
```
will define a new command `sp` that will execute `spec` with the `-I`
argument.
Aliases cannot override existing commands, and this is ensured with a test.
We cannot currently alias subcommands. Spack will warn about any aliases
containing a space, but will not error, which leaves room for subcommand
aliases in the future.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* Test that setup_run_environment changes to CC/CXX/FC/F77 are dropped in build env
* compilers set in run env shouldn't impact build
Adds `drop` to EnvironmentModifications courtesy of @haampie, and uses
it to clear modifications of CC, CXX, F77 and FC made by
`setup_{,dependent_}run_environment` routines when producing an
environment in BUILD context.
* comment / style
* comment
---------
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
This adds a rather trivial context manager that lets you deduplicate repeated
arguments in directives, e.g.
```python
depends_on("py-x@1", when="@1", type=("build", "run"))
depends_on("py-x@2", when="@2", type=("build", "run"))
depends_on("py-x@3", when="@3", type=("build", "run"))
depends_on("py-x@4", when="@4", type=("build", "run"))
```
can be condensed to
```python
with default_args(type=("build", "run")):
depends_on("py-x@1", when="@1")
depends_on("py-x@2", when="@2")
depends_on("py-x@3", when="@3")
depends_on("py-x@4", when="@4")
```
The advantage is it's clear for humans, the downside it's less clear for type checkers due to type erasure.
Create chains of causation for error messages.
The current implementation is only completed for some of the many errors presented by the concretizer. The rest will need to be filled out over time, but this demonstrates the capability.
The basic idea is to associate conditions in the solver with one another in causal relationships, and to associate errors with the proximate causes of their facts in the condition graph. Then we can construct causal trees to explain errors, which will hopefully present users with useful information to avoid the error or report issues.
Technically, this is implemented as a secondary solve. The concretizer computes the optimal model, and if the optimal model contains an error, then a secondary solve computes causation information about the error(s) in the concretizer output.
Examples:
$ spack solve hdf5 ^cmake@3.0.1
==> Error: concretization failed for the following reasons:
1. Cannot satisfy 'cmake@3.0.1'
2. Cannot satisfy 'cmake@3.0.1'
required because hdf5 ^cmake@3.0.1 requested from CLI
3. Cannot satisfy 'cmake@3.18:' and 'cmake@3.0.1
required because hdf5 ^cmake@3.0.1 requested from CLI
required because hdf5 depends on cmake@3.18: when @1.13:
required because hdf5 ^cmake@3.0.1 requested from CLI
4. Cannot satisfy 'cmake@3.12:' and 'cmake@3.0.1
required because hdf5 depends on cmake@3.12:
required because hdf5 ^cmake@3.0.1 requested from CLI
required because hdf5 ^cmake@3.0.1 requested from CLI
$ spack spec cmake ^curl~ldap # <-- with curl configured non-buildable and an external with `+ldap`
==> Error: concretization failed for the following reasons:
1. Attempted to use external for 'curl' which does not satisfy any configured external spec
2. Attempted to build package curl which is not buildable and does not have a satisfying external
attr('variant_value', 'curl', 'ldap', 'True') is an external constraint for curl which was not satisfied
3. Attempted to build package curl which is not buildable and does not have a satisfying external
attr('variant_value', 'curl', 'gssapi', 'True') is an external constraint for curl which was not satisfied
4. Attempted to build package curl which is not buildable and does not have a satisfying external
'curl+ldap' is an external constraint for curl which was not satisfied
'curl~ldap' required
required because cmake ^curl~ldap requested from CLI
$ spack solve yambo+mpi ^hdf5~mpi
==> Error: concretization failed for the following reasons:
1. 'hdf5' required multiple values for single-valued variant 'mpi'
2. 'hdf5' required multiple values for single-valued variant 'mpi'
Requested '~mpi' and '+mpi'
required because yambo depends on hdf5+mpi when +mpi
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo+mpi ^hdf5~mpi requested from CLI
3. 'hdf5' required multiple values for single-valued variant 'mpi'
Requested '~mpi' and '+mpi'
required because netcdf-c depends on hdf5+mpi when +mpi
required because netcdf-fortran depends on netcdf-c
required because yambo depends on netcdf-fortran
required because yambo+mpi ^hdf5~mpi requested from CLI
required because netcdf-fortran depends on netcdf-c@4.7.4: when @4.5.3:
required because yambo depends on netcdf-fortran
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo depends on netcdf-c
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo depends on netcdf-c+mpi when +mpi
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo+mpi ^hdf5~mpi requested from CLI
Future work:
In addition to fleshing out the causes of other errors, I would like to find a way to associate different components of the error messages with different causes. In this example it's pretty easy to infer which part is which, but I'm not confident that will always be the case.
See the previous PR #34500 for discussion of how the condition chains are incomplete. In the future, we may need custom logic for individual attributes to associate some important choice rules with conditions such that clingo choices or other derivations can be part of the explanation.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This PR implements the concept of "default environment", which doesn't have to be
created explicitly. The aim is to lower the barrier for adopting environments.
To (create and) activate the default environment, run
```
$ spack env activate
```
This mimics the behavior of
```
$ cd
```
which brings you to your home directory.
This is not a breaking change, since `spack env activate` without arguments
currently errors. It is similar to the already existing `spack env activate --temp`
command which always creates an env in a temporary directory, the difference
is that the default environment is a managed / named environment named `default`.
The name `default` is not a reserved name, it's just that `spack env activate`
creates it for you if you don't have it already.
With this change, you can get started with environments faster:
```
$ spack env activate [--prompt]
$ spack install --add x y z
```
instead of
```
$ spack env create default
==> Created environment 'default in /Users/harmenstoppels/spack/var/spack/environments/default
==> You can activate this environment with:
==> spack env activate default
$ spack env activate [--prompt] default
$ spack install --add x y z
```
Notice that Spack supports switching (but not stacking) environments, so the
parallel with `cd` is pretty clear:
```
$ spack env activate named_env
$ spack env status
==> In environment named_env
$ spack env activate
$ spack env status
==> In environment default
```
* Add command suggestions
This adds suggestions of similar commands in case users mistype a
command. Before:
```
$ spack spack
==> Error: spack is not a recognized Spack command or extension command; check with `spack commands`.
```
After:
```
$ spack spack
==> Error: spack is not a recognized Spack command or extension command; check with `spack commands`.
Did you mean one of the following commands?
spec
patch
```
* Add package name suggestions
* Remove suggestion to run spack clean -m
This completes to `spack concretize`:
```
spack conc<tab>
```
but this still gets hung up on the difference between `concretize` and `concretise`:
```
spack -e . conc<tab>
```
We were checking `"$COMP_CWORD" = 1`, which tracks the word on the command line
including any flags and their args, but we should track `"$COMP_CWORD_NO_FLAGS" = 1` to
figure out if the arg we're completing is the first real command.
This PR adds support for including separate definitions from `spack.yaml`.
Supporting the inclusion of files with definitions enables user to make
curated/standardized collections of packages that can re-used by others.
Currently module globals aren't set before running
`setup_[dependent_]run_environment` to compute environment modifications
for module files. This commit fixes that.
Looking at the memory profiles of concurrent solves
for environment with unify:false, it seems memory
is only ramping up.
This exchange in the potassco mailing list:
https://sourceforge.net/p/potassco/mailman/potassco-users/thread/b55b5b8c2e8945409abb3fa3c935c27e%40lohn.at/#msg36517698
Seems to suggest that clingo doesn't release memory
until end of the application.
Since when unify:false we distribute work to processes,
here we give a maxtaskperchild=1, so we clean memory
after each solve.
Some providers must provide virtuals "together", i.e.
if they provide one virtual of a set, they must be the
providers also of the others.
There was a bug though, where we were not checking if
the other virtuals in the set were needed at all in
the DAG.
This commit fixes the bug.
* libtheora: regenerate Makefile.in during autoreconf
The patch to inhibit running of configure would exit autogen.sh so early
that it did not yet run autoconf/automake/...
Instead of patching autogen.sh, just pass -V as argument, as this is
passed on to configure and lets it just print its version instead of
configuring the build tree.
Also drop arguments from autogen.sh, as they are unused when configure
does not run.
* libtheora: fix build on macos
Apply upstream patches in order to avoid unresolved symbols during building of libtheoraenc.
These patches require re-running automake/autoconf/...
Error messages:
libtool: link: /Users/ma/git/spack/lib/spack/env/clang/clang -dynamiclib -o .libs/libtheoraenc.1.dylib .libs/apiwrapper.o .libs/fragment.o .libs/idct.o .libs/internal.o .libs/state.o .libs/quant.o .l
ibs/analyze.o .libs/fdct.o .libs/encfrag.o .libs/encapiwrapper.o .libs/encinfo.o .libs/encode.o .libs/enquant.o .libs/huffenc.o .libs/mathops.o .libs/mcenc.o .libs/rate.o .libs/tokenize.o -L/opt/spac
k/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib -logg -lm -Wl,-exported_symbols_list -Wl,/var/folders/zv/qr55pmd9065glf0mcltpx5bm000102/T/ma/spack-stage/spac
k-stage-libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/spack-src/lib/theoraenc.exp -install_name /opt/spack/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib
/libtheoraenc.1.dylib -compatibility_version 3 -current_version 3.2
ld: warning: search path '/opt/spack/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib' not found
ld: Undefined symbols:
_th_comment_add, referenced from:
_theora_comment_add in apiwrapper.o
_th_comment_add_tag, referenced from:
_theora_comment_add_tag in apiwrapper.o
_th_comment_clear, referenced from:
_theora_comment_clear in apiwrapper.o
_th_comment_init, referenced from:
_theora_comment_init in apiwrapper.o
_th_comment_query, referenced from:
_theora_comment_query in apiwrapper.o
_th_comment_query_count, referenced from:
_theora_comment_query_count in apiwrapper.o
* libtheora: add git versions
stable as version name for theora-1.1 branch was chosen so that it sorts between 1.1.x and master
* libtheora: remove unused patch
thanks to @michaelkuhn for noticing
* Add 2023.09-0 for x86_64, aarch64, and ppc64le
extend the anaconda3 package.py to support aarch64 and ppc64le.
add the latest version of anaconda3 to each new platform, including the existing x86_64
* formatting
* Fix py-pyside2 to build with newer llvm and to use spack libglx and libxcb headers where system headers are missing
pyside2 needs LLVM_INSTALL_DIR to be set when using llvm 11: and expects system headers for libglx and libxcb and won't build otherwise.
* Fix styling
* remove raw string type
* Update var/spack/repos/builtin/packages/py-pyside2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
This PR makes it possible to select only a subset of virtual dependencies from a spec that _may_ provide more. To select providers, a syntax to specify edge attributes is introduced:
```
hdf5 ^[virtuals=mpi] mpich
```
With that syntax we can concretize specs like:
```console
$ spack spec strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas
```
On `develop` this would currently fail with:
```console
$ spack spec strumpack ^intel-parallel-studio+mkl ^openblas
==> Error: Spec cannot include multiple providers for virtual 'blas'
Requested 'intel-parallel-studio' and 'openblas'
```
In package recipes, virtual specs that are declared in the same `provides` directive need to be provided _together_. This means that e.g. `openblas`, which has:
```python
provides("blas", "lapack")
```
needs to provide both `lapack` and `blas` when requested to provide at least one of them.
## Additional notes
This capability is needed to model compilers. Assuming that languages are treated like virtual dependencies, we might want e.g. to use LLVM to compile C/C++ and Gnu GCC to compile Fortran. This can be accomplished by the following[^1]:
```
hdf5 ^[virtuals=c,cxx] llvm ^[virtuals=fortran] gcc
```
[^1]: We plan to add some syntactic sugar around this syntax, and reuse the `%` sigil to avoid having a lot of boilerplate around compilers.
Modifications:
- [x] Add syntax to interact with edge attributes from spec literals
- [x] Add concretization logic to be able to cherry-pick virtual dependencies
- [x] Extend semantic of the `provides` directive to express when virtuals need to be provided together
- [x] Add unit-tests and documentation
* vcftools: adding new version 0.1.16
* Update var/spack/repos/builtin/packages/vcftools/package.py
Co-authored-by: Alec Scott <alec@bcs.sh>
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
* Make sure sio is in dependent build env for podio
* podio: Fix likely(?) typo in root dependency
* podio: Add latest tag and new variants + dependencies
* podio: Add v00-16-07 tag
* podio: Fix dependencies flagged by package audit
* podio: Simplify root dependency
* podio: Add 0.17.1 tag
Allowing white space around `:` in version ranges introduces an ambiguity:
```
a@1: b
```
parses as `a@1:b` but should really be parsed as two separate specs `a@1:` and `b`.
With white space disallowed around `:` in ranges, the ambiguity is resolved.
Call setup_dependent_run_environment on both link and run edges,
instead of only run edges, which restores old behavior.
Move setup_build_environment into get_env_modifications
Also call setup_run_environment on direct build deps, since their run
environment has to be set up.
* update SPERR package
* remove blank line
* update SPERR to be version 0.7.1
* a little clean up
* bound versions that require zstd
* add USE_ZSTD
* add libpressio-sperr version upbound
* update libpressio-sperr
* address review comments
* improve format
---------
Co-authored-by: Samuel Li <Sam@Navada>
Co-authored-by: Samuel Li <sam@cisl-m121a>
* Add tests to ensure variant propagation syntax can round-trip to/from string
* Add a regression test for the bug in 35298
* Reconstruct the spec constraints in the worker process
Specs do not preserve any information on propagation of variants
when round-tripping to/from JSON (which we use to pickle), but
preserve it when round-tripping to/from strings.
Therefore, we pass a spec literal to the worker and reconstruct
the Spec objects there.
* Added NVML support to the slurm package
* dbus package is required for cgroup support
* Fixing formatting
* Style fix
* Added PAM support
* Added ROCm SMI support
- [x] Add links to information people are going to want to know when adding license
information to their packages (namely OSI licenses and SPDX identifiers).
- [x] Update the packaging docs for `license()` with Spack as an example for `when=`.
After all, it's a dual-licensed package that changed once in the past.
- [x] Add link to https://spdx.org/licenses/ in the `spack create` boilerplate as well.
* Load the script file during enviroment setup so that all the enviroment variables are set properly
* Patch csh/tcsh so that it uses spacks via env
* Update SHA for latest version
* Extend shebang to perl and fix up the regex
* Add liggght patched for newer compiler
Add C++ 17 support
Add Clang and Oneapi support
* Add maintainers
* Fix format in liggghts
* Fix maintainers before versions
Co-authored-by: Alec Scott <alec@bcs.sh>
* Fix style and user to usr
* Update package.py
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
* [add] py-moarchiving: new package
* py-moarchiving: update from review: description, variant default value is False, switch when and type
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* py-bluepyemodel: new package with dependencies
* py-morphio: add MPI as dependency to avoid failing builds
* Formatting
* py-bluepyefe: no need to set NEURON_INIT_MPI
* py-morphio: unifurcation branch is ancient history
* py-bluepyopt: only set NEURON_INIT_MPI with +neuron
* py-efel: get rid of old version
* py-morph{-tool,io}: rename develop to master to match branch
* py-bluepyefe: unset PMI_RANK is also neuron-related
* py-bluepyopt: PMI_RANK is also neuron-related
* Implement review remarks
* py-morph-tool, py-neurom: small fixes
* py-morphio: reword dependencies
Typically MSVC is detected via the VSWhere program. However, this may
not be available, or may be installed in an unpredictable location.
This PR adds an additional approach via Windows Registry queries to
determine VS install location root.
Additionally:
* Construct vs_install_paths after class-definition time (move it to
variable-access time).
* Skip over keys for which a user does not have read permissions
when performing searches (previously the presence of these keys
would have caused an error, regardless of whether they were
needed).
* Extend helper functionality with option for regex matching on
registry keys vs. exact string matching.
* Some internal refactoring: remove boolean parameters in some cases
where the function was always called with the same value
(e.g. `find_subkey`)
* e4s ci: add exago +cuda, +rocm builds
* exago: rename 5-18-2022-snapshot to snapshot.5-18-2022
* disable exago +rocm for non-external rocm ci install
* note that hiop +rocm fails to find hip libraries when they are spack-installed
.bat or .exe files can be considered executable on Windows. This PR
expands the regex for detectable packages to allow for the detection
of packages that vendor .bat wrappers (intel mpi for example).
Additional changes:
* Outside of Windows, when searching for executables `path_hints=None`
was used to indicate that default path hints should be provided,
and `[]` was taken to mean that no defaults should be chosen
(in that case, nothing is searched); behavior on Windows has
now been updated to match.
* Above logic for handling of `path_hints=[]` has also been extended
to library search (for both Linux and Windows).
* All exceptions for external packages were documented as timeout
errors: this commit adds a distinction for other types of errors
in warning messages to the user.
Credits to @ChristianKniep for advocating the idea of OCI image layers
being identical to spack buildcache tarballs.
With this you can configure an OCI registry as a buildcache:
```console
$ spack mirror add my_registry oci://user/image # Dockerhub
$ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR
$ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login credentials
```
which should result in this config:
```yaml
mirrors:
my_registry:
url: oci://ghcr.io/haampie/spack-test
push:
access_pair: [<username>, <password>]
```
It can be used like any other registry
```
spack buildcache push my_registry [specs...]
```
It will upload the Spack tarballs in parallel, as well as manifest + config
files s.t. the binaries are compatible with `docker pull` or `skopeo copy`.
In fact, a base image can be added to get a _runnable_ image:
```console
$ spack buildcache push --base-image ubuntu:23.04 my_registry python
Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
$ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
```
which should really be a game changer for sharing binaries.
Further, all content-addressable blobs that are downloaded and verified
will be cached in Spack's download cache. This should make repeated
`push` commands faster, as well as `push` followed by a separate
`update-index` command.
An end to end example of how to use this in Github Actions is here:
**https://github.com/haampie/spack-oci-buildcache-example**
TODO:
- [x] Generate environment modifications in config so PATH is set up
- [x] Enrich config with Spack's `spec` json (this is allowed in the OCI specification)
- [x] When ^ is done, add logic to create an index in say `<image>:index` by fetching all config files (using OCI distribution discovery API)
- [x] Add logic to use object storage in an OCI registry in `spack install`.
- [x] Make the user pick the base image for generated OCI images.
- [x] Update buildcache install logic to deal with absolute paths in tarballs
- [x] Merge with `spack buildcache` command
- [x] Merge #37441 (included here)
- [x] Merge #39077 (included here)
- [x] #39187 + #39285
- [x] #39341
- [x] Not a blocker: #35737 fixes correctness run env for the generated container images
NOTE:
1. `oci://` is unfortunately taken, so it's being abused in this PR to mean "oci type mirror". `skopeo` uses `docker://` which I'd like to avoid, given that classical docker v1 registries are not supported.
2. this is currently `https`-only, given that basic auth is used to login. I _could_ be convinced to allow http, but I'd prefer not to, given that for a `spack buildcache push` command multiple domains can be involved (auth server, source of base image, destination registry). Right now, no urllib http handler is added, so redirects to https and auth servers with http urls will simply result in a hard failure.
CAVEATS:
1. Signing is not implemented in this PR. `gpg --clearsign` is not the nicest solution, since (a) the spec.json is merged into the image config, which must be valid json, and (b) it would be better to sign the manifest (referencing both config/spec file and tarball) using more conventional image signing tools
2. `spack.binary_distribution.push` is not yet implemented for the OCI buildcache, only `spack buildcache push` is. This is because I'd like to always push images + deps to the registry, so that it's `docker pull`-able, whereas in `spack ci` we really wanna push an individual package without its deps to say `pr-xyz`, while its deps reside in some `develop` buildcache.
3. The `push -j ...` flag only works for OCI buildcache, not for others
* itk: patch missing include for newer compilers
* itk: The package doesn't use MPI
* itk: package requires the high-level hdf5 api
* itk: patch url with ?full_index=1
* itk: point to 4041 commit in master
* itk: don't constrain hdf5 with ~mpi
* rtmpdump: New package
* curl: Fix librtmp variant
Add the previously missing dependency required for rtmp support.
The variant has been broken since its addition in PR #25166.
Fixes one of the two issues reported in #26887.
* spack checksum pkg@1.2, use as version filter
Currently pkg@1.2 splits on @ and looks for 1.2 specifically, with this
PR pkg@1.2 is a filter so any matching 1.2, 1.2.1, ..., 1.2.10 version
is displayed.
* fix tests
* fix style
Update Tcl modulefile template to simplify generated `append-path`,
`prepend-path` and `remove-path` commands and improve their readability.
If path element delimiter is colon character, do not set the `--delim`
option as it is the default delimiter value.
Renames exclude_implicits to hide_implicits
When hide_implicits option is enabled, generate modulefile of
implicitly installed software and hide them. Even if implicit, those
modulefiles may be referred as dependency in other modulefiles thus they
should be generated to make module properly load dependent module.
A new hidden property is added to BaseConfiguration class.
To hide modulefiles, modulercs are generated along modulefiles. Such rc
files contain specific module command to indicate a module should be
hidden (for instance when using "module avail").
A modulerc property is added to TclFileLayout and LmodFileLayout classes
to get fully qualified path name of the modulerc associated to a given
modulefile.
Modulerc files will be located in each module directory, next to the
version modulefiles. This scheme is supported by both module tool
implementations.
modulerc_header and hide_cmd_format attributes are added to
TclModulefileWriter and LmodModulefileWriter. They help to know how to
generate a modulerc file with hidden commands for each module tool.
Tcl modulerc file requires an header. As we use a command introduced on
Modules 4.7 (module-hide --hidden-loaded), a version requirement is
added to header string.
For lmod, modules that open up a hierarchy are never hidden, even if
they are implicitly installed.
Modulerc is created, updated or removed when associated modulefile is
written or removed. If an implicit modulefile becomes explicit, hidden
command in modulerc for this modulefile is removed. If modulerc becomes
empty, this file is removed. Modulerc file is not rewritten when no
content change is detected.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* paraview: rebase the adios2 patch for 5.12-to-be
* paraview: disable fastfloat and token for 5.12-to-be
* paraview: require older protobuf for 5.12 as well
* paraview: require C++11-supporting protobuf for `master` too
* ci: don't register detectable compilers
Cause they go out of sync...
* remove intel compiler, it can be detected too
* Do not run spack compiler find since compilers are registered in concretize job already
* trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
message: "If you are referencing Spack in a publication, please cite the paper below."
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
abstract: >-
Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools.
Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible.
However, managing many configurations is difficult because the configuration space is combinatorial in size.
We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages and dependencies.
It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment.
We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.
Large HPC centers spend considerable time supporting software for thousands of users, but the
complexity of HPC software is quickly outpacing the capabilities of existing software management
tools. Scientific applications require specific versions of compilers, MPI, and other dependency
libraries, so using a single, standard software stack is infeasible. However, managing many
configurations is difficult because the configuration space is combinatorial in size. We
introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages
and dependencies. It allows any number of builds to coexist on the same system, and it ensures
that installed packages can find their dependencies, regardless of the environment. We show
through real-world use cases that Spack supports diverse and demanding applications, bringing
order to HPC software chaos.
preferred-citation:
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
type: conference-paper
@@ -71,7 +75,7 @@ preferred-citation:
type: doi
value: 10.1145/2807591.2807623
- description: "The DOE Document Release Number of the work"
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
