* gitlab ci: release fixes and improvements
- use rules to reduce boilerplate in .gitlab-ci.yml
- support copy-only pipeline jobs
- make pipelines for release branches rebuild everything
- make pipelines for protected tags copy-only
* gitlab ci: remove url changes used in testing
* gitlab ci: tag mirrors need public key
Make sure that mirrors associated with release branches and tags
contain the public key needed to verify the signed binaries. This
also ensures that when stack-specific mirror contents are copied
to the root, the root mirror has the public key as well.
* review: be more specific about tags, curl flags
* Make the check in ci.yaml consistent with the .gitlab-ci.yml
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
The flags --mirror-name / --mirror-url / --directory were deprecated in
favor of just passing a positional name, url or directory, and letting spack
figure it out.
---------
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
`spack buildcache create` is a misnomer cause it's the only way to push to
an existing buildcache (and it in fact calls binary_distribution.push).
Also we have `spack buildcache update-index` but for create the flag is
`--rebuild-index`, which is confusing (and also... why "rebuild"
something if the command is "create" in the first place, that implies it
wasn't there to begin with).
So, after this PR, you can use either
```
spack buildcache create --rebuild-index
```
or
```
spack buildcache push --update-index
```
Also, alias `spack buildcache rebuild-index` to `spack buildcache
update-index`.
Paths with spaces are an issue on Windows and our current powershell
scripts are not sufficiently hardended against their use.
This PR removes promlematic commandlets that do not work well with paths
with spaces and adds escape quotes in other areas where this could be an
issue.
* DaV SDK: Enable ParaView raytracing with in SDK
* CI: Drop swr testing from Data Vis SDK
* ISPC: extend LLVM requirement to main
* DaV SDK: Disallow concretizing develop unifyfs
No longer needed after mochi-margo patch
* CI: Fixup docs for bootstrap.
* CI: Add compatibility shim
* Add an update method for CI
Update requires manually renaming section to `ci`. After
this patch, updating and using the deprecated `gitlab-ci` section
should be possible.
* Fix typos in generate warnings
* Fixup CI schema validation
* Add unit tests for legacy CI
* Add deprecated CI stack for continuous testing
* Allow updating gitlab-ci section directly with env update
* Make warning give good advice for updating gitlab-ci
* Fix typo in CI name
* Remove white space
* Remove unneeded component of deprected-ci
* ECP-SDK: enable hdf5 VOL adapters
- When +hdf5, enable VOL adapters suitable for the SDK.
- Each VOL package must prepend to the HDF5_PLUGIN_PATH.
- hdf5: 1.13.3 will break existing VOL packages, constrain
VOLs related to SDK and add note to keep 1.13.2 available.
- hdf5-vol-async:
- Do not set HDF5_VOL_CONNECTOR, consumers must opt-in.
- Enforce DAG constraints on MPI to require threaded version.
- Depend on an explicit version of argbots to relax
concretization issues in other spack environments.
- paraview: fix compiler flag usage for the 110 ABI (followup to #33617).
* ECP Data and ViS: Add constraits for HDF5 VOLS
* CI: HDF5 1.14 builds without VisIt
* hdf5-vol-async: Update docs string
---------
Co-authored-by: Stephen McDowell <stephen.mcdowell@kitware.com>
- Update default image to Ubuntu 22.04 (previously was still Ubuntu 18.04)
- Optionally use depfiles to install the environment within the container
- Allow extending Dockerfile Jinja2 template
- Allow extending Singularity definition file Jinja2 template
- Deprecate previous options to add extra instructions
Update tcl and lmod modulefile template to provide more information on
help message (name, version and target) like done on whatis for lmod
modulefiles.
Adapt tcl and lmod modulefile templates to generate append-path or
remove-path commands in modulefile when respectively append_flags or
remove_flags commands are defined in package for run environment.
Fixes#10299.
Simplify environment modification block in modulefile Tcl template by
always setting a path delimiter to the prepend-path, append-path and
remove-path commands.
Remove --delim option to the setenv command as this command does not
allow such option.
Update test_prepend_path_separator test to explicitly check the 6
path-like commands that should be present in generated modulefile.
Example one:
```
spack install --add x y z
```
is equivalent to
```
spack add x y z
spack concretize
spack install --only-concrete
```
where `--only-concrete` installs without modifying spack.yaml/spack.lock
Example two:
```
spack install
```
concretizes current spack.yaml if outdated and installs all specs.
Example three:
```
spack install x y z
```
concretizes current spack.yaml if outdated and installs *only* concrete
specs in the environment that match abstract specs `x`, `y`, or `z`.
Adapt tcl modulefile template to call "module load" on autoload
dependency without testing if this dependency is already loaded or not.
The is-loaded test is not necessary, as module commands know how to cope
with an already loaded module. With environment-modules 4.2+ (released
in 2018) it is also important to have this "module load" command even if
dependency is already loaded in order to record that the modulefile
declares such dependency. This is important if you want to keep a
consistent environment when a dependent module is unloaded.
The "Autoloading" verbose message is also removed as recent module
commands will report such information to the user (depending on the
verbosity configured for the module command).
Such change has been test successfully with Modules 3.2 (EL7), 4.5 (EL8)
and 5.2 (latest) and also with Lmod 7 and 8 (as it is mentionned in
Spack docs that Lmod can be used along with tcl modules). Dependencies
are correctly loaded or unloaded, whether they are loaded/unloaded or
not.
This change fixes Tcl quoting issue introduced in #32853.
Fixes#19155.
* py-pytorch-lightning: add v2.0.0
* py-lightning-utilities: add v0.8.0
* Update all PyTorch packages
* Open-CE does not yet have patches for PyTorch 2 on ppc64le
This adds a new mode for `concretizer:reuse` called `dependencies`,
which only reuses dependencies. Currently, `spack install foo` will
reuse older versions of `foo`, which might be surprising to users.
* ci: version bump for ghcr.io/spack/e4s-amazonlinux-2
This new image comes with GnuPG v2.4.0
* py-cython: upperbounds for Python versions
* fix py-gevent nonsense
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* CI configuration boilerplate reduction and refactor
Configuration:
- New notation for list concatenation (prepend/append)
- New notation for string concatenation (prepend/append)
- Break out configuration files for: ci.yaml, cdash.yaml, view.yaml
- Spack CI section refactored to improve self-consistency and
composability
- Scripts are now lists of lists and/or lists of strings
- Job attributes are now listed under precedence ordered list that are
composed/merged using Spack config merge rules.
- "service-jobs" are identified explicitly rather than as a batch
CI:
- Consolidate common, platform, and architecture configurations for all CI stacks into composable configuration files
- Make padding consistent across all stacks (256)
- Merge all package -> runner mappings to be consistent across all
stacks
Unit Test:
- Refactor CI module unit-tests for refactor configuration
Docs:
- Add docs for new notations in configuration.rst
- Rewrite docs on CI pipelines to be consistent with refactored CI
workflow
* Script verbose environ, dev bootstrap
* Port #35409
By setting the traversal depth to 1, only specs matching the changed
package and direct dependents of those (and of course all dependencies
of that set) are removed from pruning candidacy.
* Style: black 23, skip magic trailing commas
* isort should use same line length as black
* Fix unused import
* Update version of black used in CI
* Update new packages
* Update new packages
