Commit Graph

462 Commits

Author SHA1 Message Date
kwryankrattiger
d227da5554 CI: Call timing script in after_script (#42166)
The main script body is over-written for power. Putting thet timing
aggregation in the after script allows it to be called on all of the
current pipelines.
2024-02-02 12:02:46 -06:00
kwryankrattiger
1f11b3844a CI: Add OIDC capability for deprecated CI (#42371)
This "breaks" the deprecated schema by allowing unknown attributes
to the attributes section of the job types. The breaking change here is
that deprecated stacks will no longer ignore attributes that are unknown
but rather assume the new CI schema behavior of injecting them into the
generated CI configuration. This change is required to secure
authentication in Spack CI.
2024-01-31 15:05:57 +01:00
Rocco Meli
e129a6f47a Add +dlaf variant to cp2k in CI (#42346) 2024-01-31 11:54:45 +01:00
Harmen Stoppels
d983ac35fe ci: bump ghcr.io/spack/linux-ubuntu22.04-x86_64_v2 tag (#42357) 2024-01-31 09:59:03 +01:00
Zack Galbreath
f03ae39fd1 Update GitLab memory requests (#42351)
* gitlab: remove commented-out duplicate entries

* gitlab: reclassify some packages from "huge" to "large"

Our observed max memory usage for these packages is as follows:

hipblas: 7.7G
qt: 6.6G
visit: 9.7G

All of these should fit within a "large" request (currently 12G).

* gitlab: remove pango from list of huge packages

This package is not currently built by any of our CI stacks.

* gitlab: update requests for high memory packages

Refine resource requests for memory-intensive packages based on
max memory usage data.
2024-01-29 19:28:58 +00:00
Dan LaManna
58b2201710 Stop passing manual AWS credentials to jobs (#42096) 2024-01-26 10:25:37 -07:00
Massimiliano Culpo
e77128dfa2 Run config audits in CI, add a new audit to detect wrongly named external specs (#42289) 2024-01-26 10:21:43 +01:00
eugeneswalker
4c7a1f541c e4s oneapi: use ghcr spack registry for runner image (#42267) 2024-01-26 02:02:58 +00:00
eugeneswalker
3f47cc8d00 e4s neoverse-v2: use ghcr.io/spack image registry (#42268) 2024-01-25 13:45:29 -08:00
eugeneswalker
266bbad8cd e4s: add gromacs (#42266) 2024-01-25 11:48:25 -08:00
Massimiliano Culpo
66813460c0 Add syntactic sugar for "strong preferences" and "conflicts" (#41832)
Currently requirements allow to express "strong preferences" and "conflicts" from
configuration using a convoluted syntax:
```yaml
packages:
  zlib-ng:
    require:
    # conflict on %clang
    - one_of: ["%clang", "@:"]
    # Strong preference for +shared
    - any_of: ["+shared", "@:"]
```
This PR adds syntactic sugar so that the same can be written as:
```yaml
packages:
  zlib-ng:
    conflict:
    - "%clang"
    prefer:
    - "+shared"
```
Preferences written in this way are "stronger" that the ones documented at:
- https://spack.readthedocs.io/en/latest/packages_yaml.html#package-preferences
2024-01-22 13:18:00 -08:00
eugeneswalker
58997f7f9a e4s ci: use latest intel/hpckit 2024 based image (#41437)
* e4s ci: use latest intel/hpckit 2024 based image

* use latest container image: ecpe4s/ubuntu22.04-runner-amd64-oneapi-2024.0.0:2023.12.01

* comment out failing specs

* update to use patched container

* remove generalized package preference for intel-oneapi-mkl@2023

* change packages commented out
2024-01-20 20:17:44 -08:00
eugeneswalker
104a2b5e11 e4s ci: switch to neoverse_v2 target (#42115) 2024-01-20 16:28:29 -08:00
eugeneswalker
5f58a4c079 add e4s aarch64 (#42066) 2024-01-14 14:26:51 -08:00
Massimiliano Culpo
c1af62783a Gitlab pipelines: check load avg also before running script (#41962) 2024-01-05 17:31:59 +01:00
Alec Scott
8f48bf7195 rust: add v1.75.0 & v1.74.0, merge related variants into +dev, add rust-analyzer (#41903)
* Add rust-analyzer as variant to rust build

* Expose cargo module only when +cargo

* rust: add v1.74.0 and v1.75.0 and remove variants in favor of +dev

* [@spackbot] updating style on behalf of alecbcs

* Fix variant typo

---------

Co-authored-by: alecbcs <alecbcs@users.noreply.github.com>
2024-01-04 11:08:20 -08:00
kwryankrattiger
a1e86781bd CI: Fix timing search paths to ignore bootstrap (#41893) 2023-12-28 20:43:12 -08:00
kwryankrattiger
d52fda6015 CI: Fix timing search paths to ignore bootstrap (#40677) 2023-12-28 16:24:37 -08:00
Harmen Stoppels
7cb873fb87 Revert "CI: Disable downloading artifacts from upstream jobs (#41432)" (#41843)
This reverts commit a43156a861.
2023-12-23 00:21:19 +01:00
Harmen Stoppels
8371bb4e19 gcc-runtime: add separate package for gcc runtime libs
The gcc-runtime package adds a separate node for gcc's dynamic runtime
libraries.

This should help with:

1. binary caches where rpaths for compiler support libs cannot be
   relocated because the compiler is missing on the target system
2. creating "minimal" container images

The package is versioned like `gcc` (in principle it could be
unversioned, but Spack doesn't always guarantee not mixing compilers)
2023-12-21 12:22:58 -08:00
Harmen Stoppels
e7f897f959 ci: use "strong preference" idiom for compilers (#41806)
to avoid duplication of conflicts / requirements in config
2023-12-21 12:50:35 +01:00
kwryankrattiger
a43156a861 CI: Disable downloading artifacts from upstream jobs (#41432)
* CI: Disable downloading artifacts from upstream jobs

* CI: Default .base-jobs are `when:manual`
2023-12-19 15:53:28 -07:00
Harmen Stoppels
4f8b856145 e4s: add julia (#41768) 2023-12-19 18:17:54 +01:00
Massimiliano Culpo
2ef8d09fc7 spack config get/blame: with no args, show entire config
This PR changes the default behavior of `spack config get` and `spack config blame`
to print a flattened version of the entire spack configuration, including any active 
environment, if the commands are invoked with no section arguments.

The new behavior is used in Gitlab CI to help debug CI configuration, but it can also
be useful when asking for more information in issues, or when simply debugging Spack.
2023-12-19 01:26:53 -08:00
Adam J. Stewart
ab50aa61db py-keras: add v3.0.0 (#41356)
* py-keras: add v3.0.0

* Older keras actually requires protobuf

* Correct url_for_version

* Capitalization is important

* Keep pil and pydot deps
2023-12-01 18:23:58 +00:00
Tom Scogland
dd607d11d5 developer tools stack try 2 (#40921)
* developer tools stack try 2

This version is actually in use locally and has largely stabilized, at
least on x86.  Some packages are still a challenge on ppc64le, but maybe
worth keeping this working as a set.

* add packages, try to get container with newer gcc

* remove reuse: true

* try to get cmake to build on medium, 25 minutes is too long

* add lsd package and add to dev tools stack

* clean up fzf dependency and sorting

* Update share/spack/gitlab/cloud_pipelines/stacks/developer_tools/spack.yaml

* cuda: add 12.3.0 (#40827)

* Switch to dashes

* yet more underscores

---------

Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
2023-11-30 18:32:21 +00:00
Luc Berger
450f938056 kokkos: add v4.2.00 (#41203)
* Kokkos: adding version 4.2.00 to the package
* Kokkos: adding AMD GPU arch
* kokkos@4.2.00 +sycl: patch numeric traits unit test

---------

Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2023-11-29 22:47:15 +00:00
eugeneswalker
430b2dff5c e4s ci: disable gpu test stack (#41296) 2023-11-28 18:02:00 +01:00
Andrey Perestoronin
3d744e7c95 intel-oneapi 2024.0.0: added new version to packages (#41135)
* oneapi 2024.0.0 release

* oneapi v2 directory support and some cleanups

* sycl abi change requires 2024 compilers for packages that use sycl

---------

Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
2023-11-27 13:13:04 -05:00
eugeneswalker
2870b6002c e4s oneapi stack: turn on +sycl: ginkgo, heffte, petsc, upcxx, warpx (#41157)
* e4s oneapi stack: turn on +sycl: ginkgo, heffte, petsc, upcxx, warpx

* comment out warpx; build fails; add note
2023-11-18 22:21:20 -08:00
Massimiliano Culpo
2ac128a3ad Add papyrus to the list of broken tests (#40923)
* Disable papyrus in the neoverse v1 pipeline
   See https://gitlab.spack.io/spack/spack/-/jobs/8983875
   The job is hanging on tests for 6 hrs.
* Add papyrus to broken tests instead of removing it
2023-11-14 07:37:29 -08:00
eugeneswalker
383ec19a0c Revert "Deactivate Cray sles, due to unavailable runner (#40291)" (#40910)
This reverts commit 4b06862a7f.
2023-11-09 12:24:18 -08:00
Massimiliano Culpo
afc693645a tcl: filter compiler wrappers to avoid pointing to Spack (#40946) 2023-11-08 19:38:41 +01:00
Greg Becker
0b344e0fd3 tutorial stack: update for changes to the basics section for SC23 (#40942) 2023-11-07 23:46:57 -08:00
Harmen Stoppels
c1f134e2a0 tutorial: use lmod@8.7.18 because @8.7.19: has bugs (#40939) 2023-11-07 23:04:45 +01:00
Harmen Stoppels
3a5c1eb5f3 tutorial pipeline: force gcc@12.3.0 (#40937) 2023-11-07 20:53:44 +01:00
Massimiliano Culpo
f51dad976e hdf5-vol-async: better specify dependency condition (#40882) 2023-11-04 20:31:52 +01:00
eugeneswalker
fd22d109a6 sundials +sycl: add cxxflags=-fsycl via flag_handler (#40845) 2023-11-04 08:55:19 -05:00
eugeneswalker
d4a1618e07 tau: update 2.33 hash, add syscall variant (#40851)
Co-authored-by: wspear <wjspear@gmail.com>
2023-11-03 07:58:00 +01:00
Harmen Stoppels
6933e1c3cb ci: bump tutorial image and toolchain (#40795) 2023-10-31 12:58:33 +01:00
Harmen Stoppels
160bfd881d tutorial: replace zlib -> gmake to avoid deprecated versions (#40769) 2023-10-31 10:04:53 +01:00
Harmen Stoppels
b1b8500eba ci: print colored specs in concretization progress (#40711) 2023-10-30 15:29:27 +01:00
eugeneswalker
ed7274a4d0 e4s ci stacks: add exago specs (#40712)
* e4s ci: add exago +cuda, +rocm builds

* exago: rename 5-18-2022-snapshot to snapshot.5-18-2022

* disable exago +rocm for non-external rocm ci install

* note that hiop +rocm fails to find hip libraries when they are spack-installed
2023-10-27 11:15:11 -07:00
Harmen Stoppels
51275df0b1 ci: spack compiler find should list extra config scopes (#40727)
otherwise it detected pre-configured compilers in an potentially different way.
2023-10-27 09:43:01 +02:00
Harmen Stoppels
c0a4be156c ci: don't put compilers in config (#40700)
* ci: don't register detectable compilers

Cause they go out of sync...

* remove intel compiler, it can be detected too

* Do not run spack compiler find since compilers are registered in concretize job already

* trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
2023-10-25 11:55:04 -07:00
Harmen Stoppels
0c30418732 ci: darwin aarch64 use apple-clang-15 tag (#40706) 2023-10-25 17:35:47 +02:00
Vicente Bolea
d0982115b3 Adios2: add kokkos variant (#40623)
* adios2: update variants and dependencies

* adios2: add kokkos rocm|cuda|sycl variant

* e4s oneapi ci stack: add adios2 +sycl

* e4s ci stack: add adios2 +rocm

* [@spackbot] updating style on behalf of vicentebolea

* Apply suggestions from code review

* adios2: fixed cuda variant

* update ecp-data-vis-sdk

* Update share/spack/gitlab/cloud_pipelines/stacks/e4s-power/spack.yaml

---------

Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Co-authored-by: vicentebolea <vicentebolea@users.noreply.github.com>
2023-10-23 13:01:57 -07:00
Harmen Stoppels
27a0425e5d concretize separately: show concretization time per spec as they concretize when verbose (#40634) 2023-10-20 17:09:19 +02:00
Harmen Stoppels
4bade7ef96 gromacs +cp2k: build in CI (#40494)
* gromacs +cp2k: build in CI

* libxsmm: x86 only

* attempt to fix dbcsr + new mpich

* use c11 standard

* gromacs: does not depend on dbcsr

* cp2k: build with cmake in CI, s.t. dbcsr is a separate package

* cp2k: cmake patches for config files and C/C++ std

* cp2k: remove unnecessary constraints due to patch
2023-10-20 16:20:20 +02:00
Scott Wittenburg
46c1a8e4c6 gitlab ci: Rework how mirrors are configured (#39939)
Improve how mirrors are used in gitlab ci, where we have until now thought
of them as only a string.

By configuring ci mirrors ahead of time using the proposed mirror templates,
and by taking advantage of the expressiveness that spack now has for mirrors,
this PR will allow us to easily switch the protocol/url we use for fetching
binary dependencies.

This change also deprecates some gitlab functionality and marks it for
removal in Spack 0.23:

    - arguments to "spack ci generate":
        * --buildcache-destination
        * --copy-to
    - gitlab configuration options:
        * enable-artifacts-buildcache
        * temporary-storage-url-prefix
2023-10-19 11:04:59 -05:00