Commit Graph

236 Commits

Author SHA1 Message Date
Harmen Stoppels
6a47339bf8
oneapi: before script load modules (#37678) 2023-05-15 18:39:58 +02:00
eugeneswalker
91c5b4aeb0
e4s ci stacks: add: hdf5-vol-{log,cache} (#37651) 2023-05-13 04:54:44 +00:00
Scott Wittenburg
c08be95d5e
gitlab ci: release fixes and improvements (#37601)
* gitlab ci: release fixes and improvements

  - use rules to reduce boilerplate in .gitlab-ci.yml
  - support copy-only pipeline jobs
  - make pipelines for release branches rebuild everything
  - make pipelines for protected tags copy-only

* gitlab ci: remove url changes used in testing

* gitlab ci: tag mirrors need public key

Make sure that mirrors associated with release branches and tags
contain the public key needed to verify the signed binaries.  This
also ensures that when stack-specific mirror contents are copied
to the root, the root mirror has the public key as well.

* review: be more specific about tags, curl flags

* Make the check in ci.yaml consistent with the .gitlab-ci.yml

---------

Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-05-12 15:22:42 -05:00
eugeneswalker
47ec6a6ae5
e4s ci: trilinos +rocm: enable belos to fix build failure (#37617) 2023-05-11 14:02:20 -07:00
Harmen Stoppels
2c17c4e632
ci: remove --mirror-url flag (#37457)
The flags --mirror-name / --mirror-url / --directory were deprecated in 
favor of just passing a positional name, url or directory, and letting spack
figure it out.

---------

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2023-05-10 16:34:29 -06:00
eugeneswalker
3446feff70
use latest trilinos for +cuda variants (#37164) 2023-05-08 12:29:54 -07:00
Harmen Stoppels
e17d6d5eee
gitlab ci: bump tutorial image (#37544) 2023-05-07 16:24:33 +02:00
Harmen Stoppels
ddea33bdc0
Update tutorial pipeline to Ubuntu 22.04 (#35451) 2023-05-05 17:52:07 +02:00
Harmen Stoppels
803425780e
ci: stop downloading recent gmake (#37458) 2023-05-05 17:09:56 +02:00
eugeneswalker
75bf6d665c
e4s ci: add xyce (#36841)
* e4s ci: add xyce

* relax trilinos contraints for xyce

* also relax trilinos constraint for e4s-power stack

* allow trilinos~shylu for xyce
2023-04-24 16:32:01 -07:00
eugeneswalker
c76023d7ac
e4s ci: add mgard (#36584) 2023-04-24 23:35:40 +02:00
Adam J. Stewart
b5d66e0144
MXNet: add v1.9 (#36590) 2023-04-24 11:13:09 -05:00
Adam J. Stewart
05215e016b
ML CI: fix mirror name (#37007) 2023-04-20 16:45:15 -07:00
kwryankrattiger
9745865250
DaV SDK: Enable ParaView raytracing with in SDK (#36844)
* DaV SDK: Enable ParaView raytracing with in SDK

* CI: Drop swr testing from Data Vis SDK

* ISPC: extend LLVM requirement to main

* DaV SDK: Disallow concretizing develop unifyfs

No longer needed after mochi-margo patch
2023-04-18 13:39:47 -05:00
Adam J. Stewart
036695ac94
CI: update Linux images in ML pipelines (#36766)
Add missing openssl/curl/pkgconfig deps to py-tokenizers

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2023-04-17 17:42:03 +02:00
Scott Wittenburg
bfa94c5781
gitlab ci: Better tagging of "service" jobs (#36846)
- Tag non-rebuild jobs to target a cheaper (and more highly available)
subset of runners.

- Add missing resource requests to these jobs as well.
2023-04-14 09:03:12 -06:00
eugeneswalker
a9db5620f5
remove x86_64_v3 tags (#36828) 2023-04-13 15:29:15 -05:00
kwryankrattiger
b940468890
CI: Update Data Vis SDK image (#36761)
Supersedes #34224
2023-04-13 09:54:42 -05:00
eugeneswalker
a88fdb216f
ci: gpu-tests stack: swap x86_64-{cuda,rocm} for x86_64 (#36759) 2023-04-10 17:01:18 -07:00
kwryankrattiger
b2310f9e64
Ci backwards compat (#36045)
* CI: Fixup docs for bootstrap.

* CI: Add compatibility shim

* Add an update method for CI

Update requires manually renaming section to `ci`. After
this patch, updating and using the deprecated `gitlab-ci` section
should be possible.

* Fix typos in generate warnings

* Fixup CI schema validation

* Add unit tests for legacy CI

* Add deprecated CI stack for continuous testing

* Allow updating gitlab-ci section directly with env update

* Make warning give good advice for updating gitlab-ci

* Fix typo in CI name

* Remove white space

* Remove unneeded component of deprected-ci
2023-04-10 16:46:45 -05:00
kwryankrattiger
4ace1e660a
Ecp hdf5 vol (#35195)
* ECP-SDK: enable hdf5 VOL adapters
- When +hdf5, enable VOL adapters suitable for the SDK.
- Each VOL package must prepend to the HDF5_PLUGIN_PATH.
- hdf5: 1.13.3 will break existing VOL packages, constrain
  VOLs related to SDK and add note to keep 1.13.2 available.
- hdf5-vol-async:
    - Do not set HDF5_VOL_CONNECTOR, consumers must opt-in.
    - Enforce DAG constraints on MPI to require threaded version.
    - Depend on an explicit version of argbots to relax
      concretization issues in other spack environments.
- paraview: fix compiler flag usage for the 110 ABI (followup to #33617).
* ECP Data and ViS: Add constraits for HDF5 VOLS
* CI: HDF5 1.14 builds without VisIt
* hdf5-vol-async: Update docs string

---------

Co-authored-by: Stephen McDowell <stephen.mcdowell@kitware.com>
2023-04-06 15:21:11 -07:00
Scott Wittenburg
08426ec492
gitlab ci: request more memory for publish job (#36560) 2023-03-31 00:19:59 +02:00
Harmen Stoppels
dba57ff113
ci: require x86_64_v3 everywhere (#36158) 2023-03-29 15:58:48 +02:00
eugeneswalker
c88b30b426
e4s power ci: ecp-data-vis-sdk: disable visit due to build issues (#36475) 2023-03-28 06:02:47 -04:00
Adam J. Stewart
a84fb716a0
Update the PyTorch ecosystem (#36132)
* py-pytorch-lightning: add v2.0.0

* py-lightning-utilities: add v0.8.0

* Update all PyTorch packages

* Open-CE does not yet have patches for PyTorch 2 on ppc64le
2023-03-17 10:13:44 +01:00
kwryankrattiger
bb73dfc02e
Hotfix: CI: Add CI target for gpu-test stack (#36136) 2023-03-15 21:13:02 -04:00
Harmen Stoppels
1691b7caac
Fix typo affecting Gitlab CI (#36103)
Introduced in #35944
2023-03-14 14:18:05 +01:00
Harmen Stoppels
2107b6bf00
Set build_jobs dynamically in CI to avoid oversubscription (#35996)
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
2023-03-13 08:29:58 +01:00
Harmen Stoppels
844701b974
get --dev and drop set -x (#36010) 2023-03-10 22:59:57 -08:00
Harmen Stoppels
08d7f47278
curl flag is not universally supported (#36009) 2023-03-10 16:20:32 -08:00
Harmen Stoppels
26fd1ac5b0
hotfix: fix double double quotes (#36005) 2023-03-10 13:41:01 -08:00
Zack Galbreath
e1301df60c
ci: version bump for ghcr.io/spack/e4s-amazonlinux-2 (#35976)
* ci: version bump for ghcr.io/spack/e4s-amazonlinux-2

This new image comes with GnuPG v2.4.0

* py-cython: upperbounds for Python versions

* fix py-gevent nonsense

---------

Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2023-03-10 13:32:11 -08:00
kwryankrattiger
f3595da600
CI boilerplate reduction (#34272)
* CI configuration boilerplate reduction and refactor

Configuration:
- New notation for list concatenation (prepend/append)
- New notation for string concatenation (prepend/append)
- Break out configuration files for: ci.yaml, cdash.yaml, view.yaml
- Spack CI section refactored to improve self-consistency and
composability
  - Scripts are now lists of lists and/or lists of strings
  - Job attributes are now listed under precedence ordered list that are
  composed/merged using Spack config merge rules.
  - "service-jobs" are identified explicitly rather than as a batch

CI:
- Consolidate common, platform, and architecture configurations for all CI stacks into composable configuration files
- Make padding consistent across all stacks (256)
- Merge all package -> runner mappings to be consistent across all
stacks

Unit Test:
- Refactor CI module unit-tests for refactor configuration

Docs:
- Add docs for new notations in configuration.rst
- Rewrite docs on CI pipelines to be consistent with refactored CI
workflow

* Script verbose environ, dev bootstrap

* Port #35409
2023-03-10 12:25:35 -07:00
Scott Wittenburg
16c67ff9b4
ci: Increase the amount of pruning possible for PR pipelines (#35944)
By setting the traversal depth to 1, only specs matching the changed
package and direct dependents of those (and of course all dependencies
of that set) are removed from pruning candidacy.
2023-03-10 11:19:52 -08:00
kwryankrattiger
895886959f
CI: Fix timeout for VisIt and ParaView (#35787)
ref. #35400
2023-03-03 09:13:14 +01:00
Massimiliano Culpo
07e251c887
Remove handling of deprecated target names (graviton) (#35537)
* Update target names for Gitlab pipelines

* Remove handling of deprecated names for graviton
2023-03-01 11:03:12 +01:00
eugeneswalker
34441c9eaa
ci: gpu test stack: turn off cuda@11.7 builds (#35467) 2023-02-13 21:54:01 -08:00
eugeneswalker
f7de22eb14
gpu test stack: add test w/ ^cuda@12 builds on A100 w/ newer driver (#35375)
* gpu test stack: test cuda@12 builds on A100 w/ newer driver

* get gpu info via nvidia-smi;

* kokkos+cuda^cuda@12 has genuine failure
2023-02-09 12:18:03 -07:00
eugeneswalker
f6fa64f979
ci: add minimal gpu testing stack (#35251)
* ci: add minimal gpu testing stack

* kokkos +cuda requires +wrapper...

* require pass

* add raja+cuda
2023-02-03 18:33:34 -08:00
eugeneswalker
99f3716346
e4s: add and prefer boost +python +filesystem +iostreams +system (#35180) 2023-01-30 07:37:35 -08:00
eugeneswalker
48a63719b2
e4s oneapi: use oneapi@2023.0.0 (#34634) 2023-01-29 13:22:14 -08:00
eugeneswalker
9f5d9266e6
e4s: add power stack (#32710)
* e4s: restore builds builds

* gitlab ci: allow UO to build protected binaries for signing

* use newer image; comment out failing builds

* gitlab-ci: Some tweaks for e4s power builds

- fix tags (no longer require generate jobs to run on aws)
- fix resource requests for generation jobs resource requests
- remove SPACK_SIGNING_KEY from protected power build jobs
- update UO signing key path
- change the CDash build group to reflect stack name
- retry pipeline generation jobs *always*

* correct double packages: section

* gitlab-ci:script: modernize

* remove new gnu make, not for ppc64le

---------

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2023-01-29 10:53:02 -06:00
eugeneswalker
a82fe2010f
e4s: add cusz +cuda (#35004) 2023-01-23 09:15:39 -08:00
Zack Galbreath
91fd6d5a84
Allocate more memory for generate jobs in all stacks (#35040)
* Allocate more memory for generate jobs in all stacks
* Add a 60 minute timeout on generate jobs
2023-01-20 11:06:49 -07:00
eugeneswalker
6713f7feda
e4s: ecp-data-vis-sdk +rocm: remove redundant amdgpu_target specification (#35022) 2023-01-20 00:18:41 +00:00
eugeneswalker
fed4eb2526
e4s ci: clean up unnecessary constraints (#35006) 2023-01-18 21:23:15 -08:00
eugeneswalker
1bf1c7de76
e4s: add sz3 (#35007) 2023-01-18 22:21:30 -06:00
eugeneswalker
ae5408de9a
e4s ci: add veloc (#35003) 2023-01-18 16:48:23 -08:00
eugeneswalker
8eb803ab4c
e4s: add py-h5py ~mpi variant (#34998) 2023-01-18 19:28:47 +00:00
eugeneswalker
886ea7c490
e4s: add py-h5py (#34914) 2023-01-16 15:52:24 +00:00