Commit Graph

431 Commits

Author SHA1 Message Date
Massimiliano Culpo
1b5b74390f
neoverse-v1: restore py-cinemasci (#44976)
Use a different tactic for determining conflicts.

Give more priority to setting False very old versions.
2024-07-01 09:53:24 +02:00
Massimiliano Culpo
c9a24bc6c5
neoverse-v1: comment out py-cinemasci (#44972) 2024-06-30 10:23:35 +02:00
eugeneswalker
35fc371222
update e4s stacks (#44813)
* update e4s stacks

* adios2 +rocm: disable kokkos due to spack issue #44832

* comment out mgard+cuda due to spack issue #44833

* comment out cabana, legion, arborx due to kokkos spack issue #44832

* comment out slepc, petsc due to petsc spack issue #44600

* comment out adios2+rocm due to kokkos rocm spack issue #44832

* comment out kokkos due to spack issue #44832
2024-06-26 15:24:34 -07:00
eugeneswalker
8d1aaef8b8
e4s ci: paraview: require +examples (#44847) 2024-06-25 15:12:56 +00:00
eugeneswalker
8479122e71
ecp rocm ci: add ecp-dav back + use updated container image w rocm-llvm-dev installed (#44827) 2024-06-24 16:48:40 -06:00
eugeneswalker
ad3d9c83fe
e4s: add glvis (#44812) 2024-06-24 05:57:54 -07:00
Andrey Perestoronin
55fe73586e
add packages from intel-oneapi-2024.2.0 release (#44789) 2024-06-21 16:17:54 -06:00
Stephen Sachs
113e231abe
Remove deprecated intel-* packages (#44700)
Packages will be removed with https://github.com/spack/spack/pull/44689. This PR
makes sure that the `aws-pcluster-x86_64_v4` stack still works as expected.
2024-06-20 07:07:27 -04:00
eugeneswalker
a389eb5a08
e4s external rocm ci: bump rocm stack to v6.1.1 (#44449)
* e4s external rocm ci: bump rocm stack to v6.1.1

* comment out exago+rocm due to issue with raja@0.14.0 see spack issue #44593

* comment out adios2+rocm due to spack issue #44594

* comment out petsc+rocm due to spack issue #44600

* comment out sundials+rocm due to spack issue #44601

* comment out slepc+rocm due to petsc spack issue #44600

* comment out tau+rocm due to spack issue #44659

* comment out ecp-data-vis-sdk due to spack issue #44745

* packages: register rocm-core as external

* re-enable tau due to issue #44659 having been resolved

* use latest ci image: ecpe4s/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.1.1:2024.06.17

* comment out paraview due to spack issue #44745

* comment out ecp-data-vis-sdk +vtkm due to issue https://gitlab.spack.io/spack/spack/-/jobs/11632511
2024-06-18 11:43:03 -07:00
Massimiliano Culpo
5e8eff24d2
Build developer-tools pipeline only on manylinux (#43811) 2024-06-14 10:13:06 +02:00
kwryankrattiger
b0a2ea3970
Generate jobs should use x86_64_v3 runners only (#44582) 2024-06-07 13:20:02 +02:00
Scott Wittenburg
0c3da1b498
gitlab ci: Remove protected publish job (#44304) 2024-06-04 11:54:05 -05:00
eugeneswalker
5ff8908ff3
e4s hopper ci: minimize root specs (#44436) 2024-05-29 10:09:04 -07:00
Massimiliano Culpo
8b7abace8b
Enforce consistency of gl providers (#44307)
* glew: rework dependency on gl

This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.

* mesa-demos: rework dependency on gl

This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.

* mesa-glu: rework dependency on gl

This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.

* paraview: fix dependency on glew

* mesa: group dependency on when("+glx")

* Add missing dependency on libxml2

* paraview: remove the "osmesa" and "egl" variant

Instead, enforce consistency using the "gl" virtual that allows
only one provider.

* visit: remove osmesa variant

* Disable paraview in the aws-isc stacks

* data-vis-sdk: rework constrains to enforce front-ends

* e4s-power: remove redundant paraview

* Pipelines: update osmesa variants

* trilinos-catalyst-ioss-adapter: make gl a run dependency
2024-05-23 20:17:51 +00:00
Massimiliano Culpo
9c88a48a73
Remove mesa18 and libosmesa (#44264)
* Remove mesa18 and libosmesa

mesa18 was introduced in #19528 as a way to maintain the old
autotools build of mesa separate from the new meson build.

We could add a second build system to mesa, but since mesa18 has
been deprecated for a long time, we'll just remove it.

libosmesa was used to multiplex the gl provider between mesa18
and mesa, and is thus unecessary. Remove it to reduce complexity
in the graphical stack.

* Remove references to mesa18 and libosmesa

* vtk: rework dependency on gl and osmesa

* memsurfer: rework dependency on vtk

* visit: minimal fix to avoid having both osmesa and glx
2024-05-21 10:18:14 -05:00
Robert Underwood
b9556c7c44
Additional packages for devtools-manylinux (#44273)
Co-authored-by: Robert Underwood <runderwood@anl.gov>
2024-05-21 13:37:04 +02:00
Stephen Sachs
297a3a1bc9
Add mpas-model and mpich to pcluster neoverse stack (#44151)
Should build now since https://github.com/spack/spack/pull/43547 has been merged.
2024-05-17 22:00:17 +02:00
John W. Parent
81fe460194
Gitlab CI: Windows Configs (#43967)
Add support for Gitlab CI on Windows

This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows 
infrastructure).

* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals

---------

Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2024-05-16 17:00:02 -06:00
Adam J. Stewart
8e9adefcd5
ML CI: update image (#43751)
* ML CI: update image

* Use main branch

* Use tagged version
2024-05-13 21:43:52 +02:00
Stephen Sachs
4a98d4db93
Add applications to aws-pcluster-* stacks (#43901)
* Add openfoam to aws-pcluster-neoverse_v1 stack

* Add more apps to aws-pcluster-x86_64_v4 stack

* Remove WRF while hdf5 cannot build in buildcache at the moment

* Update comment

* Add more apps for aws-pcluster-neoverse_v1 stack

* Remove apps that currently do not build

* Disable those packages that won't build

* Modify syntax such that correct cflags are used

* Changing syntax again to what works with other packages

* Fix overriding packages.yaml entry for gettext

* Use new `prefer` and `require:when` clauses to clarify intent

* Use newer spack version to install intel compiler

This removes the need for patches and makes sure the `prefer` directives in
`package.yaml` are understood.

* `prefer` not strong enough, need to set compilers

* Revert "Use newer spack version to install intel compiler"

This reverts commit ecb25a192c.

Cannot update the spack version to install intel compiler as this changes the
compiler hash but not the version. This leads to incompatible compiler paths. If
we update this spack version in the future make sure the compiler version also updates.

Tested-by: Stephen Sachs <stesachs@amazon.com>

* Remove `prefer` clause as it is not strong enough for our needs

This way we can safely go back to installing the intel compiler with an older
spack version.

* Prefer gcc or oneapi to build gettext

* Pin gettext version compatible with system glibc-headers

* relax gettext version requirement to enable later versions

* oneapi cannot build older gettext version
2024-05-12 10:48:02 -07:00
Harmen Stoppels
65d1ae083c
gitlab ci: tutorial: add julia and vim (#44073) 2024-05-08 14:18:12 +02:00
Harmen Stoppels
26759249ca
gitlab: dont build paraview for neoverse v2 (#44060) 2024-05-07 18:59:12 +02:00
Harmen Stoppels
a16ee3348b
Do not cache indices in Gitlab (#44029) 2024-05-06 15:54:21 +02:00
Scott Wittenburg
aae7a22d39
gitlab: release branch pipelines rebuild what changed (#43990) 2024-05-04 10:11:48 +02:00
eugeneswalker
1d59637051
e4s ci: add py-amrex (#43904) 2024-05-02 08:57:26 -06:00
Harmen Stoppels
9b62a9c238
gitlab ci: cache user cache (#43952) 2024-05-02 12:06:30 +02:00
Massimiliano Culpo
877e09dcc1
Delete leftover file (#43869)
This file is not needed anymore, was introduced in #19688
2024-04-29 22:45:41 +02:00
Adam J. Stewart
1cae1299eb
CI: remove ML ROCm stack (#43825) 2024-04-25 12:56:59 +02:00
Adam J. Stewart
f29fa1cfdf
CI: remove MXNet (#43704) 2024-04-18 10:04:03 -07:00
Adam J. Stewart
f406f27d9c
ML CI: remove extra xgboost (#43709) 2024-04-18 09:08:25 -07:00
kwryankrattiger
747cd374df
Run after_script aggregator with spack python (#43669) 2024-04-16 19:03:44 +02:00
eugeneswalker
6391559fb6
e4s ci: add: netcdf-fortran, fpm, e4s-cl (#43601) 2024-04-11 21:01:38 -06:00
kwryankrattiger
663e20fcc4
ParaView: add v5.12.0 (#42943)
* ParaView: Update version 5.12.0

Add 5.12.0 release
Update default to 5.12.0

* Add patch for building ParaView 5.12 with kits

* Drop VTKm from neoverse
2024-04-06 04:12:48 +00:00
eugeneswalker
6428132ebb
e4s ci: enable lammps variants from presets/most.cmake (#43522) 2024-04-05 20:56:18 -07:00
eugeneswalker
171958cf09
py-deephyper: add v0.6.0 (#43492)
* py-deephyper: add latest version: v0.6.0

* e4s: add py-deephyper

* v0.6.0: depend on python@3.7:3.11

* add py-packaging constraint so arm64 builds work

* [@spackbot] updating style on behalf of eugeneswalker
2024-04-06 00:28:37 +00:00
eugeneswalker
35f8b43a54
e4s ci: add nekbone (#43515)
* e4s ci: add nekbone, nek5000

* remove nek5000
2024-04-05 16:36:13 -07:00
kwryankrattiger
c68d739825
CI: Add debug to the log aggregation script (#42562)
* CI: Add debug to the log aggregation script
2024-04-05 14:00:27 -05:00
Adrien Bernede
92b1c8f763
RADIUSS packages update (Starting over #39613) (#41375) 2024-04-02 15:03:07 -07:00
eugeneswalker
7756c8f4fc
ci devtools manylinux2014: update ci image with compatible gpg (#43421) 2024-03-29 16:12:55 -06:00
kwryankrattiger
8b89287084
CI Reproducer on Metal (#43411)
* MacOS image remove requires override syntax

* Metal reproducer auto start and cross-platform
2024-03-29 12:32:54 -05:00
kwryankrattiger
e97787691b
force oneapi compiler unless specified otherwise (#43419) 2024-03-29 09:20:26 -07:00
eugeneswalker
ed40c3210e
ci: add developer-tools-manylinux2014 stack (#43128)
* ci: add developer-tools-manylinux2014 stack

* add libtree, patchelf
2024-03-26 08:02:16 -07:00
Stephen Sachs
aa39465188
Re enable aws pcluster buildcache stack (#38931)
* Changes to re-enable aws-pcluster pipelines

- Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
- Install gcc from hash so there is no risk of building gcc from source in pipleine.
- `packages.yam` files are now part of the pipelines.
- No more eternal `postinstall.sh`. The necessary steps are in `setup=pcluster.sh` and will be version controlled within this repo.
- Re-enable pipelines.

* Add  and

* Debugging output & mv skylake -> skylake_avx512

* Explicilty check for packages

* Handle case with no intel compiler

* compatibility when using setup-pcluster.sh on a pre-installed cluster.

* Disable palace as parser cannot read require clause at the moment

* ifort cannot build superlu in buildcache

`ifort` is unable to handle such long file names as used when cmake compiles
test programs inside build cache.

* Fix spack commit for intel compiler installation

* Need to fetch other commits before using them

* fix style

* Add TODO

* Update packages.yaml to not use 'compiler:', 'target:' or 'provider:'

Synchronize with changes in https://github.com/spack/spack-configs/blob/main/AWS/parallelcluster/

* Use Intel compiler from later version (orig commit no longer found)

* Use envsubst to deal with quoted newlines

This is cleaner than the `eval` command used.

* Need to fetch tags for checkout on version number

* Intel compiler needs to be from version that has compatible DB

* Install intel compiler with commit that has DB ver 7

* Decouple the intel compiler installation from current commit

- Use a completely different spack installation such that this current pipeline
commit remains untouched.
- Make the script suceed even if the compiler installation fails (e.g. because
the Database version has been updated)
- Make the install targets fall back to gcc in case the compiler did not install
correctly.

* Use generic target for x86_64_vX

There is no way to provision a skylake/icelake/zen runner. They are all in the
same pools under x86_64_v3 and x86_64_v4.

* Find the intel compiler in the current spack installation

* Remove SPACK_TARGET_ARCH

* Fix virtual package index & use package.yaml for intel compiler

* Use only one stack & pipeline per generic architecture

* Fix yaml format

* Cleanup typos

* Include fix for ifx.cfg to get the right gcc toolchain when linking

* [removeme] Adding timeout to debug hang in make (palace)

* Revert "[removeme] Adding timeout to debug hang in make (palace)"

This reverts commit fee8a01580489a4ea364368459e9353b46d0d7e2.

* palace x86_64_v4 gets stuck when compiling try newer oneapi

* Update comment

* Use the latest container image

* Update gcc_hashes to match new container

* Use only one tag providing tags per extends call

Also removed an unnecessary tag.

* Move generic setup script out of individual stack

* Cleanup from last commit

* Enable checking signature for packages available on the container

* Remove commented packages / Add comment for palace

* Enable openmpi@5 which needs pmix>3

* don't look for intel compiler on aarch64
2024-03-21 14:45:05 -05:00
Massimiliano Culpo
6f7f9528e5
cray-rhel: add a lower bound to mgard (#43187) 2024-03-15 11:25:56 +01:00
eugeneswalker
ba39924046
e4s cray ci: mgard is broken, disable spec (#43194) 2024-03-14 14:06:36 -07:00
eugeneswalker
97e29e501d
e4s ci stacks: add cp2k cpu and gpu specs (#42454)
* e4s ci stacks: add cp2k cpu and gpu specs

* remove non-building cp2k specs
2024-03-11 09:29:51 -07:00
Adam J. Stewart
9786bd932b
Update TensorFlow ecosystem (#41069) 2024-03-11 14:33:16 +01:00
Adam J. Stewart
8bcf6a31ae
ML CI: variants are now required (#42851) 2024-03-07 11:42:13 +01:00
eugeneswalker
5c482d0d7e
reduce size of e4s to deal with large rebuild artifact (#42884) 2024-02-29 13:44:01 -07:00
eugeneswalker
13709bb7b7
e4s: new packages: glvis, laghos (#42847)
* e4s: new packages: glvis, laghos

* gl: require: osmesa

* be explicit: glvis ^llvm so that llvm-amdgpu not chosen

* glvis fails on oneapi stack due to issue 42839
2024-02-28 09:26:53 -08:00