Commit Graph

89 Commits

Author SHA1 Message Date
Chuck Atkins
4fbb822072
visit: Overhaul to build in the DAV SDK (#30594)
* mesa-glu and mesa-demos: Fix conflicts with glu and osmesa

* visit: Update visit dependencies

* ecp-data-vis-sdk: Enable +visit

* ci[data-vis-sdk]: Enable +visit
2022-05-20 19:17:30 -04:00
Zack Galbreath
bee311edf3
Update GitLab environment variable name (#30671)
Use the IAM credentials that correspond to our new binary mirror
(s3://spack-binaries vs. s3://spack-binaries-develop)
2022-05-14 16:33:32 -06:00
Scott Wittenburg
0dd373846f gitlab ci: switch over to new bucket for all stacks 2022-05-13 10:45:12 -07:00
Massimiliano Culpo
d900ac2003
Reuse concretization by default (#30396)
* Enable reuse by default in Spack
* Update documentation to match new default
* Configure pipelines not to reuse software
2022-05-13 09:11:10 -07:00
Todd Gamblin
e0bed2d6a7
tutorial stack: allow deprecated versions (#30648)
For tutorial builds, we should continue to allow deprecated builds to be installed. We
can update them as needed when we update the tutorial, but we don't need to correct them
immediately on deprecation in CI.

- [x] add `deprecated:true` to tutorial `spack.yaml` config.
2022-05-13 04:09:39 -06:00
Scott Wittenburg
a65e00392c
gitlab ci: do not override .generate tags for e4s (#30571) 2022-05-10 08:05:19 -07:00
eugeneswalker
8575afac4e
e4s on mac ci: set SPACK_DISABLE_LOCAL_CONFIG=1 (#30568)
* e4s on mac ci: set SPACK_DISABLE_LOCAL_CONFIG=1
* export SPACK_USER_CACHE_PATH so that ~/.spack/... isn't used
2022-05-09 21:13:34 -06:00
Chuck Atkins
d8e010a9f5
ci: Enable the ParaView GUI in the DAVSDK pipeline (#30473)
* ci: Enable the ParaView GUI in the DAVSDK pipeline

* qt: Patch for long paths in ci
2022-05-06 09:36:56 -04:00
Scott Wittenburg
6b6147d5a0
gitlab: Remove temporary storage url from all stacks (#29949)
Gitlab pipelines run for spack already have other S3 storage locations
configured for storage of binaries, so this PR removes the redundant
per-pipeline mirror.  As a result, the "cleanup" jobs will no longer be
generated at the end of each pipeline, removing one possible point of
pipeline failure.
2022-05-04 09:18:47 -06:00
eugeneswalker
f7a9456553
e4s ci: uncomment umpire (#29776) 2022-04-27 18:22:46 +00:00
Christopher Kotfila
4067a28182
Set resource requests on package builds (#29922)
gitlab ci: Set resource requests explicitly

This PR sets resource requests for the Kubernetes executor, which should aid in
better workload scheduling in the cluster.  The specific values were derived from
profile data taken from several full "from scratch" rebuilds in a separate worker pool.

Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
2022-04-19 10:09:32 -06:00
eugeneswalker
5f996edde9
e4s ci: expand mac mini stack (#29929) 2022-04-07 15:28:33 -07:00
Scott Wittenburg
685e3d7ae9
spack ci: filter untouched pkgs from PR pipelines (#29697)
We've previously generated CI pipelines for PRs, and they rebuild any packages that don't have
a binary in an existing build cache.  The assumption we were making was that ALL prior merged
builds would be in cache, but due to the way we do security in the pipeline, they aren't. `develop`
pipelines can take a while to catch up with the latest PRs, and while it does that, there may be a
bunch of redundant builds on PRs that duplicate things being rebuilt on `develop`.  Until we can
do better caching of PR builds, we'll have this problem.

We can do better in PRs, though, by *only* rebuilding things in the CI environment that are actually
touched by the PR.  This change computes exactly what packages are changed by a PR branch and
*only* includes those packages' dependents and dependencies in the generated pipeline.  Other
as-yet unbuilt packages are pruned from CI for the PR.

For `develop` pipelines, we still want to build everything to ensure that the stack works, and to ensure
that `develop` catches up with PRs. This is especially true since we do not do rebuilds for *every* commit
on `develop` -- just the most recent one after each `develop` pipeline finishes.  Since we skip around,
we may end up missing builds unless we ensure that we rebuild everything.

We differentiate between `develop` and PR pipelines in `.gitlab-ci.yml` by setting 
`SPACK_PRUNE_UNTOUCHED` for PRs. `develop` will still have the old behavior.

- [x] Add `SPACK_PRUNE_UNTOUCHED` variable to `spack ci`
- [x] Refactor `spack pkg` command by moving historical package checking logic to `spack.repo`
- [x] Implement pruning logic in `spack ci` to remove untouched packages
- [x] add tests
2022-03-30 17:17:29 -07:00
eugeneswalker
5424e4bdc1
e4s ci spack.yaml: add h5bench (#29755) 2022-03-28 20:55:14 +00:00
eugeneswalker
ad6d28d9ad
e4s mini mac stack: add bzip2 (#29650) 2022-03-21 21:22:21 -06:00
eugeneswalker
c3b6b3036d
ci: add e4s mac stack (#29476) 2022-03-14 09:32:09 -07:00
Seth R. Johnson
5bea24526f
Fix overconstrained HDF5 variants (#29132)
* hdf5: mark +fortran+shared conflict for older version

This version was only activated unintentionally by silo's conflict
statement, but `@1.8.15+shared+fortran+cxx` errors out in configure:

```
CMake Error at CMakeLists.txt:814 (message):
   **** Shared FORTRAN libraries are unsupported ****
```

* silo: refine hdf5 conflicts to avoid building old version

Before this, `silo+hdf5` concretized to 1.10.7 or sometimes 1.8.15. Now
I've verified it works for the following configurations:
```
silo@4.10.2 patches=7b5a1dc,952d3c9
^   hdf5@1.10.7 api=default

silo@4.10.2 patches=7b5a1dc,952d3c9,eb2a3a0
^   hdf5@1.10.8 api=v18

silo@4.10.2 patches=7b5a1dc,952d3c9,eb2a3a0
^   hdf5@1.12.1 api=v110

silo@4.11-bsd patches=eb2a3a0
^   hdf5@1.12.1 api=v110

silo@4.11-bsd patches=eb2a3a0
^   hdf5@1.10.8 api=default

silo@4.11-bsd patches=eb2a3a0
^   hdf5@1.12.1 api=default
```

and verified that the following fail:
```
silo@4.10.2 ^hdf5@1.12.1 api=default
silo@4.11 ^hdf5 api=v18
silo@4.11-bsd ^hdf5@1.13.0 api=v12
silo@4.11-bsd ^hdf5@1.13.0 api=default
```

and have updated the constraints to match. Hdf5 no longer has to be
downgraded to work with Silo.

* silo: fix dependency conflicts

* py-h5py: shorten and add comments to py-h5py hdf5 dependencies

* e4s: remove slightly outdated hdf5 requirement

* e4s: remove excessive hdf5 variant constraints

These I think are holdovers from the old concretizer.

- `hdf5_compat` can be expressed as `+hdf5 ^hdf5@1.8`
- The extra variants on hdf5 shouldn't break conduit
- axom unnecessarily restricts hdf5 version

* conduit: restore hdf5_compat flag
2022-03-08 13:15:10 -05:00
eugeneswalker
1cb82dc542
e4s ci: packages: prefer openturns@1.18 (#29154) 2022-02-22 17:17:34 -08:00
Massimiliano Culpo
4437ff1fc3
Stabilize the concretization of ecp-data-vis-sdk by preferring conduit@0.7.2 (#28871) 2022-02-10 21:45:07 +01:00
eugeneswalker
81a6d17f4c
e4s: new specs: nccmp, nco, wannier90, lammps (#28833) 2022-02-10 10:47:19 -08:00
eugeneswalker
4c04a0c0b7
e4s ci: uncomment variorum following pr #28754 (#28763) 2022-02-04 11:15:35 +01:00
Adam J. Stewart
3576e5f3d6
Revert "Deprecate Python 2 installations" (#28411)
This reverts commit 7b76e3982f.
2022-02-02 10:12:33 -08:00
eugeneswalker
6e99f328b6
e4s ci: add spec: gptune (#28604) 2022-01-31 18:44:25 -08:00
Seth R. Johnson
2fd26be988
Trilinos: minimize E4S CUDA build (#28591)
* trilinos: update dependencies

Use the tribits deps to clarify some dependencies, and group some together
using `with` statements, eliminating some transitive conflict duplication.

* trilinos: Restricit cuda incompatibility

* e4s: vastly reduce number of packages in trilinos-cuda build

Not clear who the customers of cuda-enabled trilinos are, or what options
they need, or which sets of options conflict...

* e4s: remove ~wrapper from trilinos+cuda
2022-01-27 21:08:15 -07:00
Massimiliano Culpo
5476e5d035
Remove tut since it requires deprecated Python 3.6 (#28360) 2022-01-12 09:34:50 +01:00
Harmen Stoppels
071778b919
llvm: make targets a multivalued variant (#27735)
* llvm: make targets a multivalued variant

* Fix the targets variant values

1. Make them lowercase and add a mapping to cmake equivalent
2. auto -> all
2. Restore composability by using a multivalued variant, so that
   `targets=all` and `targets=x86` is combined to `targets=all,x86`
   which is then transformed into LLVM_TARGETS_TO_BUILD=all.

* use targets=x86 in iwyu

* Default to nvptx/amdgpu/host arch targets

* default to none

* Update var/spack/repos/builtin/packages/zig/package.py
2022-01-05 14:11:05 -07:00
Chris White
0a43dd1019
axom: add v0.6.1, fixed e4s builds (#28067) 2021-12-21 10:40:16 +01:00
eugeneswalker
b9a2e71a8b
e4s ci: expand +cuda specs (#27272) 2021-12-08 16:13:45 -08:00
kwryankrattiger
203ccdd976
LLVM: Revert build_llvm_dylib to llvm_dylib (#27761) 2021-12-03 23:47:05 +01:00
Harmen Stoppels
40a6ac62d3
llvm: introduce [build/link]_llvm_dylib (#27450)
Apart from building a single dylib for LLVM, users should also be able
to link tools against it.
2021-11-17 20:10:59 -07:00
eugeneswalker
a35d3b895b
e4s ci: enable tau +mpi +python variants (#27273) 2021-11-11 17:14:31 -08:00
Tamara Dahlgren
080f1872b8
Add the spack tutorial environment as a cloud pipeline stack (#27137) 2021-11-04 15:14:46 -07:00
Chuck Atkins
15d407c674
ci: Enable more packages in the DVSDK CI pipeline (#27025)
* ci: Enable more packages in the DVSDK CI pipeline

* doxygen: Add conflicts for gcc bugs

* dray: Add version constraints for api breakage with newer deps
2021-11-01 08:54:50 -07:00
Scott Wittenburg
f2a36bdf14
pipelines: llvm kills the xlarge, use huge (#27079) 2021-10-29 22:01:35 +00:00
eugeneswalker
0baec34dd7
E4S amd64 CI: add parsec (#26906) 2021-10-22 15:02:47 -06:00
eugeneswalker
3720d06e26
update E4S CI environments in preparation for 21.11 release (#26826)
* update E4S CI environments in preparation for 21.11 release

* e4s ci env: use clingo
2021-10-21 07:06:02 -07:00
Massimiliano Culpo
4c082a5357
Relax os constraints in e4s pipelines (#26547) 2021-10-19 10:51:37 -05:00
Massimiliano Culpo
79c92062a8
Gitlab pipelines: use images from the Spack organization (#26796) 2021-10-19 14:38:39 +02:00
Scott Wittenburg
9f09156923 Retry pipeline generation jobs in certain cases 2021-10-01 10:12:37 -07:00
Scott Wittenburg
d11156f361 Add DAG scheduling to child pipelines 2021-10-01 10:12:37 -07:00
Scott Wittenburg
4637c51c7f Service jobs do not need an active environment 2021-10-01 10:12:37 -07:00
Scott Wittenburg
d233c20725 Activate concrete env dir in all service jobs 2021-10-01 10:12:37 -07:00
Scott Wittenburg
5f527075bf Use same image to build e4s as to generate it 2021-10-01 10:12:37 -07:00
Scott Wittenburg
f6fcfd2acc Use same image to build dvsdk as to generate it 2021-10-01 10:12:37 -07:00
Scott Wittenburg
ae092915ac Use default runner image for radiuss 2021-10-01 10:12:37 -07:00
Scott Wittenburg
45b70d9798
Pipelines: Disable ppc builds until we have resources or make it smaller (#26238) 2021-09-24 08:24:36 -07:00
Massimiliano Culpo
c52426ea7a
Make clingo the default solver (#25502)
Modifications:
- [x] Change `defaults/config.yaml`
- [x] Add a fix for bootstrapping patchelf from sources if `compilers.yaml` is empty
- [x] Make `SPACK_TEST_SOLVER=clingo` the default for unit-tests
- [x] Fix package failures in the e4s pipeline

Caveats:
1. CentOS 6 still uses the original concretizer as it can't connect to the buildcache due to issues with `ssl` (bootstrapping from sources requires a C++14 capable compiler)
1. I had to update the image tag for GitlabCI in e699f14.  
1. libtool v2.4.2 has been deprecated and other packages received some update
2021-09-14 22:44:16 -07:00
Scott Wittenburg
bf7c12b4df
Pipelines: (Re)enable E4S on Power stack (#25921)
Pipelines: (Re)enable E4S on Power stack
2021-09-14 14:55:50 -06:00
Chuck Atkins
060582a21d
ci: Add ecp-data-vis-sdk CI pipeline (#22179)
* ci: Add a minimal subset of the ECP Data & Vis SDK CI pipeline

* ci: Expand the ECP Data & Vis SDK pipeline with more variants
2021-09-13 11:34:13 -06:00
Scott Wittenburg
81962f100c
Pipelines: build kokkos-kernels on bigger instance (#25845) 2021-09-10 08:27:42 -07:00