* Fix cdash reporter time stamps (#38818).
The cdash reporter is created before packages are installed so save the
starttime then instead of the endtime.
* Use endtime instead of starttime for the endtime of update
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Participation in the venerable Spack google group has dwindled, though we still have
540+ subscribers there. I've made the mailing list announcement-only, and I've given
a few maintainers posting privileges.
This PR adds some notes to the README indicating that the mailing list is only for
announcements.
* Ensure that additional environment variables are set when a module
file is generated.
* Fixed the detection of the opal_prefix / MPI_ROOT field to use ompi_info.
---------
Co-authored-by: Greg Becker <becker33@llnl.gov>
* depend_on python
There is an ill-named variant "python" that enables the pytrilinos1
variant. This made it through our testing but broke on our actual
CI test machines.
* adjust "python" variant based on Trilinos version
For Trilinos <= 14, enable PyTrilinos(1). For later versions
of Trilinos, enable PyTrilinos2.
We still support directly enabling PyTrilinos2 via the "pytrilinos2"
variant.
* remove pytrilinos2 variant
* correct depends_on constraints
- we don't have a fallback if make is not installed
- we assume file system locking works
- we don't verify that make is gnu make (bootstrapping fails on FreeBSD as a result)
- there are some weird race conditions in writing spack.yaml on concurrent spack install
- the view is updated after every package install instead of post environment install.
Forbid nested dependencies in depends_on declarations, by running an audit in CI.
Fix the packages not passing the new audit:
- amd-aocl
- exago
- palace
- shapemapper
- xsdk-examples
ginkgo: add a commit sha to v1.5.0.glu_experimental
* New variants:
- `tmvz-cpu`
- `tmvz-gpu`
- `tmvz-pymva`
- `tmvz-sofie`
* Improve X-related dependencies.
* Improve TMVA-related dependencies with more specificity.
* Patch possible missing standard header include in Eve7.
* Patch Protobuf handling to support new Protobuf-provided CMake config
files required to handle transitive `abseil-cpp` dependence.
* Add missing terminal newline to `webgui` patch to remove patch
warning.
* Handle deprecated/removed build options.
* Handle unwanted system paths in various `PATH`-like environment
variables.
The condition on swig can be interpreted as "true if true,
false if false" and gives clingo the option to add swig
or not.
If not other optimization criteria break the tie, then
the concretization is non-deterministic.
* added external libxc/elpa choice
* fixed formatting issues and 1 unused variant found by reviewer
* try to fix a string formatting issue
* try to fix some other string formatting issues
* fixed 1 flake8 style issue
* use explicit fftw-api@3
* Add `url_list` to facilitate finding new versions.
* `cxxstd` is not meaningful when `@:2.99.99` as it was a header-only
package before v3.
* Support C++20/23, remove C++14 support.
* Add @greenc-FNAL to maintainers.
* Add CMake arguments to support testing, build of extras and examples.
* Only build tests for proj package if required
Even if tests are not explictly required to be built, proj build them
anyway and tries to download Google Test.
* proj: fix name of test activation flag
* proj: Always set test activation flag
* proj: Patch test activation logic for versions 5.x
* py-python-pptx: new package
* py-python-pptx: use pil instead of pillow, remove version constraint on python
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* py-tldextract: new package
* py-tldextract: add version 5.1.1
* py-tldextract: fix version constraint on py-setuptools-scm
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* Add a new version of ruff
* Add a comment about where the dependency can be found
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
* Update py-werkzeug version dependency for py-graphene-tornado@2.6.1
* Add note on diverging version requirements for py-werkzeug in py-graphene-tornado
* legion: correct cuda dependency for cr version
* Update var/spack/repos/builtin/packages/legion/package.py
---------
Co-authored-by: Davis Herring <herring@lanl.gov>
* py-gidgethub: add new package
* Add main branch version and scope flit/flit-core dependency
* Update var/spack/repos/builtin/packages/py-gidgethub/package.py
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* Add optional dependencies as variants of package
* Add git url for main version
* Fix variant and dependency ordering
---------
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* gidgetlab: add new package
* Convert both cachetools and aiohttp to optional deps with variants
* Fix forgotten variant conditional on cachetools dependency
* Add git url and main version for dev workflows
* Fix variant and dependency ordering
* Remove cachetools variant and merge dependency with aiohttp variant
* grep WM_PROJECT_VERSION from etc/bashrc
This fixes the problem when building from a manually checked out repo
which might have a different version wrt the one defined in the spack
package (e.g. anything later than 5.0 is known as 5.x by the build
system)
* patch applies to just 5.0, in newer versions it is already addressed
In `5.20171030` there's a commit
c66fba323c
very similar (almost identical) to what the patch `50-etc.patch` does.
So the patch should not be applied to other than `5.0` otherwise it errors.
References:
- https://github.com/OpenFOAM/OpenFOAM-5.x/commits/20171030/etc/bashrc
- 197d9d3bf2/etc/bashrc (L45-L47)
This was missed while backporting the new `spack info` command from #40326.
Variants should be sorted by name when invoking `spack info --variants-by-name`.
Intel made an incompatible change in XED in 2023.08.21 that breaks
hpctoolkit (at run time). Hpctoolkit develop can adapt (soon will),
but older versions must use xed :2023.07.09.
* Updating the LBANN, Hydrogen, and DiHydrogen recipes for both new
variants and to make sure that RPATHs are properly setup.
Co-authored-by: bvanessen <bvanessen@users.noreply.github.com>
Deprecating intel package, which contains intel classic compilers. This package has not been updated in 3 years. Please use intel-oneapi-compilers instead.
* gipaw.x installed by cmake if version >= 5c4a4ce.
gipaw.x will only be installed with cmake if the qe-gipaw version
is >= 5c4a4ce. Currently, QE source uses the older f5823521 one.
Here a patch to the submodule_commit_hash_records to use a newer
qe-gipaw version.
* Update package.py
* Delete var/spack/repos/builtin/packages/quantum-espresso/gipaw-eccee44.patch
* Update package.py
* Restoring gipaw-eccee44 patch
* Update package.py
* Add fox variant in quantum-espresso
* Fix an issue introduced in #36484. Patches are 7.1 only.
* Change plugin handling.
* formatting.
* Typo correction
* Refine conflict
---------
Co-authored-by: S. Alexis Paz <alexis.paz@gmail.com>
* Add EGL support to ParaView and Glew
add a package for egl that provides GL but also adds
EGL libs and headers for projects that need them
Fix a header problem with the opengl package
Format files using black
* better description for egl variant description
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* better check/setup of non egl variant dependencies
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* Add biddisco as maintainer
* Fix unused var style warning
* Add egl conflicts for other gl providers
---------
Co-authored-by: Vicente Bolea <vicente.bolea@gmail.com>
* py-pyglet: version bump
* py-pyglet: use zip instead of whl, update dependencies
* py-pyglet: 2.0.9 and 2.0.10 zips should be downloaded from github
* py-pyglet: style
* py-pyglet: use virtual packages in dependencies
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* py-pyglet: doesn't depend on py-future any more
* py-pyglet: remove glx dependency
* py-pyglet: back to the pypi zipfiles with patch instead
---------
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* Add initial version of verible to spack
* Update to use explict url path for each release, as the release tagh includes extra data, also added the bottom most point of gcc, gcc9
* py-pycma: new package
* rename py-pycma in py-cma; py-cma: use pypi instead of github sources
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* lammps: add new stable version 20230802.1
* lammps: add missing potential download for +mesont
* lammps: fix python package install
* Update var/spack/repos/builtin/packages/lammps/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* lammps: py-numpy and py-mpi4py should be build and run deps
* lammps: add new 20231121 release
- MPIIO package has been removed -> disable mpiio variant
- LAMMPS_EXCEPTIONS is now always on -> disable exceptions variant
- CMake 3.16+ is now required
- Kokkos 4.1.0 is now supported
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add missing runtime dependency on py-colorama to py-ansimarkup
* Add py-metomi-isodatetime@3.1.0
* New package py-graphql-relay
* Update py-cylc-flowi, add version 8.2.3
* Fix merge conflict
* Revert mistake in var/spack/repos/builtin/packages/py-cylc-flow/package.py
* Update py-metomi-isodatetime dependencies for py-cylc-flow
* Add 'climbfuji' to list of maintainers for py-cylc-flow
* py-beartype: new package with version 0.15.0
* Update var/spack/repos/builtin/packages/py-beartype/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-beartype: depend on python 3.8 or higher
* py-beartype: add new version 0.16.2
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-mpi4py: fix build with Apple Clang
* [@spackbot] updating style on behalf of adamjstewart
---------
Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
* mochi-thallium: added a few newer versions
* mochi-thallium: added constraint on the version of margo required
* removed thallium 0.12 which needs updating
* mochi-thallium: fixed hash for version 0.12.0
* removed thallium 0.12 which needs updating (again)
When using `spack external find acfl`, we get the full version string
with 4 components in `packages.yaml`. This PR truncates the version
nubmer when finding the `armpl` component to be able to run without
intervention.
* developer tools stack try 2
This version is actually in use locally and has largely stabilized, at
least on x86. Some packages are still a challenge on ppc64le, but maybe
worth keeping this working as a set.
* add packages, try to get container with newer gcc
* remove reuse: true
* try to get cmake to build on medium, 25 minutes is too long
* add lsd package and add to dev tools stack
* clean up fzf dependency and sorting
* Update share/spack/gitlab/cloud_pipelines/stacks/developer_tools/spack.yaml
* cuda: add 12.3.0 (#40827)
* Switch to dashes
* yet more underscores
---------
Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
This looks to me like the best compromise regarding externals in a
build cache. I wouldn't want `spack install` on my machine to install
specs that were marked external on another. At the same time there are
centers that control the target systems on which spack is used, and
would want to use external in buildcaches.
As a solution, reuse concretization will now consider those externals
used in buildcaches that match a locally configured external in
packages.yaml.
So for example person A installs and pushes specs with this config:
```yaml
packages:
ncurses:
externals:
- spec: ncurses@6.0.12345 +feature
prefix: /usr
```
and person B concretizes and installs using that buildcache with the
following config:
```yaml
packages:
ncurses:
externals:
- spec: ncurses@6
prefix: /usr
```
the spec will be reused (or rather, will be considered for reuse...)
* gipaw.x installed by cmake if version >= 5c4a4ce.
gipaw.x will only be installed with cmake if the qe-gipaw version
is >= 5c4a4ce. Currently, QE source uses the older f5823521 one.
Here a patch to the submodule_commit_hash_records to use a newer
qe-gipaw version.
* initial commit to update hipblas rocalution, rocsolver, rocsparse to new syntax
* add rocblas test changes and fixes for hipblas and rocsolver tests
* fix styling
* remove updates for rocblas
* solver: use a unique counter for condition, triggers and effects
* Do not reset counters when re-running setup
What we need is just a unique ID, it doesn't need
to start from zero every time.
* Update SST packages to 13.1.0
* Allow mismatch between sst-core dependency and current macro version
* SST does not work with Python 3.12 yet
* Sanity check install binaries for sst-core
* Elements compiles with OTF2 but not OTF
* Version bounds in specs are inclusive
* Remove not-strictly-necessary file check
* oneapi 2024.0.0 release
* oneapi v2 directory support and some cleanups
* sycl abi change requires 2024 compilers for packages that use sycl
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* update spack recipe
* [@spackbot] updating style on behalf of toxa81
* change from @develop to @7.5.0
* return dependency on boost_filesystem
* return dependency on boost_filesystem
* remove boost filesystem as agreed by @RMeli and @simonpintarelli
---------
Co-authored-by: toxa81 <toxa81@users.noreply.github.com>
* Update to latest version
* Add dependency
* revert
* address PR comments
* Correct dependencies for 0.7 to 0.8 transition
* Fix cmake line.
* Update nanobind dep
---------
Co-authored-by: Matt Archer <ma595@cam.ac.uk>
Co-authored-by: Jack S. Hale <mail@jackhale.co.uk>
Co-authored-by: Garth N. Wells <gnw20@cam.ac.uk>
* add 0.12.0
* remove whitespace
* update deps
* Update var/spack/repos/builtin/packages/py-neo/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add dep for python 3.8+
* add dep for python 3.8+ with 0.12.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
PR #40929 reverted the argument parsing to make `spack --verbose
install` work again. It looks like `--verbose` is the only instance
where this kind of argument inheritance is used since all other commands
override arguments with the same name instead. For instance, `spack
--bootstrap clean` does not invoke `spack clean --bootstrap`.
Therefore, fix multi-line aliases again by parsing the resolved
arguments and instead explicitly pass down `args.verbose` to commands.
This commit discards type mismatches or failures to validate a package preference during concretization. The values discarded are logged as debug level messages. It also adds a config audit to help users spot misconfigurations in packages.yaml preferences.
This roughly restores the order of operation from Spack 0.20,
where where `AutotoolsPackage.setup_build_environment` would
override the env variable set in `setup_platform_environment` on
macOS.
When improving the error message, we started #showing in the
answer set a lot more symbols - but we forgot to suppress the
debug messages warning about UNKNOWN SYMBOLs
* onnxruntime: fix the call to as_string() operator
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-onnxruntime/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-onnxruntime: rm now-unused stringpiece_1_10.patch
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* packages/hub: add new version, update to module
Hub now uses a go module to build, needs different env vars, and we're
on a very, very old version before that. Deprecate the old ones so we
can clean out that old build once we pass a spack version.
* cleanup suggested by @adamjstewart
* Update to latest version
* Fix linebreak
* Make suggested changes
* bumped to 0.6.1
* Update var/spack/repos/builtin/packages/py-scikit-build-core/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Chris Richardson <cnr12@cam.ac.uk>
Co-authored-by: Matt Archer <ma595@cam.ac.uk>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
In practice, one can only compiler for the Intel Data Center Max GPU
via a SYCL build and the oneAPI compiler. This is unlikely to change,
so we can be explicit about that.
* elbencho add new version and git master branch
* Update var/spack/repos/builtin/packages/elbencho/package.py
Co-authored-by: Alec Scott <alec@bcs.sh>
* formatting fix requested by @alecbcs
* remove whitespace added in blank line by github auto resolve
---------
Co-authored-by: Ethan W <mail@ethanwilliams.xyz>
Co-authored-by: Alec Scott <alec@bcs.sh>
* gromacs: Add new variants and clarify existing ones
Add new variants that reflect existing capabilities and defaults in
the upstream build system. Add other existing constraints that were
not yet specified.
* conform to style
* Fix missing hyphens
* Correct cmake variable names
* add elephant version v0.12.0 and 0.13.0
* update copyright
* reformat according to black format errors
* restore maintainers directive
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add dependency python 3.8+
* sorted dependencies
* sort dependencies from newest to oldest
* add deps for @master
* removed dependency for master, since it is included in 0.12.0:
* removed dependency for python 3.7+ , since 3.7+ is the lowest supported version anyway
* removed specific deps for master, since master is always newer than all stable releases
* updated numpy dependency for Elephant 0.12.0:
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-elephant/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* removed upper bounds for py-quantities, omitting v0.14.0
* add elephant v0.14.0
* update required quantities version
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: add versions 2.0.0 2.0.1; add maintainers
* py-cleo: add forgotten dependence
* py-cleo: update from review: remove preferred version, remove a dependence, fix py-rapidfuzz version
* py-cleo: deprecated version 1.0.0a5; add version 1.0.0; update dependences
* py-cleo: add version 2.1.0; update version range of dependences
* py-crashtest: add version 0.4.1, dependence of py-cleo
* py-cleo: update dependence
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: update dependence py-clikit
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cleo: update dependence py-rapidfuzz
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-rapidfuzz: add version 2.2.0 dependence of py-cleo@2
* py-cleo: fix version range of py-crashtest
* py-rapidfuzz: fix dependences; add py-rapidfuzz-capi and py-jarowinkler
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Using Python 3.12 in a freshly cloned Spack repository results in
warnings such as this:
```
==> Warning: invalid escape sequence '\$'
==> Warning: invalid escape sequence '\('
==> Warning: invalid escape sequence '\.'
==> Warning: invalid escape sequence '\.'
```
These will turn into errors in 3.13, so fix them. All of them actually
do not need to be regexes, so convert them into normal strings.
* add version 0.14.1
* formatting
* style checks
* fix style errors
* remove old versions
* fix typo
* style
* update maintainers directive
* sort dependencies from newest to oldest
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* sort dependencies from newest to oldest
* removed upper bounds for python version
* Update var/spack/repos/builtin/packages/py-quantities/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* remove dependency on Python 3.7 +, since 3.7 is the lowest supported version anyway
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Added recent versions to ecflow/package.py, as well as added a cxxstd variant that is
needed to set BOOST_NO_CXX98_FUNCTION_BASE appropriately when building with C++17 standard.
* Fixed pep8 style error in the ecflow package.py script.
* Remov
* Removed cxxstd variant since the ecflow cmake configuration was already specifying
to use the c++17 standard for newer versions. The use of the BOOST_NO_CXX98_FUNCTION_BASE
define is now triggered by the ecflow version.
* Permit packages that depend on Intel oneAPI packages to access sdk
* Implement and use IntelOneapiLibraryPackageWithSdk
* Restore libs property to IntelOneapiLibraryPackage
* Conform to style
* Provide new class to infrastructure
* Treat sdk/include as the main include
* root: new version 6.30.00
There is a new release of ROOT, v6.30.00, with release notes at https://root.cern/doc/v630/release-notes.html.
In addition to some deprecations of build options, this updates the C++ standard to 17 or higher (well, 20), and increases the vc minimum version.
* vc: new version 1.4.4
* [@spackbot] updating style on behalf of wdconinc
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Improves the warning for deprecated preferences, and adds a configuration
audit to get files:lines details of the issues.
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* package/lemon: improve
* fix bug
* final improvements
* use f strings for boolean options, add soplex as TODO
* leave +coin as TODO
* depends on bzip2 when +coin
* tidy
---------
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
Co-authored-by: Sinan81 <Sinan81@users.noreply.github.com>
* initial commit to enable rocAL and add MIVisionX tests
* fix styling
* updated checksum for libjpeg patches
* update for 5.6
* use satisfies for checking spec version
xsdk: add +sycl variant - with amrex, arborx, ginkgo, petsc, sundials
xsdk: add +pflotran variant
xsdk: enable hypre+rocm
xsdk: enable superlu-dist for GPU - but use trilinos~superlu-dist [as that breaks builds]
xsdk: dealii: disable oce as it can cause intel-tbb-2017.6 to be picked up for some builds (for ex: gcc=13) and result in subsequent build failures
Some environments may have `dd4hep` as a concretized package without having it installed (yet). For those environments, `dd4hep` has property `libs` that is an empty list. Nevertheless, it can be added to a run environment (for example in case `dd4hep` is part of an environment). This results in an IndexError:
```
==> Warning: couldn't load runtime environment due to IndexError: list index out of range
```
To avoid the IndexError, only prepend the `dd4hep` libs if there are actually libs found.
Tests didn't cover the new `--variants-by-name` parameter in #40998.
Add some parameterization to hit that.
This changeset makes me think that the main section-printing loop in `spack info` isn't
factored so well. It makes it difficult to pass different arguments to different helper
functions. I could break it out into if statements if folks think that would be cleaner.
* dealii: 9.5.0
* kokkos+cuda_lambda
* dealii ^kokkos@3.7: require +cuda +cuda_lambda +wrapper
* Added 9.5.1, try ~cgal when +cuda
* Forward Cuda architecture request
* Remove workaround
* Try not enforcing the Kokkos compiler
* Enforce using nvcc_wrapper with Trilinos+Cuda
* Don't define CMAKE_*_COMPILER to point to MPI wrappers
* Use the same compiler as Trilinos/Kokkos
* Only check for Trilinos compiler
* Disable Trilinos+Cuda
* Disable Cuda support
* Try CUDA build without ninja
* Combined examples and examples_compile
* Use f-string for cuda_arch
* p -> _package
* Indentation
* Fix up f-string
---------
Co-authored-by: Luca Heltai <luca.heltai@sissa.it>
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
We have two ways to concretize now:
* `spack concretize` concretizes only the root specs that are not concrete in the environment.
* `spack concretize -f` eliminates all cached concretization data and reconcretizes the *entire* environment.
This PR adds `spack deconcretize`, which eliminates cached concretization data for a spec. This allows
users greater control over what is preserved from their `spack.lock` file and what is reused when not
using `spack concretize -f`. If you want to update a spec installed in your environment, you can call
`spack deconcretize` on it, and that spec and any relevant dependents will be removed from the lock file.
`spack concretize` has two options:
* `--root`: limits deconcretized specs to *specific* roots in the environment. You can use this to
deconcretize exactly one root in a `unify: false` environment. i.e., if `foo` root is a dependent
of `bar`, both roots, `spack deconcretize bar` will *not* deconcretize `foo`.
* `--all`: deconcretize *all* specs that match the input spec. By default `spack deconcretize`
will complain about multiple matches, like `spack uninstall`.
The ^mkl pattern was used to refer to three packages
even though none of software using it was depending
on "mkl".
This pattern, which follows Hyrum's law, is now being
removed in favor of a more explicit one.
In this PR gromacs, abinit, lammps, and quantum-espresso
are modified.
Intel packages are also modified to provide "lapack"
and "blas" together.
And improve the error message (load vs unload).
Of course you could have some uninstalled dependency too, but as long as
it doesn't implement `setup_run_environment` etc, I don't think it hurts
to attempt to load the root anyways, given that failure to do so is a
warning, not a fatal error.
This changes variant display to use a much more legible format, and to use screen space
much better (particularly on narrow terminals). It also adds color the variant display
to match other parts of `spack info`.
Descriptions and variant value lists that were frequently squished into a tiny column
before now have closer to the full terminal width.
This change also preserves any whitespace formatting present in `package.py`, so package
maintainers can make easer-to-read descriptions of variant values if they want. For
example, `gasnet` has had a nice description of the `conduits` variant for a while, but
it was wrapped and made illegible by `spack info`. That is now fixed and the original
newlines are kept.
Conditional variants are grouped by their when clauses by default, but if you do not
like the grouping, you can display all the variants in order with `--variants-by-name`.
I'm not sure when people will prefer this, but it makes it easier to tell that a
particular variant is/isn't there. I do think grouping by `when` is the better default.
* [lcov] Add build and runtime deps necessary for lcov@2.0.0:
+ Many additional Perl package dependecies are required for the new version of lcov.
+ Some of the new dependencies were not known to spack until now.
* Style fix
This commit improves forward compatibility of Spack with newer build cache metadata formats.
Before this commit, invalid or unrecognized metadata would be fatal errors, now they just cause
a mirror to be skipped.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Libgit2 requires python as build dependency. I was getting an error because it was falling back to system Python which is compiled with Intel compilers and thus, `libgit2` was failing because it couldn't find `libimf.so` (which doesn't make sense).
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
The libevent release tarballs ship with a `configure` script generated by an old `libtool`. The `libtool` generated by `configure` is not compatible with `MACOSX_DEPLOYMENT_VERSION` > 10. Regeneration of the `configure` scripts fixes build on macOS.
Original configure contains:
```
case $host_os in
rhapsody* | darwin1.[012])
_lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
darwin1.*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
darwin*) # darwin 5.x on
# if running on 10.5 or later, the deployment target defaults
# to the OS version, if on x86, and 10.4, the deployment
# target defaults to 10.4. Don't you love it?
case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
10.0,*86*-darwin8*|10.0,*-darwin[91]*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
10.[012][,.]*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
10.*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
esac
```
After re-running `autogen.sh`:
```
case $host_os in
rhapsody* | darwin1.[012])
_lt_dar_allow_undefined='$wl-undefined ${wl}suppress' ;;
darwin1.*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
darwin*)
case $MACOSX_DEPLOYMENT_TARGET,$host in
10.[012],*|,*powerpc*-darwin[5-8]*)
_lt_dar_allow_undefined='$wl-flat_namespace $wl-undefined ${wl}suppress' ;;
*)
_lt_dar_allow_undefined='$wl-undefined ${wl}dynamic_lookup' ;;
esac
```
* lcov: add version2, perl dep at build and runtime
* lcov: add runtime deps
* namespace-autoclean: new perl package
* datetime: dep on autoclean
* formatting
* abinit: add v9.10.3
Changed configure arguments for specfying how to use Wannier90 for versions
after 9.8.
When the mpi variant is requested, set the F90 environment variable to point
to the MPI Fortran wrapper when building versions after 9.8 instead of FC.
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 (#40676)
* exago: fix v1.5.1 tag; only allow python up to 3.10 for for @:1.5 due to pybind error with py 3.11
* hiop@:1.0 +cuda: constrain to cuda@:11.9
* fixes syntax of maintainers
---------
Co-authored-by: eugeneswalker <38933153+eugeneswalker@users.noreply.github.com>
* intel-xed: fix git hash for mbuild, add version 2023.10.11
Fixes#40912
* Fix the git commit hash for mbuild 2022.04.17. This was broken in
commit eef9939c21 by mixing up the hashes for xed versus mbuild.
* Add versions 2023.08.21 and 2023.10.11.
* fix style
Before this PR, variant were not propagated to leaf nodes that could accept
the propagated value, if some intermediate node couldn't accept it.
This PR fixes that issue by marking nodes as "candidate" for propagation
and by setting the variant only if it can be accepted by the node.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Problem: the current configure arguments are added lists to a list,
and this needs to be adding strings to the same list.
Solution: ensure we add each item (string) separately.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Co-authored-by: vsoch <vsoch@users.noreply.github.com>
* Use `major.minor.patch`, `major.minor`, `major` in tags
* Ensure `latest` is the semver largest version, and not "latest in time"
* Remove Ubuntu 18.04 from the list of images
Modify the packages.yaml schema so that soft-preferences on targets,
compilers and providers can only be specified under the "all" attribute.
This makes them effectively global preferences.
Version preferences instead can only be specified under a package
specific section.
If a preference attribute is found in a section where it should
not be, it will be ignored and a warning is printed to screen.
Most queries will end up calling `spec.satisfies(query)` on everything in the DB, which
will cause Spack to ask whether the query spec is virtual if its name doesn't match the
target spec's. This can be expensive, because it can cause Spack to check if any new
virtuals showed up in *all* the packages it knows about. That can currently trigger
thousands of `stat()` calls.
We can avoid the virtual check for most successful queries if we consider that if there
*is* a match by name, the query spec *can't* be virtual. This PR adds an optimization to
the query loop to save any comparisons that would trigger a virtual check for last.
- [x] Add a `deferred` list to the `query()` loop.
- [x] First run through the `query()` loop *only* checks for name matches.
- [x] Query loop now returns early if there's a name match, skipping most `satisfies()` calls.
- [x] Second run through the `deferred()` list only runs if query spec is virtual.
- [x] Fix up handling of concrete specs.
- [x] Add test for querying virtuals in DB.
- [x] Avoid allocating deferred if not necessary.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Currently there's some hacky logic in the AppleClang compiler that makes
it also accept `gfortran` as a fortran compiler if `flang` is not found.
This is guarded by `if sys.platform` checks s.t. it only applies to
Darwin.
But on Linux the feature of detecting mixed toolchains is highly
requested too, cause it's rather annoying to run into a failed build of
`openblas` after dozens of minutes of compiling its dependencies, just
because clang doesn't have a fortran compiler.
In particular in CI where the system compilers may change during system
updates, it's typically impossible to fix compilers in a hand-written
compilers.yaml config file: the config will almost certainly be outdated
sooner or later, and maintaining one config file per target machine and
writing logic to select the correct config is rather undesirable too.
---
This PR introduces a flag `spack compiler find --mixed-toolchain` that
fills out missing `fc` and `f77` entries in `clang` / `apple-clang` by
picking the best matching `gcc`.
It is enabled by default on macOS, but not on Linux, matching current
behavior of `spack compiler find`.
The "best matching gcc" logic and compiler path updates are identical to
how compiler path dictionaries are currently flattened "horizontally"
(per compiler id). This just adds logic to do the same "vertically"
(across different compiler ids).
So, with this change on Ubuntu 22.04:
```
$ spack compiler find --mixed-toolchain
==> Added 6 new compilers to /home/harmen/.spack/linux/compilers.yaml
gcc@13.1.0 gcc@12.3.0 gcc@11.4.0 gcc@10.5.0 clang@16.0.0 clang@15.0.7
==> Compilers are defined in the following files:
/home/harmen/.spack/linux/compilers.yaml
```
you finally get:
```
compilers:
- compiler:
spec: clang@=15.0.7
paths:
cc: /usr/bin/clang
cxx: /usr/bin/clang++
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
flags: {}
operating_system: ubuntu23.04
target: x86_64
modules: []
environment: {}
extra_rpaths: []
- compiler:
spec: clang@=16.0.0
paths:
cc: /usr/bin/clang-16
cxx: /usr/bin/clang++-16
f77: /usr/bin/gfortran
fc: /usr/bin/gfortran
flags: {}
operating_system: ubuntu23.04
target: x86_64
modules: []
environment: {}
extra_rpaths: []
```
The "best gcc" is automatically default system gcc, since it has no
suffixes / prefixes.
Add a new config section: `config:aliases`, which is a dictionary mapping aliases
to commands.
For instance:
```yaml
config:
aliases:
sp: spec -I
```
will define a new command `sp` that will execute `spec` with the `-I`
argument.
Aliases cannot override existing commands, and this is ensured with a test.
We cannot currently alias subcommands. Spack will warn about any aliases
containing a space, but will not error, which leaves room for subcommand
aliases in the future.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* Test that setup_run_environment changes to CC/CXX/FC/F77 are dropped in build env
* compilers set in run env shouldn't impact build
Adds `drop` to EnvironmentModifications courtesy of @haampie, and uses
it to clear modifications of CC, CXX, F77 and FC made by
`setup_{,dependent_}run_environment` routines when producing an
environment in BUILD context.
* comment / style
* comment
---------
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
This adds a rather trivial context manager that lets you deduplicate repeated
arguments in directives, e.g.
```python
depends_on("py-x@1", when="@1", type=("build", "run"))
depends_on("py-x@2", when="@2", type=("build", "run"))
depends_on("py-x@3", when="@3", type=("build", "run"))
depends_on("py-x@4", when="@4", type=("build", "run"))
```
can be condensed to
```python
with default_args(type=("build", "run")):
depends_on("py-x@1", when="@1")
depends_on("py-x@2", when="@2")
depends_on("py-x@3", when="@3")
depends_on("py-x@4", when="@4")
```
The advantage is it's clear for humans, the downside it's less clear for type checkers due to type erasure.
Create chains of causation for error messages.
The current implementation is only completed for some of the many errors presented by the concretizer. The rest will need to be filled out over time, but this demonstrates the capability.
The basic idea is to associate conditions in the solver with one another in causal relationships, and to associate errors with the proximate causes of their facts in the condition graph. Then we can construct causal trees to explain errors, which will hopefully present users with useful information to avoid the error or report issues.
Technically, this is implemented as a secondary solve. The concretizer computes the optimal model, and if the optimal model contains an error, then a secondary solve computes causation information about the error(s) in the concretizer output.
Examples:
$ spack solve hdf5 ^cmake@3.0.1
==> Error: concretization failed for the following reasons:
1. Cannot satisfy 'cmake@3.0.1'
2. Cannot satisfy 'cmake@3.0.1'
required because hdf5 ^cmake@3.0.1 requested from CLI
3. Cannot satisfy 'cmake@3.18:' and 'cmake@3.0.1
required because hdf5 ^cmake@3.0.1 requested from CLI
required because hdf5 depends on cmake@3.18: when @1.13:
required because hdf5 ^cmake@3.0.1 requested from CLI
4. Cannot satisfy 'cmake@3.12:' and 'cmake@3.0.1
required because hdf5 depends on cmake@3.12:
required because hdf5 ^cmake@3.0.1 requested from CLI
required because hdf5 ^cmake@3.0.1 requested from CLI
$ spack spec cmake ^curl~ldap # <-- with curl configured non-buildable and an external with `+ldap`
==> Error: concretization failed for the following reasons:
1. Attempted to use external for 'curl' which does not satisfy any configured external spec
2. Attempted to build package curl which is not buildable and does not have a satisfying external
attr('variant_value', 'curl', 'ldap', 'True') is an external constraint for curl which was not satisfied
3. Attempted to build package curl which is not buildable and does not have a satisfying external
attr('variant_value', 'curl', 'gssapi', 'True') is an external constraint for curl which was not satisfied
4. Attempted to build package curl which is not buildable and does not have a satisfying external
'curl+ldap' is an external constraint for curl which was not satisfied
'curl~ldap' required
required because cmake ^curl~ldap requested from CLI
$ spack solve yambo+mpi ^hdf5~mpi
==> Error: concretization failed for the following reasons:
1. 'hdf5' required multiple values for single-valued variant 'mpi'
2. 'hdf5' required multiple values for single-valued variant 'mpi'
Requested '~mpi' and '+mpi'
required because yambo depends on hdf5+mpi when +mpi
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo+mpi ^hdf5~mpi requested from CLI
3. 'hdf5' required multiple values for single-valued variant 'mpi'
Requested '~mpi' and '+mpi'
required because netcdf-c depends on hdf5+mpi when +mpi
required because netcdf-fortran depends on netcdf-c
required because yambo depends on netcdf-fortran
required because yambo+mpi ^hdf5~mpi requested from CLI
required because netcdf-fortran depends on netcdf-c@4.7.4: when @4.5.3:
required because yambo depends on netcdf-fortran
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo depends on netcdf-c
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo depends on netcdf-c+mpi when +mpi
required because yambo+mpi ^hdf5~mpi requested from CLI
required because yambo+mpi ^hdf5~mpi requested from CLI
Future work:
In addition to fleshing out the causes of other errors, I would like to find a way to associate different components of the error messages with different causes. In this example it's pretty easy to infer which part is which, but I'm not confident that will always be the case.
See the previous PR #34500 for discussion of how the condition chains are incomplete. In the future, we may need custom logic for individual attributes to associate some important choice rules with conditions such that clingo choices or other derivations can be part of the explanation.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This PR implements the concept of "default environment", which doesn't have to be
created explicitly. The aim is to lower the barrier for adopting environments.
To (create and) activate the default environment, run
```
$ spack env activate
```
This mimics the behavior of
```
$ cd
```
which brings you to your home directory.
This is not a breaking change, since `spack env activate` without arguments
currently errors. It is similar to the already existing `spack env activate --temp`
command which always creates an env in a temporary directory, the difference
is that the default environment is a managed / named environment named `default`.
The name `default` is not a reserved name, it's just that `spack env activate`
creates it for you if you don't have it already.
With this change, you can get started with environments faster:
```
$ spack env activate [--prompt]
$ spack install --add x y z
```
instead of
```
$ spack env create default
==> Created environment 'default in /Users/harmenstoppels/spack/var/spack/environments/default
==> You can activate this environment with:
==> spack env activate default
$ spack env activate [--prompt] default
$ spack install --add x y z
```
Notice that Spack supports switching (but not stacking) environments, so the
parallel with `cd` is pretty clear:
```
$ spack env activate named_env
$ spack env status
==> In environment named_env
$ spack env activate
$ spack env status
==> In environment default
```
* Add command suggestions
This adds suggestions of similar commands in case users mistype a
command. Before:
```
$ spack spack
==> Error: spack is not a recognized Spack command or extension command; check with `spack commands`.
```
After:
```
$ spack spack
==> Error: spack is not a recognized Spack command or extension command; check with `spack commands`.
Did you mean one of the following commands?
spec
patch
```
* Add package name suggestions
* Remove suggestion to run spack clean -m
This completes to `spack concretize`:
```
spack conc<tab>
```
but this still gets hung up on the difference between `concretize` and `concretise`:
```
spack -e . conc<tab>
```
We were checking `"$COMP_CWORD" = 1`, which tracks the word on the command line
including any flags and their args, but we should track `"$COMP_CWORD_NO_FLAGS" = 1` to
figure out if the arg we're completing is the first real command.
This PR adds support for including separate definitions from `spack.yaml`.
Supporting the inclusion of files with definitions enables user to make
curated/standardized collections of packages that can re-used by others.
Currently module globals aren't set before running
`setup_[dependent_]run_environment` to compute environment modifications
for module files. This commit fixes that.
Looking at the memory profiles of concurrent solves
for environment with unify:false, it seems memory
is only ramping up.
This exchange in the potassco mailing list:
https://sourceforge.net/p/potassco/mailman/potassco-users/thread/b55b5b8c2e8945409abb3fa3c935c27e%40lohn.at/#msg36517698
Seems to suggest that clingo doesn't release memory
until end of the application.
Since when unify:false we distribute work to processes,
here we give a maxtaskperchild=1, so we clean memory
after each solve.
Some providers must provide virtuals "together", i.e.
if they provide one virtual of a set, they must be the
providers also of the others.
There was a bug though, where we were not checking if
the other virtuals in the set were needed at all in
the DAG.
This commit fixes the bug.
* libtheora: regenerate Makefile.in during autoreconf
The patch to inhibit running of configure would exit autogen.sh so early
that it did not yet run autoconf/automake/...
Instead of patching autogen.sh, just pass -V as argument, as this is
passed on to configure and lets it just print its version instead of
configuring the build tree.
Also drop arguments from autogen.sh, as they are unused when configure
does not run.
* libtheora: fix build on macos
Apply upstream patches in order to avoid unresolved symbols during building of libtheoraenc.
These patches require re-running automake/autoconf/...
Error messages:
libtool: link: /Users/ma/git/spack/lib/spack/env/clang/clang -dynamiclib -o .libs/libtheoraenc.1.dylib .libs/apiwrapper.o .libs/fragment.o .libs/idct.o .libs/internal.o .libs/state.o .libs/quant.o .l
ibs/analyze.o .libs/fdct.o .libs/encfrag.o .libs/encapiwrapper.o .libs/encinfo.o .libs/encode.o .libs/enquant.o .libs/huffenc.o .libs/mathops.o .libs/mcenc.o .libs/rate.o .libs/tokenize.o -L/opt/spac
k/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib -logg -lm -Wl,-exported_symbols_list -Wl,/var/folders/zv/qr55pmd9065glf0mcltpx5bm000102/T/ma/spack-stage/spac
k-stage-libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/spack-src/lib/theoraenc.exp -install_name /opt/spack/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib
/libtheoraenc.1.dylib -compatibility_version 3 -current_version 3.2
ld: warning: search path '/opt/spack/darwin-sonoma-m1/apple-clang-15.0.0/libtheora-1.1.1-uflq3jvysewnrmlj5x5tvltst65ho3v4/lib' not found
ld: Undefined symbols:
_th_comment_add, referenced from:
_theora_comment_add in apiwrapper.o
_th_comment_add_tag, referenced from:
_theora_comment_add_tag in apiwrapper.o
_th_comment_clear, referenced from:
_theora_comment_clear in apiwrapper.o
_th_comment_init, referenced from:
_theora_comment_init in apiwrapper.o
_th_comment_query, referenced from:
_theora_comment_query in apiwrapper.o
_th_comment_query_count, referenced from:
_theora_comment_query_count in apiwrapper.o
* libtheora: add git versions
stable as version name for theora-1.1 branch was chosen so that it sorts between 1.1.x and master
* libtheora: remove unused patch
thanks to @michaelkuhn for noticing
* Add 2023.09-0 for x86_64, aarch64, and ppc64le
extend the anaconda3 package.py to support aarch64 and ppc64le.
add the latest version of anaconda3 to each new platform, including the existing x86_64
* formatting
* Fix py-pyside2 to build with newer llvm and to use spack libglx and libxcb headers where system headers are missing
pyside2 needs LLVM_INSTALL_DIR to be set when using llvm 11: and expects system headers for libglx and libxcb and won't build otherwise.
* Fix styling
* remove raw string type
* Update var/spack/repos/builtin/packages/py-pyside2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
This PR makes it possible to select only a subset of virtual dependencies from a spec that _may_ provide more. To select providers, a syntax to specify edge attributes is introduced:
```
hdf5 ^[virtuals=mpi] mpich
```
With that syntax we can concretize specs like:
```console
$ spack spec strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas
```
On `develop` this would currently fail with:
```console
$ spack spec strumpack ^intel-parallel-studio+mkl ^openblas
==> Error: Spec cannot include multiple providers for virtual 'blas'
Requested 'intel-parallel-studio' and 'openblas'
```
In package recipes, virtual specs that are declared in the same `provides` directive need to be provided _together_. This means that e.g. `openblas`, which has:
```python
provides("blas", "lapack")
```
needs to provide both `lapack` and `blas` when requested to provide at least one of them.
## Additional notes
This capability is needed to model compilers. Assuming that languages are treated like virtual dependencies, we might want e.g. to use LLVM to compile C/C++ and Gnu GCC to compile Fortran. This can be accomplished by the following[^1]:
```
hdf5 ^[virtuals=c,cxx] llvm ^[virtuals=fortran] gcc
```
[^1]: We plan to add some syntactic sugar around this syntax, and reuse the `%` sigil to avoid having a lot of boilerplate around compilers.
Modifications:
- [x] Add syntax to interact with edge attributes from spec literals
- [x] Add concretization logic to be able to cherry-pick virtual dependencies
- [x] Extend semantic of the `provides` directive to express when virtuals need to be provided together
- [x] Add unit-tests and documentation
* vcftools: adding new version 0.1.16
* Update var/spack/repos/builtin/packages/vcftools/package.py
Co-authored-by: Alec Scott <alec@bcs.sh>
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
* Make sure sio is in dependent build env for podio
* podio: Fix likely(?) typo in root dependency
* podio: Add latest tag and new variants + dependencies
* podio: Add v00-16-07 tag
* podio: Fix dependencies flagged by package audit
* podio: Simplify root dependency
* podio: Add 0.17.1 tag
Allowing white space around `:` in version ranges introduces an ambiguity:
```
a@1: b
```
parses as `a@1:b` but should really be parsed as two separate specs `a@1:` and `b`.
With white space disallowed around `:` in ranges, the ambiguity is resolved.
Call setup_dependent_run_environment on both link and run edges,
instead of only run edges, which restores old behavior.
Move setup_build_environment into get_env_modifications
Also call setup_run_environment on direct build deps, since their run
environment has to be set up.
* update SPERR package
* remove blank line
* update SPERR to be version 0.7.1
* a little clean up
* bound versions that require zstd
* add USE_ZSTD
* add libpressio-sperr version upbound
* update libpressio-sperr
* address review comments
* improve format
---------
Co-authored-by: Samuel Li <Sam@Navada>
Co-authored-by: Samuel Li <sam@cisl-m121a>
* Add tests to ensure variant propagation syntax can round-trip to/from string
* Add a regression test for the bug in 35298
* Reconstruct the spec constraints in the worker process
Specs do not preserve any information on propagation of variants
when round-tripping to/from JSON (which we use to pickle), but
preserve it when round-tripping to/from strings.
Therefore, we pass a spec literal to the worker and reconstruct
the Spec objects there.
* Added NVML support to the slurm package
* dbus package is required for cgroup support
* Fixing formatting
* Style fix
* Added PAM support
* Added ROCm SMI support
- [x] Add links to information people are going to want to know when adding license
information to their packages (namely OSI licenses and SPDX identifiers).
- [x] Update the packaging docs for `license()` with Spack as an example for `when=`.
After all, it's a dual-licensed package that changed once in the past.
- [x] Add link to https://spdx.org/licenses/ in the `spack create` boilerplate as well.
* Load the script file during enviroment setup so that all the enviroment variables are set properly
* Patch csh/tcsh so that it uses spacks via env
* Update SHA for latest version
* Extend shebang to perl and fix up the regex
* Add liggght patched for newer compiler
Add C++ 17 support
Add Clang and Oneapi support
* Add maintainers
* Fix format in liggghts
* Fix maintainers before versions
Co-authored-by: Alec Scott <alec@bcs.sh>
* Fix style and user to usr
* Update package.py
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
* [add] py-moarchiving: new package
* py-moarchiving: update from review: description, variant default value is False, switch when and type
---------
Co-authored-by: LydDeb <lyderic.debusschere@eolen.com>
* py-bluepyemodel: new package with dependencies
* py-morphio: add MPI as dependency to avoid failing builds
* Formatting
* py-bluepyefe: no need to set NEURON_INIT_MPI
* py-morphio: unifurcation branch is ancient history
* py-bluepyopt: only set NEURON_INIT_MPI with +neuron
* py-efel: get rid of old version
* py-morph{-tool,io}: rename develop to master to match branch
* py-bluepyefe: unset PMI_RANK is also neuron-related
* py-bluepyopt: PMI_RANK is also neuron-related
* Implement review remarks
* py-morph-tool, py-neurom: small fixes
* py-morphio: reword dependencies
Typically MSVC is detected via the VSWhere program. However, this may
not be available, or may be installed in an unpredictable location.
This PR adds an additional approach via Windows Registry queries to
determine VS install location root.
Additionally:
* Construct vs_install_paths after class-definition time (move it to
variable-access time).
* Skip over keys for which a user does not have read permissions
when performing searches (previously the presence of these keys
would have caused an error, regardless of whether they were
needed).
* Extend helper functionality with option for regex matching on
registry keys vs. exact string matching.
* Some internal refactoring: remove boolean parameters in some cases
where the function was always called with the same value
(e.g. `find_subkey`)
* e4s ci: add exago +cuda, +rocm builds
* exago: rename 5-18-2022-snapshot to snapshot.5-18-2022
* disable exago +rocm for non-external rocm ci install
* note that hiop +rocm fails to find hip libraries when they are spack-installed
.bat or .exe files can be considered executable on Windows. This PR
expands the regex for detectable packages to allow for the detection
of packages that vendor .bat wrappers (intel mpi for example).
Additional changes:
* Outside of Windows, when searching for executables `path_hints=None`
was used to indicate that default path hints should be provided,
and `[]` was taken to mean that no defaults should be chosen
(in that case, nothing is searched); behavior on Windows has
now been updated to match.
* Above logic for handling of `path_hints=[]` has also been extended
to library search (for both Linux and Windows).
* All exceptions for external packages were documented as timeout
errors: this commit adds a distinction for other types of errors
in warning messages to the user.
Credits to @ChristianKniep for advocating the idea of OCI image layers
being identical to spack buildcache tarballs.
With this you can configure an OCI registry as a buildcache:
```console
$ spack mirror add my_registry oci://user/image # Dockerhub
$ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR
$ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login credentials
```
which should result in this config:
```yaml
mirrors:
my_registry:
url: oci://ghcr.io/haampie/spack-test
push:
access_pair: [<username>, <password>]
```
It can be used like any other registry
```
spack buildcache push my_registry [specs...]
```
It will upload the Spack tarballs in parallel, as well as manifest + config
files s.t. the binaries are compatible with `docker pull` or `skopeo copy`.
In fact, a base image can be added to get a _runnable_ image:
```console
$ spack buildcache push --base-image ubuntu:23.04 my_registry python
Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
$ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
```
which should really be a game changer for sharing binaries.
Further, all content-addressable blobs that are downloaded and verified
will be cached in Spack's download cache. This should make repeated
`push` commands faster, as well as `push` followed by a separate
`update-index` command.
An end to end example of how to use this in Github Actions is here:
**https://github.com/haampie/spack-oci-buildcache-example**
TODO:
- [x] Generate environment modifications in config so PATH is set up
- [x] Enrich config with Spack's `spec` json (this is allowed in the OCI specification)
- [x] When ^ is done, add logic to create an index in say `<image>:index` by fetching all config files (using OCI distribution discovery API)
- [x] Add logic to use object storage in an OCI registry in `spack install`.
- [x] Make the user pick the base image for generated OCI images.
- [x] Update buildcache install logic to deal with absolute paths in tarballs
- [x] Merge with `spack buildcache` command
- [x] Merge #37441 (included here)
- [x] Merge #39077 (included here)
- [x] #39187 + #39285
- [x] #39341
- [x] Not a blocker: #35737 fixes correctness run env for the generated container images
NOTE:
1. `oci://` is unfortunately taken, so it's being abused in this PR to mean "oci type mirror". `skopeo` uses `docker://` which I'd like to avoid, given that classical docker v1 registries are not supported.
2. this is currently `https`-only, given that basic auth is used to login. I _could_ be convinced to allow http, but I'd prefer not to, given that for a `spack buildcache push` command multiple domains can be involved (auth server, source of base image, destination registry). Right now, no urllib http handler is added, so redirects to https and auth servers with http urls will simply result in a hard failure.
CAVEATS:
1. Signing is not implemented in this PR. `gpg --clearsign` is not the nicest solution, since (a) the spec.json is merged into the image config, which must be valid json, and (b) it would be better to sign the manifest (referencing both config/spec file and tarball) using more conventional image signing tools
2. `spack.binary_distribution.push` is not yet implemented for the OCI buildcache, only `spack buildcache push` is. This is because I'd like to always push images + deps to the registry, so that it's `docker pull`-able, whereas in `spack ci` we really wanna push an individual package without its deps to say `pr-xyz`, while its deps reside in some `develop` buildcache.
3. The `push -j ...` flag only works for OCI buildcache, not for others
* itk: patch missing include for newer compilers
* itk: The package doesn't use MPI
* itk: package requires the high-level hdf5 api
* itk: patch url with ?full_index=1
* itk: point to 4041 commit in master
* itk: don't constrain hdf5 with ~mpi
* rtmpdump: New package
* curl: Fix librtmp variant
Add the previously missing dependency required for rtmp support.
The variant has been broken since its addition in PR #25166.
Fixes one of the two issues reported in #26887.
* spack checksum pkg@1.2, use as version filter
Currently pkg@1.2 splits on @ and looks for 1.2 specifically, with this
PR pkg@1.2 is a filter so any matching 1.2, 1.2.1, ..., 1.2.10 version
is displayed.
* fix tests
* fix style
Update Tcl modulefile template to simplify generated `append-path`,
`prepend-path` and `remove-path` commands and improve their readability.
If path element delimiter is colon character, do not set the `--delim`
option as it is the default delimiter value.
Renames exclude_implicits to hide_implicits
When hide_implicits option is enabled, generate modulefile of
implicitly installed software and hide them. Even if implicit, those
modulefiles may be referred as dependency in other modulefiles thus they
should be generated to make module properly load dependent module.
A new hidden property is added to BaseConfiguration class.
To hide modulefiles, modulercs are generated along modulefiles. Such rc
files contain specific module command to indicate a module should be
hidden (for instance when using "module avail").
A modulerc property is added to TclFileLayout and LmodFileLayout classes
to get fully qualified path name of the modulerc associated to a given
modulefile.
Modulerc files will be located in each module directory, next to the
version modulefiles. This scheme is supported by both module tool
implementations.
modulerc_header and hide_cmd_format attributes are added to
TclModulefileWriter and LmodModulefileWriter. They help to know how to
generate a modulerc file with hidden commands for each module tool.
Tcl modulerc file requires an header. As we use a command introduced on
Modules 4.7 (module-hide --hidden-loaded), a version requirement is
added to header string.
For lmod, modules that open up a hierarchy are never hidden, even if
they are implicitly installed.
Modulerc is created, updated or removed when associated modulefile is
written or removed. If an implicit modulefile becomes explicit, hidden
command in modulerc for this modulefile is removed. If modulerc becomes
empty, this file is removed. Modulerc file is not rewritten when no
content change is detected.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* paraview: rebase the adios2 patch for 5.12-to-be
* paraview: disable fastfloat and token for 5.12-to-be
* paraview: require older protobuf for 5.12 as well
* paraview: require C++11-supporting protobuf for `master` too
* ci: don't register detectable compilers
Cause they go out of sync...
* remove intel compiler, it can be detected too
* Do not run spack compiler find since compilers are registered in concretize job already
* trilinos: work around +stokhos +cuda +superlu-dist bug due to EMPTY macro
Previously, we only searched for `patch` inside of whatever Git
installation was available because the most common installation of Git
available on Windows had `patch`. That's not true for all possible
installations of Git though, so this updates the search to also check
PATH.
GitLab's .patch URLs only provide abbreviated hashes, while .diff URLs
provide full hashes. There does not seem to be a parameter to force
.patch URLs to also return full hashes, so we should make sure to use
the .diff ones.
While e.g. GNU patch 2.7.6 (as provided by homebrew) would apply the previous
version of this patch without problems, Apple's patch 2.0-12u11-Apple fails
to find out which file to patch.
Adding two lines to the patch fixes that. Renamed the patch in order to
not require a `spack clean -m`.
* gromacs +cp2k: build in CI
* libxsmm: x86 only
* attempt to fix dbcsr + new mpich
* use c11 standard
* gromacs: does not depend on dbcsr
* cp2k: build with cmake in CI, s.t. dbcsr is a separate package
* cp2k: cmake patches for config files and C/C++ std
* cp2k: remove unnecessary constraints due to patch
With the introduction of multiple build dependencies from the same package in the DAG, we need to minimize a few weights accounting for edges rather than nodes. If we don't do that we might have multiple "optimal" solutions that differ only in how the same nodes are connected together. This commit ensures optimal versions are picked per parent in case of multiple choices for a dependency.
* py-statsmodels: add 0.14.0
* Fix style
* Update var/spack/repos/builtin/packages/py-statsmodels/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-statsmodels/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Remove python limits
* Remove comment
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cykhash: adding new package py-cykhash
* py-hmmlearn: adding new package py-hmmlearn
* py-macs3: adding new package py-macs3
* py-macs3: adding python version restriction and other changes.
Fix the following syntax which validates only the first array entry:
```python
"compilers": {
"type": "array",
"items": [
{
"type": ...
}
]
}
```
to
```python
"compilers": {
"type": "array",
"items": {
"type": ...
}
}
```
which validates the entire array.
Oops...
On some systems, multiple pythonx.y are placed in the same prefix as
pythonx (where only one of them is associated with that pythonx).
Spack external detection for Python was willing to register all of
these as external versions. Moreover, the `package.py` for Python
was able to distinguish these.
This can cause an issue for some build systems, which will just look
for python3 for example, so if that python3 is actually python3.6,
and the build system needs 3.7 (which spack may have found in the
same prefix, and offered as a suitable external), it will fail when
invoking python3.
To avoid that issue, we simply avoid treating pythonx.y as external
candidates. In the above case, Spack would only detect a Python 3.6
external, and the build would be forced to use a Spack-built Python
3.7 (which we consider a good thing).
This adds a `SetupContext` class which is responsible for setting
package.py module globals, and computing the changes to environment
variables for the build, test or run context.
The class uses `effective_deptypes` which takes a list of specs (e.g. single
item of a spec to build, or a list of environment roots) and a context
(build, run, test), and outputs a flat list of specs that affect the
environment together with a flag in what way they do so. This list is
topologically ordered from root to leaf, so that one can be assured that
dependents override variables set by dependencies, not the other way
around.
This is used to replace the logic in `modifications_from_dependencies`,
which has several issues: missing calls to `setup_run_environment`, and
the order in which operations are applied.
Further, it should improve performance a bit in certain cases, since
`effective_deptypes` run in O(v + e) time, whereas `spack env activate`
currently can take up to O(v^2 + e) time due to loops over roots. Each
edge in the DAG is visited once by calling `effective_deptypes` with
`env.concrete_roots()`.
By marking and propagating flags through the DAG, this commit also fixes
a bug where Spack wouldn't call `setup_run_environment` for runtime
dependencies of link dependencies. And this PR ensures that Spack
correctly sets up the runtime environment of direct build dependencies.
Regarding test dependencies: in a build context they are are build-time
test deps, whereas in a test context they are install-time test deps.
Since there are no means to distinguish the build/install type test deps,
they're both.
Further changes:
- all `package.py` module globals are guaranteed to be set before any of the
`setup_(dependent)_(run|build)_env` functions is called
- traversal order during setup: first the group of externals, then the group
of non-externals, with specs in each group traversed topological (dependencies
are setup before dependents)
- modules: only ever call `setup_dependent_run_environment` of *direct* link/run
type deps
- the marker in `set_module_variables_for_package` is dropped, since we should
call the method once per spec. This allows us to set only a cheap subset of
globals on the module: for example it's not necessary to compute the expensive
`cmake_args` and w/e if the spec under consideration is not the root node to be
built.
- `spack load`'s `--only` is deprecated (it has no effect now), and `spack load x`
now means: do everything that's required for `x` to work at runtime, which
requires runtime deps to be setup -- just like `spack env activate`.
- `spack load` no longer loads build deps (of build deps) ...
- `spack env activate` on partially installed or broken environments: this is all
or nothing now. If some spec errors during setup of its runtime env, you'll only
get the unconditional variables + a warning that says the runtime changes for
specs couldn't be applied.
- Remove traversal in upward direction from `setup_dependent_*` in packages.
Upward traversal may iterate to specs that aren't children of the roots
(e.g. zlib / python have hundreds of dependents, only a small fraction is
reachable from the roots. Packages should only modify the direct dependent
they receive as an argument)
The ability to select the top N versions got removed in the checksum overhaul,
cause initially numbers were used for commands.
Now that we settled on characters for commands, let's make numbers pick the top
N again.
This flag was only relevant when targeting powerpc from apple-clang,
which we don't do. The flag is removed from apple-clang@15. Let's drop
it unconditionally.
Improve how mirrors are used in gitlab ci, where we have until now thought
of them as only a string.
By configuring ci mirrors ahead of time using the proposed mirror templates,
and by taking advantage of the expressiveness that spack now has for mirrors,
this PR will allow us to easily switch the protocol/url we use for fetching
binary dependencies.
This change also deprecates some gitlab functionality and marks it for
removal in Spack 0.23:
- arguments to "spack ci generate":
* --buildcache-destination
* --copy-to
- gitlab configuration options:
* enable-artifacts-buildcache
* temporary-storage-url-prefix
* petsc: add variant +sycl
* petsc: add in gmake as dependency - so that consistent make gets used between petsc and slepc builds [that can have different env for each of the builds]
Reused specs used to be referenced directly into the built spec.
This might cause issues like in issue 39570 where two objects in
memory represent the same node, because two reused specs were
loaded from different sources but referred to the same spec
by DAG hash.
The issue is solved by copying concrete specs to a dictionary keyed
by dag hash.
`spack dev-build` would incorrectly set `keep_stage=True` for the
entire DAG, including for non-dev specs, even though the dev specs
have a DIYStage which never deletes sources.
py-werkzeug@:0.12 does not work with python@3.10:
Test with py-werkzeug 0.12.2 and python 3.10:
```
$ python3.10 -c 'import werkzeug'
py-werkzeug-0.12.2/lib/python3.11/site-packages/werkzeug/datastructures.py", line 16, in <module>
from collections import Container, Iterable, MutableSet
ImportError: cannot import name 'Container' from 'collections'
```
Test with py-werkzeug 0.12.2 and python 3.9:
```
python3.9 -c "from collections import Container"
<string>:1: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
```
This patch adds in a license directive to get the ball rolling on adding in license
information about packages to spack. I'm primarily interested in just adding
license into spack, but this would also help with other efforts that people are
interested in such as adding license information to the ASP solve for
concretization to make sure licenses are compatible.
Usage:
Specifying the specific license that a package is released under in a project's
`package.py` is good practice. To specify a license, find the SPDX identifier for
a project and then add it using the license directive:
```python
license("<SPDX Identifier HERE>")
```
For example, for Apache 2.0, you might write:
```python
license("Apache-2.0")
```
Note that specifying a license without a when clause makes it apply to all
versions and variants of the package, which might not actually be the case.
For example, a project might have switched licenses at some point or have
certain build configurations that include files that are licensed differently.
To account for this, you can specify when licenses should be applied. For
example, to specify that a specific license identifier should only apply
to versionup to and including 1.5, you could write the following directive:
```python
license("MIT", when="@:1.5")
```
`xmllint` is called by `xmlto` during generation of `libzmq`'s docs, so
adding `libxml2`.
The docbook deps and the patches are taken from
https://src.fedoraproject.org/rpms/xmlto/blob/rawhide/f/xmlto.spec
There are still many more dependencies missing, but this is out of scope
of this patch (which is only concerned about the use case of `libzmq`).
This commit allows version specifiers to refer to git branches that contain
forward slashes. For example, the following is valid syntax now:
pkg@git.releases/1.0
It also adds a new method `Spec.format_path(fmt)` which is like `Spec.format`,
but also maps unsafe characters to `_` after interpolation. The difference is
as follows:
>>> Spec("pkg@git.releases/1.0").format("{name}/{version}")
'pkg/git.releases/1.0'
>>> Spec("pkg@git.releases/1.0").format_path("{name}/{version}")
'pkg/git.releases_1.0'
The `format_path` method is used in all projections. Notice that this method
also maps `=` to `_`
>>> Spec("pkg@git.main=1.0").format_path("{name}/{version}")
'pkg/git.main_1.0'
which should avoid syntax issues when `Spec.prefix` is literally copied into a
Makefile as sometimes happens in AutotoolsPackage or MakefilePackage
Currently `spack env activate --with-view` exists, but is a no-op.
So, it is not too much of a breaking change to make this redundant flag
accept a value `spack env activate --with-view <name>` which activates
a particular view by name.
The view name is stored in `SPACK_ENV_VIEW`.
This also fixes an issue where deactivating a view that was activated
with `--without-view` possibly removes entries from PATH, since now we
keep track of whether the default view was "enabled" or not.
A few packages have encoded an idiom that pre-dates the introduction
of the 'requires' directive, and they cycle over all compilers
to conflict with the ones that are not supported.
Here instead we reverse the logic, and require the ones that
are supported.
* Added initial package for building Beatnik with spack
* Fixed github ID for Jason as a maintainer.
* Major revision of beatnik spack package to properly support GPU spack builds with CUDA (and ROCm, though that it untested)
* Marked that beatnik 1.0 will require cabana 0.6.0. We will wait for the cabana 0.6.0 release before we release beatnik
* Update to beatnik package spec to compile with hipcc when +rocm
* Updated spack package for cabana for version 0.6.0 and appropriate heffte dependency
* Updated beatnik package to require cabana 0.6.0
* More updates to cabana and beatnik to build with cabana 0.6.0
* Finish removing BLT dependence from beatnik
* More updates to beatnik package for compiling on cray systems
* Updated beatnik package for new cabana package
* Changes to silo package for new silo version
* Fixed version specs for heffte to be able to concretize and build
* Fixed spack style issues for beatnik and silo packages
* More spack formatting fixes to beatnik and silo
* Patrick adopting silo package as maintainer for now
* Should address final style changes to beatnik package spec
* Yet more style fixes.
* Perhaps this is the final style fixes? :)
* Minor fix to cabana package on required versions
* Updating patch to add flag mcode-object-version=none when
device libs is buils as part of llvm-amdgpu
* Limiting patch to +rocm-device-libs variant and adding
appropriate comment for the patch
* Updating llvmpatch as per the mailine code
Updating hsa-rocr patch as per the latest code
Updating the if elif condition for the hip test src path
* Updating flags for 5.5 relases and above
* Updating build flags and patches
* Fix version incompatibilities of py-pandas and py-openpyxl
* Add variant excel for py-pandas
* Add package py-pyxlsb
* Add versios for py-xlsxwriter
* Define excel dependencies for py-pandas 1.4, 1.5, 2.0, 2.1
* Fix variant excel in py-pandas
* Add package py-odfpy, which is also a dependency for py-pandas@2.0:
* Rearrange excel dependencies for py-pandas
* Change url to pypi
* Add missing newline to fix style in py-odfpy
* Uodate for Basix 0.7
* Version fix for nanobind dependency
* Simplification
* Version update and simplify dependencies
* Add comment on location of pyproject.toml
* Update var/spack/repos/builtin/packages/py-fenics-basix/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
This tweaks the matrix description to indicate that it's bridged with Slack. So people
don't think they're missing out (even though the icon says there are only 3 users on
Matrix).
* Bug fix in var/spack/repos/builtin/packages/py-awscrt/package.py: on Linux, tell aws-crt-python to use libcrypto from spack (openssl)
* Bug fix in var/spack/repos/builtin/packages/py-awscrt/package.py: add missing build dependencies cmake (for all), openssl (for linux)
* Update var/spack/repos/builtin/packages/py-awscrt/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [add] py-graphene-tornado: new recipe, required by py-cylc-uiserver
* py-graphene-tornado: Taking reviewing into account
* py-graphene-tornado: add type run in dependences py-jinja, py-tornado and py-werkzeug
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: LydDeb <lyderic.debusschere.tgcc@cea.fr>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-urllib3: add 2.0.5
* Add py-brotli package
* Group brotli dependencies and make limits more specific
* Add minimum version limits to variants
* Remove python upper limit for py-brotli
* Fix restrictions for py-brotli dependency
* Fix py-brotli dependency
* py-urllib3: add 2.0.6
* New package: cpr
* Support libcpr version 1.9
* Fix build phase for git
* Update var/spack/repos/builtin/packages/cpr/package.py
Co-authored-by: Alec Scott <alec@bcs.sh>
* [@spackbot] updating style on behalf of sethrj
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
* wayland: dot is a build dependency
otherwise this build failure happens:
../spack-src/doc/meson.build:5:6: ERROR: Program 'dot' not found or not executable
* wayland: make building of documentation optional
renders several dependencies optional
* spack checksum: improve interactive filtering
* fix signature of executable
* Fix restart when using editor
* Don't show [x version(s) are new] when no known versions (e.g. in spack create <url>)
* Test ^D in test_checksum_interactive_quit_from_ask_each
* formatting
* colorize / skip header on invalid command
* show original total, not modified total
* use colify for command list
* Warn about possible URL changes
* show possible URL change as comment
* make mypy happy
* drop numbers
* [o]pen editor -> [e]dit
* Fpocket: fix edit() positional args + add install()
* Remove comments
* Fix line too long
* Fix line too long
* Remove extension specification in version
Co-authored-by: Alec Scott <alec@bcs.sh>
* Use f-strings
Co-authored-by: Alec Scott <alec@bcs.sh>
* Fix styling
* Use the default MakefilePackage install stage
---------
Co-authored-by: Alec Scott <alec@bcs.sh>
A complete texinfo install includes both `info` and `makeinfo`. Some
system installations of texinfo may exclude one or the other. This
updates the external finding logic to require both.
* openimagedenoise: checksum 2.0.1
* ospray: new versions 2.11.0 and 2.12.0
- both depend on embree@4
- also update dependency versions for rkcommon, openvkl, openimagedenois and ispc
- expose that dependency on openvkl is optional since @2.11 with variant "volumes"
* ospray: limit embree to @3 for ospray @:2.10
After the merge of #37957 (Add static and pic variants), if a gettext install
from a build before that merge is present, building any package using gettext
fails with keyerror: "shared" because the use of self.spec.variants["shared"]
does not check for the presence of the new variant in the old installation
but expects that the new key variants["shared"] exists always.
Fix it with a fallback to the default of True and update gettext to v22.3
Co-authored-by: Bernharad Kaindl <43588962+bernhardkaindl@users.noreply.github.com>
* initial commit to fix ck build for 5.6.1
* disable mlir for miopen-hip
* use satisfies for checking specs and add nlohmann-json dependency for 5.4 onwards
Because those end up being passed to ar which does not understand linker
arguments. This was making ldflags largely unusuable for statically
linked cmake packages.
* py-jupyter-packaging: remove duplicate packages
* Allow py-jupyter-packaging to be duplicated in DAG
* Deprecate version of py-jupyterlab that requires py-jupyter-packaging at run-time
* [py-torch-sparse] New version 0.6.17
* [py-torch-sparse] added dependency on parallel-hashmap
* [py-torch-sparse]
- spack only supports python@3.7:
- py-pytest-runner only needed with old versions
You will note the `Cite this repository` link is not working.
This commit fixes the underlying file...
* `authors` was not indented
* `authors` required by `preferred-citation`
* `authors` list required at top level (I simply duplicated)
* `"USA"` not correct country code
* `month` requires an integer month number
* Added URL to the actual pdf of the cited paper
* Used `identifiers` for doi and LLNL doc number
* added `abstract` copied from paper
Various fixes were confirmed by `cffconvert` using `docker run -v `pwd`:/app citationcff/cffconvert --validate`
* [add] py-metomi-rose: new recipe, required by py-cylc-rose
* py-metomi-rose: remove version constraint on python
---------
Co-authored-by: LydDeb <lyderic.debusschere.tgcc@cea.fr>
* Allow branching out of the "generic build" unification set
For cases like the one in https://github.com/spack/spack/pull/39661
we need to relax rules on unification sets.
The issue is that, right now, nodes in the "generic build" unification
set are unified together with their build dependencies. This was done
out of caution to avoid the risk of circular dependencies, which would
ultimately cause a very slow solve.
For build-tools like Cython, however, the build dependencies is masked
by a long chain of "build, run" dependencies that belong in the
"generic build" unification space.
To allow splitting on cases like this, we relax the rule disallowing
branching out of the "generic build" unification set.
* Fix issue with pure build virtual dependencies
Pure build virtual dependencies were not accounted properly in the
list of possible virtuals. This caused some facts connecting virtuals
to the corresponding providers to not be emitted, and in the end
lead to unsat problems.
* Fixed a few issues in packages
py-gevent: restore dependency on py-cython@3
jsoncpp: fix typo in build dependency
ecp-data-vis-sdk: update spack.yaml and cmake recipe
py-statsmodels: add v0.13.5
* Make dependency on "blt" of type "build"
We run pip with `--no-build-isolation` because we don't wanna let pip
install build deps.
As a consequence, when pip runs hooks, it runs hooks of *any* package it
can find in `sys.path`.
For Spack-built Python this includes user site packages -- there
shouldn't be any system site packages. So in this case it suffices to
set the environment variable PYTHONNOUSERSITE=1.
For external Python, more needs to be done, cause there is no env
variable that disables both system and user site packages; setting the
`python -S` flag doesn't work because pip runs subprocesses that don't
inherit this flag (and there is no API to know if -S was passed)
So, for external Python, an empty venv is created before invoking pip in
Spack's build env ensures that pip can no longer see anything but
standard libraries and `PYTHONPATH`.
The downside of this is that pip will generate shebangs that point to
the python executable from the venv. So, for external python an extra
step is necessary where we fix up shebangs post install.
* Add new package awscli-v2 and its missing dependency awscrt
* Remove boilerplate comments from awscli-v2 and awscrt packages
* Fix typos in var/spack/repos/builtin/packages/awscli-v2/package.py
* Update var/spack/repos/builtin/packages/awscli-v2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/awscli-v2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/awscli-v2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/awscli-v2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/awscli-v2/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Address reviewer comments
* Remove py-pip version dependency from var/spack/repos/builtin/packages/awscli-v2/package.py
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-isort: needs setuptools build dep before v5
Detected in #40224.
In the past, system setuptools could be picked up when using an external
python, so py-isort@4 would install fine. With the linked PR, pip can
only consider packages that Spack controls from PYTHONPATH, so the issue
of missing py-setuptools showed up.
* py-importlib-metadata: fix lowerbounds on python
* review
* py-isort unconditionally add optional setuptools dep to prevent picking up user package at runtime
* style
* drop optional py-setuptools run dep
* e4s amd64 gcc ci stack: sync with e4s-23.08
* e4s amd64 oneapi ci stack: sync with e4s-23.08
* e4s ppc64 gcc ci stack: sync with e4s-23.08
* add new ci stack: e4s amd64 gcc w/ external rocm
* add new ci stack: e4s arm gcc ci
* updates
* py-scipy: -fvisibility issue is resolved in 2023.1.0: #39464
* paraview oneapi fails
* comment out pkgs that fail to build on power
* fix arm stack name
* fix cabana +cuda specification
* comment out failing spces
* visit fails build on arm
* comment out slepc arm builds due to make issue
* comment out failing dealii arm builds
* Fix python versioning issue for py-biom-format
* Update deps according to feedback
* Remove version requirement for py-cython
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Only depend on py-six for versions >=2.1.10
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add py-future as non-dependency for 2.1.15
* There we are. Everything anyone could ever want
* Missed cython version change
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-dipy: Update version to support python@3.10
py-dipy only adds support to python@3.10 in py-dipy@1.5.0
See #40228
* py-dipy: fix formatting issues
* py-dipy: another formatting fix
* py-dipy: Use depends instead of conflicts
* py-dipy: formatting fix
* py-dipy: Updating for @1.7.0
Added new minimum version requirements for
py-cython
py-numpy
py-scipy
py-h5py
as suggested by @manuelakuhn
(py-nibabel min version unchanged for @1.7.0)
* [add] py-graphql-relay: new package
* py-graphql-relay: Update package.py
Remove leftovers from version 3.2.0:
* archive name in pypi
* dependencies
* [fix] py-graphql-relay: remove constraint on python version; add dependence py-rx because of ModuleNotFoundError during spack test
* [fix] py-graphql-relay: remove py-rx dependence; py-graphql-core: add dependencies for version 2.3.2
* py-graphql-core: Update from review; set build backend, py-poetry for version 3: and py-setuptools for version 2
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Two changes in this PR:
1. Register absolute paths in tarballs, which makes it easier
to use them as container image layers, or rootfs in general, outside
of Spack. Spack supports this already on develop.
2. Assemble the tarfile entries "by hand", which has a few advantages:
1. Avoid reading `/etc/passwd`, `/etc/groups`, `/etc/nsswitch.conf`
which `tar.add(dir)` does _for each file it adds_
2. Reduce the number of stat calls per file added by a factor two,
compared to `tar.add`, which should help with slow, shared filesystems
where these calls are expensive
4. Create normalized `TarInfo` entries from the start, instead of letting
Python create them and patching them after the fact
5. Don't recurse into subdirs before processing files, to avoid
keeping nested directories opened. (this changes the tar entry
order slightly, it's like sorting by `(not is_dir, name)`.
* 5.6.0 updates
* Rocm 5.6.0 updates
* Style and audit corrections for 5.6
* Patching smi path for tests.
* Style correction
* 5.6.1 updates
* Updated hip tests for ci build failure
Updated hiprand with the release tag
Taken care the review comment rocsolver
* Adding rocm-smi path for 5.6
* Adding the patch file
* Setting library directory uniform
* gl depends on mesa but it should not be llvm variant
* Fix for the issue 39520 by setting CMAKE_INSTALL_LIBDIR=lib
* i1 muls can sometimes happen after SCEV. They resulted in
ISel failures because we were missing the patterns for them.
* 5.6.0 & 5.6.1 updates for migraphx, miopen-hip, mivisionx
* Revert "5.6.0 & 5.6.1 updates for migraphx, miopen-hip, mivisionx"
This reverts commit f54c9c6c67.
* Revert operator mixup fix
* Splitting compiler-rt-linkage-for-host and operator mixup patch
* Adding missing patch for reverting operator mixup
* 5.6 update for composable-kernel,migraphx,miopen-hip and mivisionx
* Updating rvs, rcd and rccl for 5.6.1. adding comment for llvm patch
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* CGNS: If fortran not enabled, do not specify parallel fortran compiler
* Update package formatting as suggested by black
* Accept suggested change
Fix issue in configure_args which resulted in duplicate "--with-ldflags" arguments (with different values) being passed to configure. And extended the fix to similar arguments.
Also, repeated some flags to "--with-libs" to "--with-ldflags" as when the flags were only in "--with-libs", they did not seem to be picked up everywhere. I suspect this is a bug in the configure script, but adding to both locations seems to solve it and should not have any adverse effects.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
- upstream patch does not apply cleanly to older versions, not required
for newer ones
- also add conflict for older versions, except for 3.13.3 which works by
chance
For a long time, the docs have generated a huge, static HTML package list. It has some
disadvantages:
* It's slow to load
* It's slow to build
* It's hard to search
We now have a nice website that can tell us about Spack packages, and it's searchable so
users can easily find the one or two packages out of 7400 that they're looking for. We
should link to this instead of including a static package list page in the docs.
- [x] Replace package list link with link to packages.spack.io
- [x] Remove `package_list.html` generation from `conf.py`.
- [x] Add a new section for "Links" to the docs.
- [x] Remove docstring notes from contribution guide (we haven't generated RST
for package docstrings for a while)
- [x] Remove referencese to `package-list` from docs.
Currently, Windows SDK detection will only pick up SDK versions
related to the current version of Windows Spack is running on.
However, in some circumstances, we want to detect other version
of the SDK, for example, for compiling on Windows 11 for Windows
10 to ensure an API is compatible with Win10.
* Make use of `prefix` in the Cray manifest schema (prepend it to
the relative CC etc.) - this was a Spack error.
* Warn people when wrong-looking compilers are found in the manifest
(i.e. non-existent CC path).
* Bypass compilers that we fail to add (don't allow a single bad
compiler to terminate the entire read-cray-manifest action).
* Refactor Cray manifest tests: module-level variables have been
replaced with fixtures, specifically using the `test_platform`
fixture, which allows the unit tests to run with the new
concretizer.
* Add unit test to check case where adding a compiler raises an
exception (check that this doesn't prevent processing the
rest of the manifest).
If you `spack install x ^y` where `y` is a pure build dep of `x`, and
then uninstall `y`, and then `spack install --overwrite x ^y`, the build
fails because `y` is not re-installed.
Same can happen when you install a develop spec, run `spack gc`,
modify sources, and install again; develop specs rely on overwrite
install to work correctly.
-- Performaing formatting changes
-- Formatting file to conform with spack style
-- Adding updates from review
-- Removing old release candidates from the specification
-- Adding external conduit support for Catalyst
-- Adding Catalyst to `CMAKE_PREFIX_PATH` for the test to find
This PR adds a new audit sub-command to check that detection of relevant packages
is performed correctly in a few scenarios mocking real use-cases. The data for each
package being tested is in a YAML file called detection_test.yaml alongside the
corresponding package.py file.
This is to allow encoding detection tests for compilers and other widely used tools,
in preparation for compilers as dependencies.
Add versions 2020.08.1 and branch 2023.08.stable. Note: the version
numbers are a little different. Here, 2023.08.1 means release no. 1
from the release/2023.08 branch.
Modifications:
- [x] Move `spack.util.string` to `llnl.string`
- [x] Remove dependency of `llnl` on `spack.error`
- [x] Move path of `spack.util.path` to `llnl.path`
- [x] Move `spack.util.environment.get_host_*` to `spack.spec`
This resolves an interesting circular dependency between gcc and glibc:
1. glibc < 2.17 depends on libgcc.a and libgcc_eh.a
2. libgcc_eh.a is only built when gcc is configured with
--enable-shared
3. but building shared libraries requires crt*.o and libc.so
Backport AT_RANDOM auxval changes to avoid dealing with wrong inline
assembly (fallback code fails on ubuntu 23.04)
Update Trilinos and dependencies to build a limited version of Trilinos
on Windows.
* Support trilinos~mpi~shared on Windows
* superlu: force CMake build on Windows
* boost: update to build on Windows (proper option formatting and
build tool names)
* pcre, openblas: add CMake-based build (keep prior build system
as default on platforms other than Windows)
* openblas: add patch when using Intel Fortran compiler (currently
this is included as part of the hybrid %msvc compiler in Spack)
Co-authored-by: John Parent <john.parent@kitware.com>
As reported in #40159, a shared library build of ffmpeg 6.0 fails with the linker that was added with XCode 15:
ld: building exports trie: duplicate symbol '_av_ac3_parse_header'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Forcing the old linker with -Wl,-ld_classic works around this.
VTK's (and therefore Paraview's) FindFreetype module required patching to
handle static import libraries from Freetype. However it did not cover
shared libraries. This adds support for importing shared freetype into the VTK build
* Rename var/spack/repos/builtin/packages/python/cray-rpath-3.1.patch as var/spack/repos/builtin/packages/python/rpath-non-gcc.patch and apply unconditionally
* Update var/spack/repos/builtin/packages/python/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-numcodecs: drop upperbound, add new version, avoid native compilation
* py-numcodecs: add entrypoints
* Remove another upperbound on python
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
The fix for the compile issue was improved by Bernhard Kaindl.
He also added to fix two classes of other build fails:
- add missing openssl dependency version limit to older openssh versions
- add missing -fcommon for new compilers building old openssh versions
Co-authored-by: snehring <snehring@iastate.edu>
* e4s cray rhel stack: expand to full spec list
* comment out gasnet; require %gcc for unzip
* require openblas@0.3.20 to get around %cce error; follow up with issue report
* comment out failing specs;
* comment out axom and xyce due to errors
* improve clarity of failing specs
* msvc.py: don't import distutils
Introduced in #27021, makes Spack forward incompatible with Python.
The module was already deprecated at the time of the PR.
* update spack package
* py-gpy: drop cython induced python upperbound
* py-gpy: bump scipy
* py-fn-py: python bounds for old version, new version w/o
* py-statsmodels: force recythonization
* py-gpy: clarifying comment about cython build type
* py-aiofiles: Add version 0.7.0 required by py-cylc-flow
* py-aiofiles: update from review
* depends on py-setuptools before 0.6
* depends on py-poetry-core after 0.7
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
* py-aiofiles: Taking reviewing into account; add a upper version constraint for Python.
* py-aiofiles: update from review, set lower version of python
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-fenics-dolfinx: add upper bound on Python version
* Small fix
* Update var/spack/repos/builtin/packages/py-fenics-dolfinx/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Fixes#39622
Add a timeout to compiler detection and allow Spack to proceed when
this timeout occurs.
In all cases, the timeout is 120 seconds: it is assumed any compiler
invocation we do for the purposes of verifying it would resolve in
that amount of time.
Also refine executables that are tested as being possible MSVC
instances, and limit where we try to detect MSVC. In more detail:
* Compiler detection should timeout after a certain period of time.
Because compiler detection executes arbitrary executables on the
system, we could encounter a program that just hangs, or even a
compiler that hangs on a license key or similar. A timeout
prevents this from hanging Spack.
* Prevents things like cl-.* from being detected as potential MSVC
installs. cl is always just cl in all cases that Spack supports.
Change the MSVC class to indicate this.
* Prevent compilers unsupported on certain platforms from being
detected there (i.e. don't look for MSVC on systems other than
Windows).
The first point alone is sufficient to address #39622, but the
next two reduce the likelihood of timeouts (which is useful since
those slow down the user even if they are survivable).
Put back normalization of the "virtuals" input as a sorted tuple.
Without this we might get edges that differ just for the order of
virtuals, or that have lists, which are not hashable.
Add unit-tests to prevent regressions.
By default, do not let deprecated versions enter the solve.
Previously you could concretize to something deprecated, only to get errors on install.
With this commit, we get errors on concretization, so the issue is caught earlier.
* initial commit to add composable kernel package
* change dependencies to type build and add amdgpu_target variant
* fix spacing
* fix styling
* remove rocmmlir from miopen-hip recipe
* enable miopen with ck after 5.5.1
* fix typo
Fix permissions for configure file in var/spack/repos/builtin/packages/hdf-eos2/package.py, fix dependencies to match what hdf provides, update compiler flags for apple-clang
* [chore] py-aniso8601: Add version 3.0.2 required by py-graphene
* py-aniso8601: Add version 7.0.0 and constraints on python version
* py-aniso8601: fix style
* [fix] py-aniso8601: remove version 3.0.2 which depends on removed version of python; set the minimal version of python to 3.7
* py-aniso8601: remove version constraint on python
* Add Scine QCMaquis recipe to builtin packages
* Format scine-qcmaquis recipe using black
* Remove import os from scine-qcmaquis recipe
* Reduce length of symmetries line in scine-qcmaquis
* Add myself as a maintainer of the scine-qcmaquis recipe
* Update Scine-QCMaquis recipe following the review
PR URL: https://github.com/spack/spack/pull/39709
Changes:
- Updated Symmetries variant to us multi feature
- Added dependency to boost+chrono since it was undocumented
- Use define_from_variant to setup CMake args
- Make version 3.1.2 be preferred since 3.1.3 build is broken
* Change build_tests boolean condition
---------
Co-authored-by: Adam Grofe <adamgrofe@microsoft.com>
PythonExtension is a base class for PythonPackage, and
is meant to be used for any package that is a Python
extension but is not built using "python_pip".
The "update_external_dependency" method in the base
class calls another method that is defined in the derived
class.
Push "get_external_python_for_prefix" up in the hierarchy
to make method calls consistent.
* Add OIDC tokens to gitlab-ci jobs
This should allow us to start issuing just-in-time generated
credentials for CI jobs that need to modify binary mirrors. The "aud"
claim of the token describes what the token is allowed to do. The
claim is verified against a set of rules on the IAM role using signed
information from GitLab. See spack-infrastructure for the claim
verification logic.
---------
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
This commit replaces the internal representation of deptypes with `int`, which is more compact
and faster to operate with.
Double loops like:
```
any(x in ys for x in xs)
```
are replaced by constant operations bool(xs & ys), where xs and ys are dependency types.
Global constants are exposed for convenience in `spack.deptypes`
Currently the configuration of the pipeline is such that
there are multiple "optimal" solutions. This is due to
the pipeline making ~lld the default for LLVM, but leaving
+libomptarget from package.py
Since LLVM recipe has a:
conflicts("~lld", "+libomptarget")
clingo is forced to change one of the two defaults, and
the penalty for doing so is the same for each. Hence, it
is random whether we'll get +libomptarget+lld
or ~libomptarget~lld.
This fixes it by changing also the default for libomptarget.
* scorep version 8.1 added
* configure finds cudart and cupti in the nvhpcsdk suite
* style fixed
* changes to find libcuda.so in cuda directory
---------
Co-authored-by: Laura Bellentani <lbellen1@login01.leonardo.local>
* cp2k: patch several old versions to help newer compilers
* cp2k: use -O2 optimization for AOCC compiler
* cp2k: do not support old AOCC compilers
* cp2k: simplify when clause due to conflicting out old compilers
* cp2k: give a more meaningful message for confilcts
Co-authored-by: Ning Li <ning.li@amd.com>
Co-authored-by: Phil Tooley <phil.tooley@amd.com>
* libfirefly: Add cmake package for v2.1.0 and master (git) versions
* Separate git URL from version declaration
Co-authored-by: Alec Scott <alec@bcs.sh>
Currently, the concretizer emits facts for all versions known to Spack, including deprecated versions, and has a specific optimization objective to minimize their use.
This commit simplifies how deprecated versions are handled by considering possible versions for a spec only if they appear in a spec literal, or if the `config:deprecated:true` is set directly or through the `--deprecated` flag. The optimization objective has also been removed, in favor of just ordering versions and having deprecated ones last.
This results in:
a) no delayed errors on install, but concretization errors when deprecated versions would be the only option. This is in particular relevant for CI where it's better to get errors early
b) a slight concretization speed-up due to fewer facts
c) a simplification of the logic program.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* unifyfs: drop upperbound on deprecated openssl
The package uses deprecated MD5 functions from OpenSSL, which causes
warnings, but (a) Spack by default disables -Werror, and (b) those
functions will continue to exist in OpenSSL 3.
* unifyfs: enable parallel build, only make check sequential
* unifyfs: order class methods by install phases
* py-maestrowf: add new version 1.1.9, deprecate development releases
* py-maestrowf: drop py-cryptography in 1.1.9
* py-maestrowf: drop py-cryptography dependency entirely, since it is not a direct dependency
* py-merlin: new version, ensure openssl 3 compat
* py-merlin: drop py-coloredlogs@10: lowerbound
* py-maestrowf: add py-rich, reorder deps
* py-celery: explain why upperbound is in spack but not in requirements.txt
* openexr: add 2.4.3, 2.5.9, 3.1.11 & 3.2.0
- 2.5.9 is the latest version compatible with OpenScenGraph
- improved compatibility with GCC 13
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* openexr: drop unsupported debug variant
--disable/enable-debug is not supported by ./configure in 1.3, 1.7, 2.0, 2.2 and 2.3
* openexr: transform into multi-build system package
simplifies package considerably, as nothing special seems to be required
* openexr: pkg-config is also used by @3
* openexr: use system libdeflate instead of internal
if no libdeflate is found, openexr would download and build its own starting with 3.2.0
* openexr: disable tests
would download lots of data during cmake and make build times noticably longer
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Make the state of the Python Front End (PFE) and Python data reader
support sticky so that the concretizer does not arbitrarily disable
them.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* package_qscintilla_build_with_qt6
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* improve
* fix Qsci.abi3.so install
* simplify, fix, tidy
* fix style
* fix style
* fix style
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* improve
* improve style
* fix style
* make black happy
* add ver 2.14.1
* update
* make black happy
* Update var/spack/repos/builtin/packages/qscintilla/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* improve
---------
Co-authored-by: Sinan81 <Sinan@world>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* simple build of py-openai
* added variants to py-openai
* py-pandas-stubs is a dependency for py-openai
* fixed format and flake8 errors for py-openai
* black format error for py-pandas-stubs
* [@spackbot] updating style on behalf of sidpbury
* made style and format changes to py-openai
* made style and format changes to py-pandas-stubs
* py-types-pytz is a dependency for py-openai
* [@spackbot] updating style on behalf of sidpbury
* updated py-openpyxl for ver 3.0.7 and 3.1.2
* Update var/spack/repos/builtin/packages/py-pandas-stubs/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ajs requested changes for py-openai
* updated py-openpyxl for supported python
* [@spackbot] updating style on behalf of sidpbury
* updated py-openpyxl
* removed requirement.txt dependencies in py-openpyxl
* removed python depends on from openpyxl
* updated package to support newer versions
* updated version of py-pygit2
* py-fairscale is a new package in support of torch
* py-pgzip is a dependency for py-fairscale
* switch fairscale pypi, added extra variant for convenience
* removed python dependency
* changed multiple requirement versions
* changes for upstream py-fairscale
* changes for upsteam py-pygit2
* Update package.py
* Update package.py
* sorted out some of the dependency versions
* removed version 1.12.2 because dependency could not be met
* updated py-cached-property dependency
* suggested changes from adamjstewart
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ensure umpire~cuda~rocm when ~cuda~rocm
* update mdanalysis
* Update var/spack/repos/builtin/packages/py-mdanalysis/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-mdanalysis/package.py
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-hatchling: add 1.18.0
* py-pipdeptree: add new package
* py-hatchling: add Python 3.8 dependency
* Apply suggestion from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Fábio de Andrade Barboza <f168817@dac.unicamp.br>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* glibc: add missing deps, drop < 2.17
Building glibc 2.17 requires linking libgcc_{s,eh}.a, which themselves
depend on whatever glibc the current gcc uses for its runtime libraries.
Newer gcc depends on gnu extensions of libdl it seems, so that means you
simply cannot build old glibc with gcc-using-new-glibc.
The relevant fix in glibc is this commit:
commit 95f5a9a866695da4e038aa4e6ccbbfd5d9cf63b7
Author: Joseph Myers <joseph@codesourcery.com>
Date: Tue Jul 3 19:14:59 2012 +0000
Avoid use of libgcc_s and libgcc_eh when building glibc.
See also references to that commit in the glibc mailing list.
* update the gmake bound
* add --without-selinux
SIRIUS was introduced in version 7 of cp2k but could be used
in practice in version 9 (input format and functionalities).
SIRIUS with version 6 and below are marked as a dependency
conflict until CP2K version 9.
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
Instead of pointing to the image on DockerHub, which rate limits us and
causes pipeline failures durying busy times, use the version at ghcr.
And we might as well use the ghcr version everywhere else too.
NMake makefiles are still called makefiles. The corresponding builder
variable was called "nmakefile", which is a bit unintuitive and lead
to a few easy-to-make, hard-to-notice mistakes when creating packages.
This commit renames the builder property to be "makefile"
Extensionless archives requiring two-stage decompression and extraction
require intermediate archives to be renamed after decompression/extraction
to prevent collision. Prior behavior attempted to cleanup the intermediate
archive with the original name, this PR ensures the renamed folder is
cleaned instead.
Co-authored-by: Dan Lipsa <dan.lipsa@khq.kitware.com>
Co-authored-by: John Parent <john.parent@kitware.com>
Smart alias completion introduced in #39499 wasn't as smart as it needed to be, and
would complete any invalid command prefix and some env names with alias names.
- [x] don't complete aliases if there are no potential completions
e.g., don't convert `spack isnotacommand` -> `spack concretize`
- [x] don't complete with an aliases if we're not looking at a top-level subcommand.
* Perform external spec detection with multiple workers
The logic to perform external spec detection has been refactored
into classes. These classes use the GoF "template" pattern to account
for the small differences between searching for "executables" and
for "libraries", while unifying the larger part of the algorithm.
A ProcessPoolExecutor is used to parallelize the work.
* Speed-up external find by tagging detectable packages automatically
Querying packages by tag is much faster than inspecting the repository,
since tags are cached. This commit adds a "detectable" tag to every
package that implements the detection protocol, and external detection
uses it to search for packages.
* Pass package names instead of package classes to workers
The slowest part of the search is importing the Python modules
associated with candidate packages. The import is done serially
before we distribute the work to the pool of executors.
This commit pushes the import of the Python module to the job
performed by the workers, and passes just the name of the packages
to the executors.
In this way imports can be done in parallel.
* Rework unit-tests for Windows
Some unit tests were doing a full e2e run of a command
just to check a input handling. Make the test more
focused by just stressing a specific function.
Mark as xfailed 2 tests on Windows, that will be fixed
by a PR in the queue. The tests are failing because we
monkeypatch internals in the parent process, but the
monkeypatching is not done in the "spawned" child
process.
* Write timing information for installs from cache
* CI: aggregate and upload install_times.json to artifacts
* CI: Don't change root directory for artifact generation
* Flat event based timer variation
Event based timer allows for easily starting and stopping timers without
wiping sub-timer data. It also requires less branching logic when
tracking time.
The json output is non-hierarchical in this version and hierarchy is
less rigidly enforced between starting and stopping.
* Add and write timers for top level install
* Update completion
* remove unused subtimer api
* Fix unit tests
* Suppress timing summary option
* Save timers summaries to user_data artifacts
* Remove completion from fish
* Move spack python to script section
* Write timer correctly for non-cache installs
* Re-add hash to timer file
* Fish completion updates
* Fix null timer yield value
* fix type hints
* Remove timer-summary-file option
* Add "." in front of non-package timer name
---------
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* seacas: Set TPL_ENABLE_Pthread=ON when +thread_safe
This should fix#39702
Basically, following suggestion of error message and setting
TPL_ENABLE_Pthread to the value of the boolean spack variant thread_safe
* seacas: Fix style issue
Add space after comment #
* cgns: Add -fPIC to Fortan/C compilation flags
This should fix#39699
* cgns: Add -fPIC compilation flags when +fortran
Incorporating suggestions of @aumuell with addition of making +pic required when +fortran
* r-sets: Add r-sets.
* r-sets: Fix R dependency for first version.
* r-sets: Remove package versions for which a compatible version of R cannot be installed.
The spiral-software package had a number of extensions, but does not
work unless they actually exist in the spiral software prefix (creating
a view is not sufficient). With the removal of "spack activate"
(different from "spack env activate"), a new approach is needed to
support optional components of `spiral-software` . This commit updates
the spiral-software package to copy the dependency installations into
its own prefix.
This commit also adds versions for `fftx` and `spiral-software`, as
well as an optional `spiral-software-jit` package.
This is a fixed version of b72a268
* That commit would discard the final key component (so if you set
"config:install_tree:root", it would discard "root" and just set
install tree).
* When setting key:"value", with the quotes, that commit would
discard the quotes, which would confuse the system if adding a
value like "{example}" (the "{" character indicates a dictionary).
This commit retains the quotes.
* adding bacio to g2 dependencies
* edited documentation
* added version 3.4.6
* starting with 3.4.6, can use any version of jasper
* adding w3emc dependency for versions up to 3.4.5
* removed t-brown as maintainer at his request
* Add hpx-kokkos 0.4.0
* Make git global package property in hpx-kokkos instead of having it version-specific
* Add variant for choosing future type in hpx-kokkos
* Add support for testing hpx-kokkos
These commands are currently broken on powershell (Windows) due to
improper use of the InvokeCommand commandlet and a lack of direct
support for the `--pwsh` argument in `spack load`, `spack unload`,
and `spack env deactivate`.
If you wanted to set a configuration option like
`config:install_tree:root` to "C:/path/to/config.yaml", Spack had
trouble parsing this because of the ":" in the value. This adds
logic to allow using quotes to enclose the value, so you can add
`config:install_tree:root:"C:/path/to/config.yaml"`.
Configuration keys should never contain a quote character, so the
presence of any quote is taken to mean that the rest of the string
is specifying the value.
* add a possibility to control default cuda version
* fix stype
* style fix
* resolve comment
* resolve comment
* Fix style in nvhpc package.py
---------
Co-authored-by: antonk <antonk@cscs.ch>
Co-authored-by: Mikael Simberg <mikael.simberg@iki.fi>
Setting the undocumented variable SPACK_CONCRETIZER_REQUIRE_CHECKSUM
now causes the solver to avoid accounting for versions that are not checksummed.
This feature is used in CI to avoid spurious concretization against e.g. develop branches.
* The flang project does not support exceptions enabled in the core llvm
library (and developer guidelines explicitly state they should not be
used). For this reason, when the flang variant is selected,
LLVM_ENABLE_EH needs to be disabled. In the current main branch of
llvm (and thus future releases), enabling flang and setting
LLVM_ENABLE_EH will cause the overall build to fail.
* Update var/spack/repos/builtin/packages/llvm/package.py
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
* fix syntax
---------
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
Currently, OneAPI's setvars scripts effectively disregard any arguments
we're passing to the MSVC vcvars env setup script, and additionally,
completely ignore the requested version of OneAPI, defaulting to whatever
the latest installed on the system is.
This leads to a scenario where we have improperly constructed Windows
native development environments, with potentially multiple versions of
MSVC and OneAPI being loaded or called in the same env. Obviously this is
far from ideal and leads to some fairly inscrutable errors such as
overlapping header files between MSVC and OneAPI and a different version
of OneAPI being called than the env was setup for.
This PR solves this issue by creating a structured invocation of each
relevant script in an order that ensures the correct values are set in
the resultant build env.
The order needs to be:
1. MSVC vcvarsall
2. The compiler specific env.bat script for the relevant version of
the oneapi compiler we're looking for. The root setvars scripts seems
to respect this as well, although it is less explicit
3. The root oneapi setvars script, which sets up everything else the
oneapi env needs and seems to respect previous env invocations.
`compgen -W` does not behave the same way in zsh as it does in bash; it seems not to
actually generate the completions we want.
- [x] add a zsh equivalent and `_compgen_w` to abstract it away
- [x] use `_compgen_w` instead of `compgen -W`
Bash completion is now smarter about handling aliases. In particular, if all completions
for some input command are aliased to the same thing, we'll just complete with that thing.
If you've already *typed* the full alias for a command, we'll complete the alias.
So, for example, here there's more than one real command involved, so all aliases are
shown:
```console
$ spack con
concretise concretize config containerise containerize
```
Here, there are two possibilities: `concretise` and `concretize`, but both map to
`concretize` so we just complete that:
```console
$ spack conc
concretize
```
And here, the user has already typed `concretis`, so we just go with it as there is only
one option:
```console
spack concretis
concretise
```
From a user:
> Aargh.
> ```
> ==> Error: concretise is not a recognized Spack command or extension command; check with `spack commands`.
> ```
To make things easier for our friends in the UK, this adds `concretise` and
`containerise` aliases for the `spack concretize` and `spack containerize` commands.
- [x] add aliases
- [x] update completions
Switch from makefile to CMake-based build. CMake support is currently
in a specific branch of the amg2023 repo, so add this branch as a
version in the package.
Set BL_USE_PARTICLES to 1, which should case boxlib build to include Particles classes
according to CMakeLists.txt.
This seems to fix#18172
The aforementioned error seems to occur in cmake phase while processing
CMakeLists.txt in Src/C_ParticleLib, and appears to be due to the
variable containing the list of src files for the add_library() call
being empty unless BL_USE_PARTICLES is set to 1.
Use full length commit sha instead of short prefixes, to improve
reproducibility (future clashes) and guard against compromised repos and
man in the middle attacks.
Abbreviated commit shas are expanded to full length, to guard against future
clashes on short hash. It also guards against compromised repos and
man in the middle attacks, where attackers can easily fabricate a malicious
commit with a shasum prefix collision.
Versions with just tags now also get a commit sha, which can later be used to
check for retagged commits.
* VTK: Add patch for python 3.8 support
* CI: Re-enable VisIt in CI
* Configure spec matrix for stack with VisIt
* Add pugixml dep for 8.2.0
* Make VTK and ParaView consistent on proj dep
* OpenMPI 3: provides MP support by default
* Add details on proj dep in ParaView
* Add python 3.8 to test mock repo
* Patches to get VisIt VTK interface
* CI: Disable VisIt with GUI in DAV
GitHub's beta private security issue reporting feature is enabled on the Spack repo now,
so we can change `SECURITY.md` to recommend using it instead of `maintainers@spack.io`.
- [x] Update `SECURITY.md` to direct people to the GitHub security tab.
- [x] Update working in `SECURITY.md` to say "last two major releases" with a link to
the releases page, instead of explicitly listing release names. This way we don't have
to update it (which we keep forgetting to do).
This reapplies 66f7540, which adds supports for hardlinks/junctions on
Windows systems where developer mode is not enabled.
The commit was reverted on account of multiple issues:
* Checks added to prevent dangling symlinks were interfering with
existing CI builds on Linux (i.e. builds that otherwise succeed were
failing for creating dangling symlinks).
* The logic also updated symlinking to perform redirection of relative
paths, which lead to malformed symlinks.
This commit fixes these issues.
There are other ways to enforce cray-pmi being loaded in environments
that use cray-mpich. This avoids breaking environments where this was
already the case and avoids forcing them to declare an external.
* AOCC and AOCL spack recipes for 4.1 release
* Fix broken checksum
* remove blank line
* Add missing `@when` for 4.1 only function
---------
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
* Create a smartsim package
* rm ss 0.4.2
* no py upper bound, add build dep
* add setup_build_env
* add comment to find ml deps lower bounds
* Apply suggestions from code review
Correct dep versions, use `python_purelib`
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* remove the cuda/rocm vars
* point editors to bin deps version constraints
* Apply suggestions from code review
Loosen `py-smartredis` constraint, enforce `setup.cfg` py version
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* style
* rm rai lower bound
* lower bound setuptools
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add magma variant to strumpack
* clarify conflicts to be excuive to rocm/cuda, without it +magma+cuda fails as it is ~rocm
modified: var/spack/repos/builtin/packages/strumpack/package.py
* add missing depends_on for magma variant
#35042 introduced lazy hash parsing, but didn't remove a
few attributes from the parser that were needed only for
concrete specs
This commit removes them, since they are effectively
dead code.
Clone is a hard dependency as of HTTP-Message v6.44. This causes
problems in packages like Roary which depend on perl-http-message:
$ spack load roary
$ roary
Use of uninitialized value in require at /var/scratch/spack/opt/spack/linux-centos8-x86_64_v3/gcc-8.5.0/perl-http-message-6.44-lzp5th4jddd3gojkjfli4hljgem2nl26/lib/perl5/HTTP/Headers.pm line 8.
Can't locate Clone.pm in @INC (you may need to install the Clone module) (@INC contains: /home/aorth/lib ...
See: https://github.com/libwww-perl/HTTP-Message/blob/master/Changes
While spack does not yet provide binutils 2.41, they might still be
installed. However, building ffmpeg on x86_64 fails with multiple errors like
this:
./libavcodec/x86/mathops.h:125: Error: operand type mismatch for `shr'
also reported here: https://trac.ffmpeg.org/ticket/10405
Add amg2023 package
Consolidate existing amg and amg2013 packages (they reference the
same code) under the amg2013 name to minimize confusion between
amg2023 and amg2013.
Co-authored-by: Riyaz Haque <haque1@llnl.gov>
The heuristic for duplicate nodes contains a few typos, and
apparently slows down the solve for specs that have a lot of
sub-optimal choices to be taken.
This is likely because with a lot of sub-optimal choices, the
low priority, flawed heuristic is being used by clingo.
Here I split the heuristic, so complex rules that matter only
if we allow multiple nodes from the same package are used
only in that case.
* use full cuda_arch list
* update compiler requirement
* update boost requirements
* propagate +kokkos to legion in non-GPU cases
* add missing graphviz dependency for +doc
Spack installs the hsa-rocr-dev and rocprofiler packages into different
directories. However, PAPI typically expects those to be under the same
directory and locates such directory through the PAPI_ROCM_ROOT env
variable.
The PAPI rocm component also allows users to override PAPI_ROCM_ROOT to
locate directly the librocprofiler64.so through the HSA_TOOLS_LIB env
variable that acts directly onto the HSA runtime tools mechanism.
Hence, in order to account for the decoupling of hsa and rocprofiler,
this patch sets HSA_TOOLS_LIB to librocprofiler64.so full path.
* Update pennylane-lightning.
* Update Lightning-Kokkos to v0.31
* Constrain scipy version.
* Update var/spack/repos/builtin/packages/py-pennylane-lightning-kokkos/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix PLK kokkos dep versioning.
* Move kokkos ver outised backend loop and reformat.
* Update package.py
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Since #34821 we are annotating virtual dependencies on
DAG edges, and reconstructing virtuals in memory when
we read a concrete spec from previous formats.
Therefore, we can remove a TODO in asp.py, and rely on
"virtual_on_edge" facts to be imposed.
Add support for PGO and LTO for gcc, clang and apple-clang, and add a
patch to allow mimalloc as an allocator in operator new/delete, give
reduces clingo runtime by about 30%.
Previous changes to this file stopped directly processing CL args to
stop batch `for` from interpolating batch reserved characters needed in
arguments like URLS. In doing so, we relied on `for` for an easy
"split" operation, however this incorrectly splits paths with spaces in
certain cases. Processing everything ourselves with manual looping via
`goto` statements allows for full control over CL parsing and handling
of both paths with spaces and reserved characters.
* WarpX 23.08
Update WarpX and related Python packages to the lastest releases.
* fix style
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
In the HPC package manager, we want the fastest `zlib` implementation by default. `zlib-ng` is up to 4x faster than stock `zlib`, and it can do things like take advantage of AVX-512 instructions. This PR makes `zlib-ng` the default `zlib-api` provider (`zlib-api` was introduced earlier, in #37372).
As far as I can see, the only issues you can encounter are:
1. Build issues with packages that heavily rely on `zlib` internals. In Gitlab CI only one out of hundreds of packages had that issue (it extended zlib with deflate stuff, and used its own copy of zlib sources).
2. Packages that like to detect `zlib-ng` separately and rely on `zlib-ng` internals. The only issue I've found with this among the hundreds of packages built in CI is `perl` trying to report more specific zlib-ng version details, and relied on some internals that got refactored. But yeah... that warrants a patch / conflict and is nothing special.
At runtime, you cannot really have any issues, given that zlib and zlib-ng export the exact same symbols (and zlib-ng tests this in their CI).
You can't really have issues with externals when using zlib-ng either. The only type of issue is when system zlib is rather new, and not marked as external; if another external uses new symbols, and Spack builds an older zlib/zlib-ng, then the external might not find the new symbols. But this is a configuration issue, and it's not an issue caused by zlib-ng, as the same would happen with older Spack zlib.
* zlib-api: use zlib-ng +compat by default
* make a trivial change to zlib-ng to trigger a rebuild
* add `haampie` as maintainer
Computing str(spec) is faster than computing hash(spec), and
since all the abstract specs we deal with come from user configuration
they cannot cover DAG structures that are not captured by str() but
are captured by hash()
message: "If you are referencing Spack in a publication, please cite the paper below."
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
abstract: >-
Large HPC centers spend considerable time supporting software for thousands of users, but the complexity of HPC software is quickly outpacing the capabilities of existing software management tools.
Scientific applications require specific versions of compilers, MPI, and other dependency libraries, so using a single, standard software stack is infeasible.
However, managing many configurations is difficult because the configuration space is combinatorial in size.
We introduce Spack, a tool used at Lawrence Livermore National Laboratory to manage this complexity.
Spack provides a novel, re- cursive specification syntax to invoke parametric builds of packages and dependencies.
It allows any number of builds to coexist on the same system, and it ensures that installed packages can find their dependencies, regardless of the environment.
We show through real-world use cases that Spack supports diverse and demanding applications, bringing order to HPC software chaos.
preferred-citation:
title: "The Spack Package Manager: Bringing Order to HPC Software Chaos"
and software development kits (e.g., `EcpDataVisSdk <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/ecp-data-vis-sdk/package.py>`_).
These versioned packages primarily consist of dependencies on the associated
software packages. They can include :ref:`variants <variants>` to ensure
common build options are consistently applied to dependencies. Known build
failures, such as not building on a platform or when certain compilers or
variants are used, can be flagged with :ref:`conflicts <packaging_conflicts>`.
Build requirements, such as only building with specific compilers, can similarly
be flagged with :ref:`requires <packaging_conflicts>`.
The ``spack create --template bundle`` command will create a skeleton
``BundlePackage````package.py`` for you:
..code-block::console
$ spack create --template bundle --name coolsdk
Now you can fill in the basic package documentation, version(s), and software
package dependencies along with any other relevant customizations.
..note::
Remember that bundle packages have no software of their own so there
is nothing to download.
^^^^^^^^^^^^^^^^^^^^^^^^^
Non-downloadable software
^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -610,7 +646,16 @@ add a line like this in the package class:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
