- Honor compiler extra_rpaths (this build bypasses spack wrappers,
so the RPATHs are added explicitly as CMake options)
- Use root_cmakelists_dir instead of adding it to the command line
- Add BLT as a dependency, allowing different versions outside of what
is in the tarball
- Put a copy of host-config in the stage directory: this allows
examination of the host-config when a build fails (before, the
host-config was just stored in the install directory, which is
deleted by default on a failed build)
* Add conflict between NVSHMEM and cuFFTMp for GROMACS package
These don't work in the same build configuration.
* [@spackbot] updating style on behalf of mabraham
* Update package.py
Also constrain NVSHMEM appropriately
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
* Update package.py
* [@spackbot] updating style on behalf of mabraham
* Update package.py
* Update package.py
* Update package.py
* [@spackbot] updating style on behalf of mabraham
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
Change the signature of Spec.from_detection to set the
external prefix, and the external modules, if they are
present.
Delete "spack.package_prefs.spec_externals" since it
is unused.
This context manager was used to prevent detecting a platform
as "cray". Since now Cray machines are detected as linux, we can
remove the context manager.
This commit adds version 0.73.0 of the detray package. As this version
drops support for pre-C++20 standards, I had to update the `cxxstd`
variant logic.
Right now if you run `spack ci generate` you get a warning about CDash credentials even
if there's no CDash configuration specified. We should only warn if there was actually a
CDash config.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Source mirrors store entries by digest and add a human readable alias of the
form 'name-version'. If no digest is available, the alias is used as the primary
storage location.
Spack erroneously fetches by alias when the digest path does not exist. This is
problematic if `version(..., sha256=...)` changes in package.py, and the mirror
is populated with the old shasum. That would result in an error when a digest
is available, but in case of git versions with a modified commit sha, the wrong
sources would be fetched without error. With this PR, only the digest path is
used, not the alias, in case a digest is available. This is also a small performance
optimization, as the number of request is halved for mirrors that don't contain
the sources.
Further, for git sources the tag was used as a digest, but this is a moving
target. Only commit sha is used now.
Also whenever the alias already existed, Spack used to keep it in place when
updating the mirror cache, which means that aliases would always point to
outdated mirror entries whenever digests are modified. With this PR the alias
is moved in place.
Lastly, fix a recent regression where `Stage.disable_mirrors` disabled mirrors
but not the local download cache, which was the intention.
* VTK-m: Point to github mirror for source tarball
The gitlab.kitware.com source location seems to have intermittent
network issues. Switching the to mirror hosted at Github may alleviate
some of the timeouts.
* Update sha256 for GitHub tarballs
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
* davix: add versions 0.8.2-0.8.7 and dependencies
This commit adds new versions 0.8.2-0.8.7 of the davix package, and it
also improves the handling of embedded packages. Davix will try to build
libcurl from its own embedded version of that code, which doesn't mesh
well with Spack's design philosophy, so I've changed the CMake
configuration to disallow the builtin libcurl and use a Spack dependency
instead. Up to version 0.8.7, RapidJSON was also builtin, but version
0.8.7 allows users to specify that they want to use a pre-installed
version of RapidJSON, so this commit also adds that as a dependency for
versions 0.8.7:.
* Fix old versions
* acts: add v36.0.0, v36.1.0 and fixes
This commit makes several changes to the Acts repository, namely:
1. It adds versions 36.0.0 and 36.1.0.
2. It adds the traccc plugin and related dependencies.
3. It updates the version requirements of some dependencies.
4. It adds the Geant4 module of GeoModel.
5. It updates the C++ standard requirement.
6. It adds a new variant determining the scalar type to use.
This commit supercedes #45851. Thanks @jmcarcell for the version
updates; I have added you as co-author.
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Updates
* alphabetic cmake args
---------
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Add missing MultiMethodMeta metaclass in builders
and remove the Python 2 fallback option in favor of hard errors to catch
similar issues going forward.
The fallback option can cause about 10K stat calls due to use of
`realpath` in the inspect module, depending on how deep Spack itself is
nested in the file system, which is ... undesirable.
* code shuffling to avoid circular import
* more reshuffling
* move reserved variant names into variants module
Currently, the llvm-amdgpu package doesn't compile on MacOS, but it is
also not marked as a conflict. This causes problems because it seems
that Spack is very happy to pull in llvm-amdgpu as the default package
to satisfy any virtual libllvm dependency, which can cause dependent
specs to fail to install on MacOS. This commit marks a conflict between
this llvm package and the Darwin platform.
* root: set +webgui~http conflict from @6.28.12:
Currently, the ROOT spec correctly identifies a conflict between +webgui
and ~http, but this conflict is marked as affecting @6.29.00: only. As a
matter of fact, ROOT 6.28.12 is also affected. This commit, therefore,
updates the when clause on the conflict to @6.28.12:.
* Remove when clause entirely
* oops
* Adds an MSBuild system + Builder to the icu4c package
* Adds custom install method as MSBuild system does not vendor an
install target
* The cxxstd variant is not supported on Windows (there are no config
options you use to tell the build system what cxx standard to
build against), so the variant definition was updated to occur
everywhere except Windows
Also, this commit defines the c/cxx..._flag properties of the MSVC
compiler (although they are not used by `icu4c` and not strictly
necessary to bundle with this PR).
This commit adds a dependency on GeoModel 4.6.0 when the GeoModel plugin
is enabled. Note that the dependency is upgraded to 6.3.0 in Acts
36.1.0, but that will need to be covered in #45851.
* geomodel: add versions 5.1.0-6.4.0
This PR adds new versions 5.1.0 through 6.4.0 of the geomodel package.
It also updates the CMake configuration to use the `define_from_variant`
mechanism and it adds me as a maintainer.
* Undo cmake change
The avx2 version can be downloaded from the ORCA [forum](https://orcaforum.kofo.mpg.de/app.php/dlext/?view=detail&df_id=214#).
The version is named `avx2-6.0.0` (as opposed to the more
natural-looking `6.0.0-avx2`) to avoid the avx2 version shadowing the
non-avx2 one. Definitely open for better suggestion.
* hsakmt-roct: new test API
* hsakmt-roct: minor change to check_install script variable name
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parallel-netcdf: new test API
* parallel-netcdf: fix test args and tweak docstring and variables
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* WarpX: Python on pyAMReX
Long overdue update for WarpX: in 2024, we updated our Python
bindings to rely on the new pyAMReX package. This deprecates the old
`py-warpx` package and adds a new dependency and variant to WarpX.
Also deprecates old versions that we will not continue to support.
* Update Cloud/E4S Pipelines for WarpX
`py-warpx` is replaced by `warpx +python`
oneAPI does not support IPO/LTO: diable for `py-amrex` even though
pybind11 strongly encourages it.
* Add `+python` variant
* `esmf` package installs Python bindings when `+python` is set
Note: this does not inherit `PythonPackage`, which force an either/or
choice between the Makefile and Pip builder: it instantiates a
`PythonPipBuilder` as needed (when `+python` is set).
Paths over 260 characters in length are not handled by `shutil.rmtree`
unless they use the extended-length path syntax (using a prefix of
"\\?\").
This fixes an issue where stage cleaning fails when paths in a stage
exceed the normal 260-character limit.
This indicates that other parts of the codebase should be examined/
refactored to handle long paths.
* harfbuzz: enable freetype in MesonBuilder to facilitate depends_on("freetype")
* spack style fix
* freetype is defined as a depends_on(), so set as enabled in MesonBuilder rather than an option/flag/variant
* add back depends_on($lang) lines for new test api
* keep original order
Previously `spack buildcache push --only=package` errored in the OCI
case, but it's been requested that OCI can be used as pure storage w/o
the need for runnable container images.
This commit makes it so that
1. manifests refer only to runtime dependencies that were selected to be
pushed
2. failure to upload a blob among the selected specs does not prevent a
manifest/tag to be created for dependents: they just don't refer to
the missing blob as a layer/dependency
This fixes the following issues:
1. dependents of non-redistributable specs can now be pushed to oci
build caches without error
2. failure to upload one tarball does not cause cascading failures for
dependents whose tarballs do upload succesfully -- so it's better
best-effort behavior
3. for some people uploading with deps caused a massive amount of
fetches of their manifests (which certain registries count as a
download of an image, even though their layers are not fetched) --
being able to specify --only=package reduces the number of fetches.
Stage objects create mirrors ad-hoc from current config.
- There is no way to prevent mirrors from being used
- There is no way to restrict mirrors to source/binary, which is of
course context dependent.
- Stage is also used in build caches, where iterating over mirrors is
already implemented differently, and wouldn't work anyways cause it's
source only, and in particular it makes no sense for OCI build caches.
This commit:
1. Injects the sensible mirrors into the stage object from contexts
where it is relevant
2. Separates mirrors from cache, so that w/o mirrors download cache can
still be used
The test_foreground_background unit test has been marked
xfail for a while, meaning:
- Nobody looks at the results of the test
- It still runs every time
That test happens to hang frequently on some Apple M1 I have access to,
so here I mark it as skip.
Also went through other xfailing and skipped tests, and applied minor changes.
* Use absolute paths instead of https:// fake mirrors (this speed-up tests by avoiding requests)
* Add a fixture to gather in a single place code that is copy/pasted in a lot of tests
* General clean-up of tests and repeated code
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* hypre: get rid of depreciated run_test method
* hypre: use mkdirp directly
* hypre: use install() for ij for addition of permissions fix
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parsec: old to new test API
* parsec: restore stand-alone test subparts; preliminary test build fixes
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* n2p2: convert from old to new test API
* n2p2: Enhance stand-alone testing checks to reduce unnecessary processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* stand-alone test API update: self.cache_extra_test_sources(...) -> cache_extra_test_sources(self, ...)
* superlu: switch to new cache_extra_test_sources API
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add options for sparse checkout in GitFetcher
Newer versions of git have a beta feature called sparse checkout
that allow users to check out a portion of a large repo.
This feature will be ideal for monolithic repo projects that want to
model their infrastructure via spack. This PR implements an addition
to the GitFetcher that allows users to add a `git_sparse_paths`
attribute to package classes or versions which will then use sparse
checkout on those directories/files for the package.
* Style
* Split git clone into multiple functions
* Add sparse-checkout impl
* Internalize src clone functions
* Docs
* Adding sparse clone test
* Add test for partial clone
* [@spackbot] updating style on behalf of psakievich
* Small fixes
* Restore default branch status
* Fix attributes for package
* Update lib/spack/docs/packaging_guide.rst
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Extend unit test to multiple git versions
* style
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Feature update py-fortranformat
Add more recent versions of py-fortranformat. The currently included release (0.2.5) is from 2014. I've added the latest point release of each of the major versions from the last 4 years.
* update homepage
* magma: old to new test API
* magma: simplify stand-alone test method/part docstrings/purposes
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* initial commit to enable asan in remaining rocm packages
* remove os import
* add rocm-opencl rocm-dbgapi rocm-debug-agent
* add libclang path to LD_LIBRARY_PATH
* enable asan for rocfft
* add f-string and add +asan dependencies for hip and rocm-opencl
* add conflicts for centos7/8 and rhel 9
When sending Repo and RepoPath over to a child process,
we go through a marshaling procedure with pickle. The
default behavior for these classes is highly inefficient,
as it serializes a lot of specs that can just be
reconstructed on the other end of the pipe.
Here we write optimized procedures to __reduce__ both
classes.
* raja: new test API
* raja: tweak test method names and docstrings
* raja: restore running tests under proper directory
* raja: cleanup skiptest message and example call
* raja: Tweak expected outputs to match current
* raja: test_views -> test_stencil_offset_layout
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* mptensor: Changed skiptest, test name, and added docstring
* mptensor: make stand-alone test method name and docstring more specific
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add version 2.0.-2
* Newer version at the top
* [@spackbot] updating style on behalf of rahmans1
* py-htgettoken: reorder comments
* Use sha256sum instead of commit id for version 2.0-2 and above
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Extracted from #44419
This adds / modifies some external detection tests for compilers,
to reproduce cases that are currently tested in unit tests.
The unit tests will later be removed.
Extracted from #45638
When adding the "detectable" tag to a package class that has the
"tag" attribute inherited from a base class, we need to copy it to
avoid modifying the base class.
* kickoff attempt
* resource similar to fortran
* delete unused install_component_codeplay
* Adding conflict for versions <= 2022.2.1, moving install to package.py, adding sha256 for version 2024.2.1
* [@spackbot] updating style on behalf of kaanolgu
---------
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
"spack buildcache push" for partially installed environments pushes all it
can by default, and only dumps errors towards the end.
If --fail-fast is provided, error out before pushing anything if any
of the packages is uninstalled
oci build caches using parallel push now use futures to ensure pushing
goes in best-effort style.
* migraphx: Old to new test API
* migraphx: tweak name and docstring to be more descriptive
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* c: new test API
* gcc: provides('c')
* c: bugfix and simplification of the new stand-alone test method
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* fortran: new test API
* fortran: add provides to gcc package
* fortran: simplify stand-alone test processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* hypre-cmake: old to new test API
* hypre-cmake: update Makefile to use installed files
* hypre-cmake: make stand-alone test method name more specific
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
The old concretizer is still used to bootstrap clingo from source. If we switch to a DAG model
where compilers are treated as nodes, we need to either:
1. fix the old concretizer to support this (which is a lot of work and possibly research), or
2. bootstrap `clingo` without the old concretizer.
This PR takes the second approach and gets rid of the old concretizer code. To bootstrap
`clingo`, we store some concrete spec prototypes as JSON, select one according to the
coarse-grained system architecture, and tweak them according to the current host.
The old concretizer and related dead code are removed. In particular, this removes
`Spec.normalize()` and related methods, which were used in many unit-tests to set
up the test context. The tests have been updated not to use `normalize()`.
- [x] Bootstrap clingo concretization based on a JSON file
- [x] Bootstrap clingo *before* patchelf
- [x] Remove any use of the old concretizer, including:
* Remove only_clingo and only_original fixtures
* Remove _old_concretize and _new_concretize
* Remove _concretize_together_old
* Remove _concretize_together_new
* Remove any use of `SPACK_TEST_SOLVER`
* Simplify CI jobs
- [x] ensure bootstrapping `clingo` works on on Darwin and Windows
- [x] Raise an intelligible error when a compiler is missing
- [x] Ensure bootstrapping works on FreeBSD
- [x] remove normalize and related methods
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`setup-env.sh` is meant to be sourced, not executed directly.
By revoking execution permissions, users who accidentally execute
the script will receive an error instead of seeing no effect.
* Remove execution permission from `setup-env.sh` and friends
* Don't make output file executable in `spack commands --update-completion`
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* On Windows it is built with CMake, however CMake is built against
libuv, so libuv must depend on cmake+ownlibs to short circuit the
circular dependency
* libuv currently fetches the -dist source distribution of libuv for
certain versions because those versions contain a pre-generated
./configure script. However those distributions have all CMake
files removed, so they cannot be used to build on Windows.
Because the source distributions are different, this means the
checksums are different, and necessitates an additional version
declaration for each version we want to support with CMake.
* Update available versions
* fix style
* Reduce added versions to just a single new version as per review
* fix style
* Set dependency versions in line with pyproject and setup.py
* add new interp variant
* add wheel as required
* Add variant description
* change +interp default, actually add packaging this time
Fixes#43494
Add a set of environment variables SPACK_ALWAYS_CFLAGS (etc.) that
are always applied by the compiler wrapper.
Unlike SPACK_CFLAGS, for example, these will also be applied to
version checks (both SPACK_CFLAGS and SPACK_ALWAYS_CFLAGS will be
applied to the other invocation modes like ccld etc.).
Using this new functionality, the classic Intel and oneAPI compilers
are updated to pass compiler flags that disable warning messages
when newer versions are invoked via their older binary names
(these warnings were also generated for version checks, hence the
need for a new wrapper variable).
---------
Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
* namd:add compile options for ROCm
* Combine --rocm-prefix and its value in a single opts.extend call to ensure they remain ordered correctly and improve code robustness.
* Standardize the code and add ROCm conflicts
* add single_node_gpu conflict
---------
Co-authored-by: U-PALLAS\boudaoud <boudaoud@pc44.pallas.cines.fr>
When clang is installed I get `clang: Permission denied`.
Setting CLANG to the full path of clang fixes this.
Signed-off-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Fixes error:
```txt
3 errors found in build log:
61 ==> xmlto: Executing phase: 'build'
62 ==> [2023-10-26-09:48:35.425903] 'make' '-j16' 'V=1'
63 make all-am
64 make[1]: Entering directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
65 /bin/sh ./ylwrap xmlif/xmlif.l .c xmlif/xmlif.c -- /bin/sh /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing flex
66 /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing: line 81: flex: command not found
>> 67 WARNING: 'flex' is missing on your system.
68 You should only need it if you modified a '.l' file.
69 You may want to install the Fast Lexical Analyzer package:
70 <http://flex.sourceforge.net/>
>> 71 make[1]: *** [Makefile:757: xmlif/xmlif.c] Error 127
72 make[1]: Leaving directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
>> 73 make: *** [Makefile:584: all] Error 2
```
For example: spack install py-pyqt5 or py-pyqt6 would install pylupdate5 pyrcc5 and
pyuic5 into in python-venv's install prefix.
Fix https://github.com/spack/spack/issues/45359
* [py-configspace] fix dependency on cython
* py-cython not needed starting from 1.0.0
* added py-configspace 1.0.0 and 1.0.1
* py-configspace: fix style
* added py-configspace version 1.1.0
* added py-configspace version 1.1.1
* py-configspace: two more versions and new maintainer
* py-configspace: fixed typo
* py-configspace: added version 1.1.4
* Allow spec queries by `namespace`
Spack specs have "namespaces" that indicate what package repository they come from, but
there has not been a way to use the spec syntax to match one.
You can say things like this:
```console
spack find builtin.zlib
spack find myrepo.zlib
```
But, because namespaces are written as a dot-separated prefix on the name, you can't say
"find me all specs in namespace myrepo". The syntax doesn't allow it.
This PR allows you to specify anonymous specs with namespaces on the CLI. Specifically
you can do queries like this:
```console
spack find namespace=builtin
spack find namespace=myrepo
```
You can use this anywhere else you use spec syntax, e.g. in a config file to separate
installations based on what repo they came from:
```yaml
spack:
config:
install_tree:
root: $spack/opt/spack
projections:
namespace=myrepo: "myrepo_special_path/{name}-{hash}"
namespace=builtin: "builtin/{name}-{hash}"
```
This PR adds a special `namespace_if_anonymous` attribute to specs, which returns the
`namespace` if the spec has no name, otherwise it returns `None`. This allows us to
print the namespace for anonymous specs but to continue hiding it for most views, as
we've done so far.
This is implemented as a special case, but it's one that already exists, along with
`platform`, `os`, `target`, etc. This also reserves existing special case names for
variants so that users cannot define them in their package files. This change is
potentially breaking, but I do not think it will be common. There are no builtin
packages with a variant called `namespace`, and defining `os`, `target`, or `platform`
as a variant would've likely caused other problems if they were already being used.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* format: allow spaces in format specifiers
Key-value pair format specifiers can now contain spaces in the key. This allows us to
add spaces to format strings that are *only* present when the attribute formatted is not
``None``. Instead of writing:
```
{arch=architecture}
```
and special casing `arch=` like a sigil in `Spec.format()`, we can now write:
```
{ arch=architecture}
```
And the space is *only* printed when `architecture` is not `None`. This allows us to
remove the special case in `Spec.format()` for `arch=`.
Previously the only `key=` prefix allowed in format specifiers was `arch=`, but this PR
removes that requirement, and the `key=` part of a key-value specifier can be any name.
It does *not* have to correspond to the formatted attribute.
- [x] modify `SPEC_FORMAT_RE` to allow arbitrary keys in key-value pairs.
- [x] remove special case for `arch=` from `Spec.format()`.
- [x] modify format strings using `{arch=architecture}` to use `{ arch=architecture}`
- [x] add more tests for formatting
This PR saves other more complex attributes like compiler flags and their spacing for later.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
VC toolset versions 144 and 143 are both associated with the platform
toolset 143; this deviates from prior version choices made by the
MSVC devs; add a special case to report platform toolset version
as 143 when VC toolset version is >= 143 (this will need revisiting
for later releases).
The setSet tool (removed in version 10) has an optional dependency on readline.
The build script will use the system readline if not provided by Spack.
* dd4hep: restrict to podio@0 to prevent failures with podio@1
* acts: restrict to podio@0 to prevent failures with podio@1
* dd4hep: close when range for podio
* acts: close when range for podio
* acts: fix when range for podio
* acts: close when range for podio
* acts,dd4hep: ensure main/master still depends on podio
* Fix +rocm variant, to ensure correct dependencies on ROCm packages
and use of AMD LLVM
* Add a +pshm variant for comm=gasnet to enable fast shared-memory
comms between co-locales
* Add logic to ensure we get the native CXI libfabric network provider
on Cray EX
* Expand dependency type for package modules to encompass runtime
dependencies
* Factor logic for setting (LD_)LIBRARY_PATH and PKG_CONFIG_PATH of
runtime dependencies
* Workaround issue #44746 that causes a transitive dependency on lua
to break SLURM
* Disable nonfunctional checkChplDoc test
* Annotate some variants as conditional, to improve spack info output
and reduce confusion
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
`apple-libuuid` includes types that aren't available in other `uuid`
providers; this cause issues in consuming packages (e.g., py-matplotlib)
that use SDKs like CarbonCore.framework when they attempt to use
`util-linux-uuid` as a `uuid` provider on Mac OS.
Tweak `util-linux-uuid` to indicate that it does not provide `uuid`
on Mac OS.
This was missed in #45394 because we don't run unit tests for package PRs, and
`test_all_virtual_packages_have_default_providers`, which would've caught it, is a unit
test, not an audit.
- [x] add a default provider for `wasi-sdk` in `etc/spack/defaults/packages.yaml` (which
we require for all virtuals)
- [x] rework `test_all_virtual_packages_have_default_providers` as an audit called
`_ensure_all_virtual_packages_have_default_providers`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* pinentry: New API
* move code around
* added back version check
* Complete check_version refactor
* Honor original handling of guis (i.e., don't try if not there)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* upgrade apptainer: security fix
apptainer 1.3.1 has a high severity security issue: CVE-2024-3727
Upgrade to 1.3.2 or preferably 1.3.3
* added comment for 1.3.1
---------
Co-authored-by: Walter de Jong <walter.dejong@surf.nl>
* intel-oneapi-compilers: update description with current compilers
* Update var/spack/repos/builtin/packages/intel-oneapi-compilers/package.py
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* intel-oneapi-compilers: break docstring line
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* kokkos: new test API
* kokkos: added import llnl.util.lang as lang because earlier versions couldn't be installed without it
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* povray: new test API
* capture output and test name change
* povray: add v3.7.0.10, deprecate 3.7.0.8
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* umpire: old to new test format
* umpire: old to new test method and refactor
* indentation
* black reformat
* last minute syntax
* docstring and checks
* black format
* change test name
* method call correction
* Resolve problem with stand-alone tests (or examples) not loading library (Fixes#44960)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* sz: new test API
* fix typo; check installed executable; conform to subpart naming convention
* skip tests early if not installed; remove unnecessary "_sz" from test part names
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Library names on Windows are not typically prefixed with lib; the default
`.libs` implementation accounts for this, but `netlib-lapack` has a
custom implementation of `.libs` that did not account for this.
The `Executable` class emits a warning when you pass quoted arguments to it:
```
> spack pkg grep '"namespace"'
==> Warning: Quotes in command arguments can confuse scripts like configure.
The following arguments may cause problems when executed:
"namespace"
Quotes aren't needed because spack doesn't use a shell. Consider removing them.
If multiple levels of quotation are required, use `ignore_quotes=True`.
```
This is to warn new package authors who aren't used to calling build commands in python.
It's not useful for `spack pkg grep`, where we really are passing args on the command
line, and if we pass a quoted string, we probably meant to.
- [x] make `ignore_quotes` an instance variable, not just an argument to ``__call__`
- [x] set `ignore_quotes` to `True` in `spack pkg grep`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* package/linux-pam: dependencies
* Adding variants to linux-pam
* Updating linux-pam variants
* Fixing variants for linux-pam after testing
* clean up flag handling
* clean up terrible tabs
* cant use default_args for compiler dependencies
* Change selinux to off by default
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Adding new versions and compilation conflict for nodejs
* Update failed version for gcc14
* Updating conflicts notes for correctness/clarity/format
* Applying spack-ized versions of fix in https://github.com/nodejs/node/issues/52223 to adddress CI failures
* Update fix-old-glibc-random-headers.patch
* Update package.py
* Update fix-old-glibc-random-headers.patch
* Update fix-old-glibc-random-headers.patch
* Adding conflict for older glibc
* Fixing patch for older systems, need to undef
* Removing overly strict conflicts
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Several tests can be enabled on Windows with no changes to logic
(either the test logic or logic being tested)
* Test for `spack location` requires modification of the test logic,
but with a minor change can be enabled on Windows
using self.compiler.cc_pic_flag here results in these errors:
==> sqlite: Executing phase: 'install'
==> Error: AttributeError: 'AutotoolsBuilder' object has no attribute 'compiler'
change it to self.pkg.compiler.cc_pic_flag instead.
* imoroved-rdock: new test API
* Make test subpart names and or descriptions more accurate
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* e4s ci: enable some disabled specs
* comment out cp2k +cuda due to unsupported cuda_arch
* comment out dealii+cuda due to concretize error
* work through concretize errors
* e4s: comment out failing builds
* e4s stack: disabled non-building specs
* comment out failing specs
* comment out failing specs
* cleanup comments
* rust: rework external find to require both rustc and cargo
* rust: handle unable to parse version
* [@spackbot] updating style on behalf of wdconinc
* rust: not x or not y -> not (x and y)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: pick first rustc found
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: list comprehensions
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
* libmesh: add missing v1.7.1 release
* libmesh: avoid pulling in petsc if only +mpi or +metis is given
* libmesh: add shared variant
Co-authored-by: rbberger <rbberger@users.noreply.github.com>
The "use_store" context manager is used to swap the value
of a global variable (spack.store.STORE), while keeping
another global variable consistent (spack.config.CONFIG).
When doing that it tries to evaluate the previous value
of the store, if that was not done already. This is wrong,
since the configuration might be in an "intermediate" state
that was never meant to trigger side effects.
Remove that operation, and add a unit test to
prevent regressions.
Add language dependencies `c`, `cxx`, and `fortran`.
These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.
The `# generated` comment can be removed by package maintainers after
validating correctness.
* detray: new package
This commit adds the detray package, a detector description library for
HEP experiments that is designed to be GPU-friendly.
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* vasp: add new version 6.4.3 and rework package
* vasp: remove redundant cuda dep
* vasp: bump fftlib variant version restriction
* vasp: honor the still existing scalapack variant
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Since the the MetaPathFinder now owns a lazily constructed RepoPath object, we need to deserialize environments before the package that needs to be restored. Before we were relying on globals to be inconsistent in a way that let the entire process go.
The patch which enables icx/ifx compilers is only added for `wrf@4.4:`. This PR prints a useful message at concretization time instead of failing the installation later on.
Co-authored-by: stephenmsachs <stephenmsachs@users.noreply.github.com>
* slate: changed from old to new format
* make code tighter
* replace assert method
* SkipTest plus other cleanup
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* vecmem: add infrastructure for working with SYCL
The vecmem package uses an unorthodox build system where, instead of
expecting a SYCL-capable compiler in the `CXX` environment variable, it
expects one in `SYCLCXX`. It also needs the correct SYCL flags to be
set. This commit adds a custom build environment for the vecmem package
which allows it to be built in this way. I've also added an extra CMake
flag to ensure that the build system doesn't download any unwanted
dependencies.
* Update var/spack/repos/builtin/packages/vecmem/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
- Add explicit conflict on ppc64le for 12.5 and newer.
- Update/fix intel compiler conflict to reflect that intel@2021 is compatible
only since 11.4.1 and not since 11.1.1.
- Add intel compiler conflicts to reflect strict support matrix since
12.2.0.
As discussed in https://github.com/spack/spack/pull/44881#issuecomment-2218411735 a `spack install py-globus-cli` fails to concretize on an Ubuntu 22.04 under Windows WSL2 because of too strict of explicit dependencies.
Let's try to remove them here (since these are "just in case" and in all honesty should be handled by `py-globus-sdk` anyways).
* Now builds with latest fmt release (11.0.1)
* Missing array include in nem_spread
* Fix timestep conssitency check in file-per-rank case if one or more dbs have no timesteps.
If an external happens to be a provider of anything, the solver would
set its weight to 0, meaning that it is most preferred, even if
packages.yaml config disagrees.
That was done so that `spack external find mpich` would be sufficent to
pick it up as mpi provider.
That may have made sense for mpi specifically, but doesn't make sense
for other virtuals. For example `glibc` provides iconv, and is an
external by design, but it's better to use libiconv as a separate
package as a provider.
Therefore, drop this rule, and instead let users add config:
```
mpi:
require: [mpich]
```
or
```
mpi:
buildable: false
```
which is well-documented.
This change makes `spack external find --not-buildable` mark virtuals
provided by detected packages as non-buildable, so that it's sufficient
for users to let spack detect say mpich and have the concretizer pick it
up as mpi provider even when openmpi is "more preferred".
Consistently with ArmPL@24:, the include directory for acfl@24:
has changed to include. The change wants to update to this change
and distinguish the include path for releases previous to 24.04
and the future ones
* rocm-cmake: changed test format from old to new
* Rename cmake variable
* post-conflict resolution: remove remaining version check
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Added Release 0.0.4 and 0.0.5
* Changed requirement for gotcha
use gotcha 1.0.5 for 0.0.2 and 0.0.3
* Combine gotcha 1.0.7 for master and develop
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
in case of inheritance the static tags prop may be updated multiple
times, and it turns out builder classes magically inherit from
traditional package classes
While trying to build packages with the OneAPI compiler version 2024.1 I
ran into the following error, indicating that the compiler is unable to
find some necessary libraries:
```
/storage/Software/oneapi/2024.1/compiler/2024.1/bin/sycl-post-link: error
while loading shared libraries: libonnxruntime.1.12.22.721.so: cannot open
shared object file: No such file or directory
icpx: error: unable to execute command: No such file or directory
icpx: error: sycl-post-link command failed due to signal (use -v to see
invocation)
```
Indeed, `libonnxruntime.1.12.22.721.so` does come bundled with the
OneAPI compiler, but it is not available in the build environment by
default. In this commit, I update the custom environment created by
OneAPI to include the `lib/` directory in which these libraries reside
in the `LD_LIBRARY_PATH` environment variable.
In my enthusiasm to add dfelibs to Spack, I didn't realise that the
unit tests of dfelibs use Boost and, as such, Boost is required as a
testing dependency.
Originally if you had `x -> y -> z`, and an env with `x` in its speclist that is concretized but not installed, then `spack find -c y` would not show anything. This was intended: `spack find` has up-until-now only ever listed out installed specs (and `-c` was for adding a preamble section about roots).
This changes `spack find` so:
* `-c` makes it search through all concretized specs in the env (in a sense it is anticipated that a concretized environment would serve as a "speculative" DB and users may want to query it like they query the DB outside of envs)
* Adds a `-i/--install-status` option, equivalent to `-I` from `spack spec`
* Shows install status for either `-c` or `-i`
* As a side effect to prior point, `spack find -i` can now distinguish different installation states (upstream/external)
Examples:
```
$ spack find -r
==> In environment findtest
==> 1 root specs
- raja
==> 6 installed packages (not shown)
==> 12 concretized packages to be installed (not shown)
```
```
$ spack find
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
berkeley-db@18.1.40 bzip2@1.0.8 diffutils@3.10 gmake@4.4.1 gnuconfig@2022-09-17 libiconv@1.17
==> 6 installed packages
==> 12 concretized packages to be installed (show with `spack find -c`)
```
```
$ spack find -c
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
[+] berkeley-db@18.1.40 [+] bzip2@1.0.8 - cmake@3.29.4 [+] diffutils@3.10 [+] gmake@4.4.1 [+] libiconv@1.17 - nghttp2@1.62.0 - pkgconf@2.2.0 - readline@8.2
- blt@0.6.2 - camp@2024.02.1 - curl@8.7.1 - gdbm@1.23 [+] gnuconfig@2022-09-17 - ncurses@6.5 - perl@5.38.2 - raja@2024.02.2 - zlib-ng@2.1.6
==> 6 installed packages
==> 12 concretized packages to be installed
```
$ spack -E find
...
==> 82 installed packages
```
* Add py-datasets version 2.20.0
* Add dependency requirements for version 2.20 + refactor
* Add missing tqdm and requests versions and to install latest py-datasets
* Add missing python requirements for 2.8.0 and 2.20.0
Fix a bug where Spec.tree with cover=nodes reduces deptypes from all
in-edges, including from nodes not reachable from the root, which almost
always happens for concrete specs
`Spec.__getitem__` queries dependent edges, which almost always point to
nodes outside the sub-dag considered. It should only ever look at edges
being traversed.
When using a newer Clang for Kokkos than supported by a given CUDA version, the
CUDA compiler detection in Legion's CMake still needs to be passed
CMAKE_CUDA_FLAGS to pass the compiler check.
This modifies heuristic to decay to clingo default
over time. The hope is that this helps with specs
that have an optimal solution with a high penalty.
Let target and compiler heuristic decay too, do not
guess compiler
* dfelibs: new package
This commits adds the `dfelibs` package, which is an Acts project fork
of dfelibs.
* dfelibs: docstring typo
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Add MatrixSwitch package
Add libOMM package
Use libOMM as external in ELSI
Add include paths for all libraries
Fortran modules need one directory up (can't use libs.directories since they are just the include/ directories)
This addresses a few issues in the kentutils package:
- Issue #44372
- Cleaning up formattting and styles
- Removing old versions that are not avaialble anymore
- Removing newer versions that are also not available anymore
- The installer does not install the static libs
that are expected by packages that depend on kentutils
- There are conflicts and less-than-desirable dependencies
that can be addressed via variants
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Bumping up to ROCm 6.1.2
* Bump up rocmlir & amdsmi to 6.1.2
* Removing llvm dependency from amdsmi
* Removed redundant for loops and brought rocm-cmake in for loop
* Removing version check of deprecated versions
This adds spack.util.spack_yaml.anchorify, which takes a non-cyclic
dict/list structure, and replaces identical values with (back)
references to the first instance, so that yaml serialization will use
anchors.
`repr` is used to identify sub-dags, which in principle is quadratic
complexity in depth of the graph, but in practice the depth is O(1) so
this should not matter.
Then this is used in CI to reduce the size of generated YAML files to
30% of their original size.
* lammps: add new versions
* lammps: add directories for plugins
* lammps-example-plugin: new package
Example illustrating how to install a LAMMPS plugin, based on the
example part of the LAMMPS distribution.
If netlib-lapack is built with ~external-blas, it internally links
liblapack.so with libblas.so, meaning that whenever netlib-lapack is
used as a lapack provider, the package must also be a blas provider.
Conversely using netli-lapack as a blas provider does not imply that it
also must provide lapack, but nothing is lost disallowing that...
In practice people don't care about having their build dependencies reinstalled with say cflags=-O3 if that is what is set at the input spec, so restrict propagation to link/run deps only.
Also simplify the encoding in asp.
* Changed test method from old method to new method
* Corrected test name and added docstring
* Split test method into stand alone-tests
* Added SkipTest
* code refactor
* removed repeated code
* Remove exe from test part purpose
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve proj version selection to avoid too new of a proj with older api usage in vtk9
* Proj constraint range around 8.2.1
---------
Co-authored-by: John Parent <john.parent@kitware.com>
Fix the following on Windows:
* `spack load --list` (this printed 0 packages even if packages were
loaded)
* `spack unload <package>` (this said that the package is not loaded
even if it was)
Update unit tests for `spack load` to also run on Windows (specifically
for ".bat"). This involved refactoring a few tests to parameterize
based on whether the unit tests are being run on a Windows system
(and to account for batch syntax).
* Adding additional versions to plink2 and switching to tarballs to allow for better version detection in the future
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
The pcluster pipeline runs spack external find and autodetects the most ancient of zlib versions on the system: 1.2.7, release date May 3, 2012.
Turns out some packages require parts of the zlib api added within the last decade.
* update e4s stacks
* adios2 +rocm: disable kokkos due to spack issue #44832
* comment out mgard+cuda due to spack issue #44833
* comment out cabana, legion, arborx due to kokkos spack issue #44832
* comment out slepc, petsc due to petsc spack issue #44600
* comment out adios2+rocm due to kokkos rocm spack issue #44832
* comment out kokkos due to spack issue #44832
When we changed how to deal with errors in November,
we didn't realize that for an unconstrained choice
rule it is more important in the heuristic to guess
what is NOT in the answer set, since it will be the
majority of options.
Previously this was following automatically from what
was in the answer set, via `1 { ... } 1` cardinality
constraints.
Here we improve the heuristic and the solve time for specs.
* Patched CMakeLists.txt to find drm include at libdrm - archlinux
* Restricted usage patch to versions 34 onwards
* Update var/spack/repos/builtin/packages/rdma-core/libdrm.patch
---------
Co-authored-by: Mark Abraham <Mark.J.Abraham@gmail.com>
* Spglib: add version 2.4.0
* DLA-Future: fix +test option
* elsi
* elsi improvements
* [@spackbot] updating style on behalf of RMeli
* remove test variants
* fix ntpoly and and use externals as default
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* py-multiqc: updating to @1.21, adding new dependency py-pyyaml-env
* Correcting dependencies
* Drop matplotlib depends_on() for versions of multiqc not included in package.py
---------
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
* Allow zlib to find external installations.
* Apply suggestions from code review
* Fixed the determine_version function to loop over all of the potential
libraries that could be installed by zlib.
---------
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
* Update hohqmesh and feq-parse versions
Update feq-parse homepage to new documentation page
Update feq-parse license to be consistent with 2.2.2 release
---------
Co-authored-by: fluidnumerics-joe <fluidnumerics-joe@users.noreply.github.com>
This commit adds new versions of actsvg. I have also taken the liberty
of adding myself a maintainer as I work actively on the ACTS project and
its dependencies, and would like to start keeping the Spack specs more
up to date.
* e4s external rocm ci: bump rocm stack to v6.1.1
* comment out exago+rocm due to issue with raja@0.14.0 see spack issue #44593
* comment out adios2+rocm due to spack issue #44594
* comment out petsc+rocm due to spack issue #44600
* comment out sundials+rocm due to spack issue #44601
* comment out slepc+rocm due to petsc spack issue #44600
* comment out tau+rocm due to spack issue #44659
* comment out ecp-data-vis-sdk due to spack issue #44745
* packages: register rocm-core as external
* re-enable tau due to issue #44659 having been resolved
* use latest ci image: ecpe4s/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.1.1:2024.06.17
* comment out paraview due to spack issue #44745
* comment out ecp-data-vis-sdk +vtkm due to issue https://gitlab.spack.io/spack/spack/-/jobs/11632511
* Update for 2.0.1
* cloudpickle dependency is only 'run'
* Follow new formatting guidelines
* black wants trailing commas
* Simplified version ranges, as recommended by @tldahlgren
* vtk-m: Changed test method names and skipping non-applicable tests from old to new approach
-----
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* add cray detection taken from upcxx
* add CUDA/ROCm support
* add numerous pass-through options to Chapel build,
like gpu_mem_strategy, comm_substrate, etc.; all variants are
translated to analogous CHPL_* environment variables. As a side
effect, this defines a number of environment variables that are
not actually used by Chapel.
* Define LD_LIBRARY_PATH, LIBRARY_PATH, and PKG_CONFIG_PATH to
help programs built with Chapel properly locate needed runtime
dependencies
---------
Co-authored-by: bonachea <dobonachea@lbl.gov>
* Projsync needs curl
This fixes a proj compilation error from paraview
* tiff does not set TIFF_INCLUDE_DIR tested by proj
* TIFF patch exists for proj 9.2
* Add comments with location of proj patch
* Use TIFF patches for different cmake versions
* Use modules starting cmake 3.28 and variables before that
* Fix black style
* Simpler patches, add newer proj
* Remove duplicated flag
* Deprecate PROJ 4.8 and older
* Remove PRIVATE
* Deprecate PROJ 7.0.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Not sure why I had this here, as `/bin/sh` will find the first spack in `PATH` just like
`env`.
- [x] remove `/usr/bin/env` and avoid an extra process launch.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* omega-h: reformatted from old test method to new test method
* omega-h: added proper output checking
--------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The ``+cluster` variant requires there be an MPI family known in the
spec. When using externals it is easy to miss-configure this requirement
leading to a runtime exception which is not desirable. This converts the
exception to a package rule.
* xorg libs: new versions
* xorg pkgs: update homepage from cgit to gitlab
* xorgs pkgs: fix homepage since, yeah, I did edit those 122 files by hand...
* libxmu: don't need the .0 patch version here
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Fixed some issues with installation breaking due to mysql_client
* Remove debugging details
* Adding client_only to deps
* Adding client_only to variant and deps
* Undoingw client_only to variant and deps
* py-pyopengl: new simple package
* py-pyopengl: Fix typo in comment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pyopengl: add variants and dependencies
* py-pyopengl: build from source and improve variants
* py-pyopengl: use corrected freeglut libs
* py-pyopengl: update copyright
* py-pyopengl: remove duplicate conflict clause
* py-pyopengl: change dependency to link
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Core change: logic for extracting RPATHs from modules may return
`None`: filter this out of the set of RPATHs that is auto-generated
* Core change: `CachedCMakePackage` no longer adds ldflags to
`CMAKE_STATIC_LINKER_FLAGS`: generally these flags are not appropriate
for static linking (e.g. invocation of `ar`)
* [mfem] Add version 4.7
* [mfem] Add variant for precision (single/double). Enforce consistency
for precision amongst mfem and hypre/petsc/mumps dependencies
* [mfem] Add cxxstd (and related constraints preventing use of
old cxxstd values for newer versions of some dependencies)
* [hypre] In line with prior point, added support for specifying
precision
* [petsc] Add config option to avoid error when building against
`superlu-dist+rocm`
* [hiop] add proper `raja`/`umpire`/`camp` version constraints for
`hiop` versions 0.3.99-0.4.x; require `+raja` for `+rocm`, and
add dependency on `hiprand` for `+rocm`
* [butterflypack, mfem, strumpack, suite-sparse] Require
`CRAYLIBS_{target-family}` env var to be defined
* [suite-sparse] versions `@7.4:` changed install location of headers:
add symlink from old location to new location
* [zlib-ng] Fix error where shared libs were not successfully built for
`%cce@17` (the build did not fail, but the finished `zlib-ng%cce@17`
install did not have shared libs)
* suite-sparse library path can be lib or lib64
* fixing style check error
* Adding the missing import os
* Style check error fix
* Updating patch for 6.1.0 with correct lib path
* Preserve higher weight for CLI git ref versions
Currently the concretizer fails if you reuse a git ref version
that has already been installed but modify the spec at all.
See #38484 for futher diagnosis
The issue here is that since there is no established provenance for
these versions the highest weight they are currently assigned is
that of prior install. Re-use checks then fail because the weight of
the version is identical to the solver.
Ironically, these versions are given the highest weights possible when
specified on the CLI for the first time. They should only appear in a
DAG if they are an exact match or if the user specifies them at the CLI.
Therefore it makes sense to preserve their higher ordering.
Getting this right is critical to moving all branch based versions to a pinned
git-ref in the future.
* [@spackbot] updating style on behalf of psakievich
* Update lib/spack/spack/solver/asp.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Add provenance specific to git ref installs
* Sensitvity to name that I could not track down
* Add regression test
* Adjust test
* Add prefer standard unit-test
* Style
* Add required mock
* Format and mark
* Make unit-test case reproduce CLI investigation
* Remove unnecessary mock package
* [@spackbot] updating style on behalf of psakievich
* Use already developed fixture
* Add zlib-ng to mocks again
* Remove accidental adds
* Remove maintainer
* [@spackbot] updating style on behalf of psakievich
* Rename test file
* [@spackbot] updating style on behalf of psakievich
* Remove unused imports
* Update tests
* [@spackbot] updating style on behalf of psakievich
* Style
* Update lib/spack/spack/test/concretize.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Update solver rule
* Duplicate installed rules for installed_git_version
* Revert "Duplicate installed rules for installed_git_version"
This reverts commit 17223fc8d1.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
* libdistributed: renamed test to check_test
* libpressio-rmetric: renamed test method
* libpressio-tools: renamed test method
* Renamed build-time test methods for: libpressio-opt, libpressio-tthresh, opened
---------
Co-authored-by: Jackson Peter Lawrence <lawrence31@borax5.llnl.gov>
* Provide instructions for building a developmental version of lcov.
* style fix
* style fix
* Promote single version git URL to the package level.
* Formatting fix.
* verilator: Some environment varibles are no longer required
* Revert depends_on for flex back to the deafult case
* Use decorator, when, for setup_run_environment insdtead of if inside function
* libtiff: add v4.6.0 and default disable opengl (#44545)
* libtiff: Fix typo in CMake key
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: Broader description of OpenGL variant
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: reformat using spack style black recommendation
* libtiff: couple opengl flag with autotools
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There hasn't been a release in almost a decade, and build failures with
GCC 14 were reported. I don't think it makes sense to patch it, since
the project moved over to `util-linux`, and Spack's default provider is
that package. Better to just get rid of it in the next Spack release.
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/pcre-8.45-33dfhul
[+] /opt/apps/spack/libpciaccess-0.17-2qdxdjo
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/qhull-2020.2-klc7ewb
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/alsa-lib-1.2.3.2-a7icjdy
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/yasm-1.3.0-v4etmmm
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/unzip-6.0-mqftjtp
[+] /opt/apps/spack/libjpeg-turbo-3.0.0-vjvivme
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/libpng-1.6.39-kiuku4y
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/swig-4.0.2-fortran-z3sbnze
[+] /opt/apps/spack/binutils-2.42-vkjcvfr
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/hdf5-1.14.3-sbuiw6q
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/ffmpeg-6.1.1-2vhrbda
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/freetype-2.13.2-in4taxi
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/hwloc-2.9.1-fhoz6al
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/lua-5.3.6-47356ia
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/elfutils-0.190-uswzaiw
[+] /opt/apps/spack/py-pytz-2023.3-ojdlhrd
[+] /opt/apps/spack/py-cycler-0.11.0-b7mjvv7
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/llvm-14.0.6-3nosumn
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-six-1.16.0-iv6iv3q
[+] /opt/apps/spack/py-pybind11-2.12.0-5rvupjv
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-pyparsing-3.1.2-fq6imlt
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/py-llvmlite-0.41.1-qom3l5h
[+] /opt/apps/spack/py-astropy-4.0.1.post1-xbojixg
[+] /opt/apps/spack/py-python-dateutil-2.8.2-kzdfskc
[+] /opt/apps/spack/py-numexpr-2.8.4-fc5xc5s
[+] /opt/apps/spack/py-h5py-3.11.0-y6drk3j
[+] /opt/apps/spack/py-tifffile-2023.8.30-oof4try
[+] /opt/apps/spack/py-pygments-2.16.1-stgrccl
[+] /opt/apps/spack/py-mdurl-0.1.2-nqk43ry
[+] /opt/apps/spack/py-scipy-1.13.0-vxjgfov
[+] /opt/apps/spack/py-contourpy-1.0.7-jg5lhss
[+] /opt/apps/spack/py-pillow-10.3.0-ijh2cju
[+] /opt/apps/spack/py-kiwisolver-1.4.5-vdh5sq5
[+] /opt/apps/spack/py-fonttools-4.39.4-x5ydbox
[+] /opt/apps/spack/py-bottleneck-1.3.7-ztsm4lg
[+] /opt/apps/spack/py-lazy-loader-0.4-k7hgvka
[+] /opt/apps/spack/py-numba-0.58.1-hzvrjwj
[+] /opt/apps/spack/py-markdown-it-py-3.0.0-l4p4qv5
[+] /opt/apps/spack/py-imageio-2.34.0-z5hu4yc
[+] /opt/apps/spack/py-matplotlib-3.8.4-azq2fzm
[+] /opt/apps/spack/py-pandas-1.5.3-p3gnh6t
[+] /opt/apps/spack/py-rich-13.4.2-okhgwwz
[+] /opt/apps/spack/py-networkx-3.1-b54br7r
[+] /opt/apps/spack/py-scikit-image-0.23.2-cvyzh3t
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
#40773 introduced python-venv, which improved build isolation and avoids issues with,
e.g., `ubuntu`'s system python modifying `sysconfig` to include a (very unwanted)
`local` directory within the default install layout.
This addresses a few cases where #40773 removed functionality, without harming the
default cases where we use `python-venv`.
Traditionally, *every* view with `python` in it was essentially a virtual environment,
because we would copy the `python` interpreter and `os.py` into every view when linking.
We now rely on `python-venv` to do that, but only when it's used (i.e. new builds) and
only for packages that have an `extends("python")` directive.
This again makes every view with `python` in it a virtual environment, but only
if we're not already using a package like `python-venv`. This uses a different
mechanism from before -- instead of using the `virtualenv` trick of copying `python`
into the prefix, we instead create a `pyvenv.cfg` like `venv` (the more modern way
to do it).
This fixes two things:
1. If you already had an environment before Spack `v0.22` that worked, it would
stop working without a reconcretize and rebuild in `v0.22`, because we no longer
copy the python interpreter on link. Adding `pyvenv.cfg` fixes this in a more
modern way, so old views will keep working.
2. If you have an env that only includes python packages that use `depends_on("python")`
instead of `extends("python")`, those packages will now be importable as before,
though they won't have the same level of build isolation you'd get with `extends`
and `python-venv`.
* views: avoid making client code deal with link functions
Users of views and ViewDescriptors shouldn't have to deal with link functions -- they
should just say what type of linking they want.
- [x] views take a link_type, not a link function
- [x] views work out the link function from the link type
- [x] view descriptors and commands now just tell the view what they want.
* python: simplify logic for avoiding pyvenv.cfg in copy views
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
In #30882, we made Spack ignore `-Werror` calls so that it could more easily build
projects that inject `-Werror` into their builds. We did this by translating them to
`-Wno-error` in the compiler wrapper. However, some compilers (like `nvhpc`) do not
support `-Wno-error`. We need to exclude them from this feature until they do.
- [x] make a property on `PackageBase` for `keep_werror` that knows not to use it for
`nvhpc`.
- [x] update property so that it keeps only the specific `-Werror=...` args for newer nvhpc's,
which support `-Wno-error` but not `-Wno-error=...`
---------
Co-authored-by: William Mou <william.mou1024@gmail.com>
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
When Spack concretizes environments, it prints every (newly concretized) root spec
individually with all of its dependencies. For most reasonably sized environments, this
is too much output. This is true for three commands:
* `spack concretize` when concretizing an environment with newly added specs
* `spack install` when installing an environment with newly added specs
* `spack spec` with no arguments in an environment
The output dates back to before we had unified environments or nicer spec traversal
routines, and we can improve it.
This PR makes environment concretization output analogous to what we do for regular
specs. Just like `spack spec` for a single spec, we show all root specs with no
indentation, so you can easily see the specs you explicitly requested. Dependencies are
shown:
1. With indentation according to their depth in a breadth-first traversal starting at
the roots;
2. Only once if they appear on paths from multiple roots
So, the default is now consistent with `spack spec` for one spec--it's `--cover=nodes`.
i.e., if there are 100 specs in your environment, you'll get 100 lines of output.
If you want to see more details, you can do that with `spack spec` using the arguments
you're already familiar with. For example, if you wanted to see dependency types and
*all* dependencies, you could use `spack spec -l --cover=edges`. Or you could add
deptypes and namespaces with, e.g. `spack spec -ltN`.
With no arguments in an environment, `spack spec` concretizes (if necessary) and shows
the concretized environment. If you run `spack concretize` *first*, inspecting the
environment repeatedly with `spack spec` will be fast, as everything is already in the
`spack.lock` file.
- [x] factor most logic of `Spec.tree()` out of `Spec` class into `spack.spec.tree()`,
which can take multiple specs as roots.
- [x] make `Spec.tree()` call `spack.spec.tree()`
- [x] `spack.environment.display_specs()` now uses `spack.spec.tree()`
- [x] Update `spack concretize`
- [x] Update `spack install`
- [x] Update `spack spec` to call `spack.spec.tree()` for environments.
- [x] Continue to output specs individually for `spack spec` when using
`--yaml` or `--json`
* Apply patch from upstream to allow vtk compilation with %gcc 12 and 14.
* Fixes#44331
* fix soec usage
* fix compiler version range
* Finalize version range, switch to .diff file
---------
Co-authored-by: Chrismarsh <Chrismarsh@users.noreply.github.com>
* Now need to explicitly depend on libfabric for Cray EX
* Ensure build uses the selected CUDA and ROCm versions
* Correct spelling on `configure --with-ldflags`
* Patch a defect regarding `configure --with-ldflags`
* Default to Spack's copy of GASNet-EX, which is newer than embedded
e2fsprogs: adding version 1.47.0 and fuse2fs variant
fuse-overlayfs: adding version 1.13
squashfuse: adding version 0.5.1 and 0.5.2
e2fsprogs: fixing audit errors
apptainer: expanding deps to cover more versions, fix binary_path logic.
apptainer: add go version deps, fix e2fsprogs patch sums, rework dep paths
fuse-overlayfs: tightening up the libfuse dep
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* util-macros: ensure url_for_version only used for older versions
* util-macros: use url.substitute_version after xz -> bz2
* util-macros: mv url_for_version down the file
`BuildcacheBootstrapper` uses `Spec.intersects` to match specs needed
for bootstrapping against the binary cache. The specs were not
sufficiently-detailed to prevent matching e.g. cached binaries for
Mac OS on Windows; this commit adds the platform to each requested
bootstrap spec to prevent that.
* initial commit for RoCm-6.1.1 release
* fix style issues
* update the version for rocmlir and rocm-cmake
* restrict the patch for 6.0.0 release
* fix build failure for hipsolver-6.1.1 release
* update the hipblaslt for rocm-6.1.1 release
* add the patch for hipsparselt for 6.1.1 rel
Remove support for `cray` as a separate platform.
Any platform previously detected as `cray` is now detected as `linux`.
Users who still need platform=cray have to stick to Spack 0.22
Fixes a bug in the concretizer where specs depending on a host
incompatible libc would be used. This bug triggers when nothing is
built.
In the case where everything is reused, there is no libc provider from
the perspective of the solver, there is only compatible_libc. This
commit ensures that we require a host compatible libc on any reused
spec, additionally to requiring compat with the chosen libc provider.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Patch to add hsa prefix/include path:wq
* Style check error fix
* The patch need to be applied to 2024.01.stable. ROCm 6.0 requires hpctoolkit 2024.01.1 or later
* Fixing style chec error
* Adding a py4j variant that requires Java via spack to avoid situations where a system doesn't have Java and py4j expects it
* Adding new versions of py-pyspark
* Adding a new variant to require java (via py4j) and clean up dependency handling
* Adding myself as a maintainer for py-pyspark and py-py4j
* Fix overlooked version bump in py4j
* Version bump to meet py-spark expectations
* Version bump to add latest compatibile version with pyspark
* Matching py-grpcio bump
* Adding variants and dependents for pyspark
* Adding runtime deps
* Changing default java requirement. I'm not sure this is the right call
* Changing py4j with java dependency handling
* Fix style
* Update package.py fix unnecessary f-strinh
* Make +java the default for both
* Fix nested deps
* Revert styles after default change
* Added new versions and GCC 14 compatbility conflicts
* Added new versions and compatibility conflicts for gcc 14
* Added new versions paired to arrow (for gcc14 compat)
* Update py-protobuf compiler conflict
* Update depends to match
See https://github.com/grpc/grpc/blob/master/src/python/grpcio_status/setup.py
* Updating dependencies and conflicts for py-googleapis-common-protos. Added new version to avoid future issues
* Remove upper bound version on py-protobuf and add default_args
* Adding new versions and updating dependencies back to versions 1.35.0
* Updating oldest numpy deps
* Fixing merge
* bit more cleaniness for var/spack/repos/builtin/packages/py-googleapis-common-protos/package.py
* Adding latest matching version of py-grpcio and py-grpcio-status
* Update package.py
https://github.com/spack/spack/pull/44263#discussion_r1612317943
* Update dependencies
* Adding additional versions for dependent packages. Deprecated two versions: 1.16 is old, built for python ~3.6, and does not build for 3.8. 1.52.0 was removed from pypi
* Revert py-grpcio-tools changes. Will include in separate PR
* Adding patches and constraints to get 1.48 to build as it's a dependency that is called out for some other packages
* Updating to account for yanked packages for dependencies
* Fix style
* Update sha256 for py-grpcio v0.16.0 to reflect change
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* py-astropy: Add version 6.1
Added build info for version 6.1 in py-astropy. Requires a new additional package, astropy-iers-data which has been included as py-astropy-iers-data to match with spack's general naming conventions.
Below is the output of the spack install showing successful build fro version 6.1 and the new py-astropy-iers-data package
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-pyerfa-2.0.0.1-kyfazhs
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/git-2.45.1-tuc5jnb
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
==> Installing py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj [52/55]
==> No binary for py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj found: installing from source
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj
Stage: 1.74s. Install: 0.93s. Post-install: 0.52s. Total: 3.36s
[+] /opt/apps/spack/py-astropy-iers-data-main-ukchsfz
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-qdhxchg
==> Installing py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo [55/55]
==> No binary for py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> Warning: Module file /opt/modulefiles/spack/Core/py-astropy/6.1.0.lua exists and will not be overwritten
==> py-astropy: Successfully installed py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo
Stage: 1.29s. Install: 1m 5.77s. Post-install: 0.60s. Total: 1m 7.94s
* Removed extra-whitespace
A blank line had white space, removed the white space
* Additional formatting changes for black
* Additional package updates
Based on previous recommendations updated py-astropy and py-astropy-iers-data packages. Also added a new version to py-pyerfa package to match the 6.1.0 dependencies better
Of importance in these updates, I did add pypi and version info to py-astropy-iers-data. Originally I had argued that this package updates quite frequently (on a weekly basis) and so it did not make sense to include pypi/versions and we should instead use the non-version git-repo structure based on the master branch. However, when I tried to build the package py-setuptools-scm errored out when trying to build the package:
/opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro/lib/python3.10/site-packages/setuptools_scm/git.py:163: UserWarning: "/tmp/root/spack-stage/spack-stage-py-astropy-iers-data-main-iw2mdzlukb37mkcbcozjjefjoefw2eyp/spack-src" is shallow and may cause errors
I believe this is due to the download file/stage directory not containing the version and instead including the branch. I changed the package to use versions instead and it worked just fine as shown below. In addition, when I had done some preliminary testing, the package installed fine using the non-version master branch. When I checked, that installation it used py-setuptools-scm@7.1 while in this installation run it used a much more recent 8.0.4 so it is possible that somewhere between scm7.1 and 8.0.04 something changed that caused this error to show up. Since the setuptools-scm package has something to do extracting package versions, I imagine it's some kind of mismatch issue
Output from build:
spack install py-astropy
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/git-2.45.1-zu6qkoc
[+] /opt/apps/spack/py-markupsafe-2.1.3-isgtki6
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
==> Installing py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a [49/57]
==> No binary for py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a found: installing from source
==> Fetching ac8a6fe91c.tar.gz
==> No patches needed for py-extension-helpers
==> py-extension-helpers: Executing phase: 'install'
==> py-extension-helpers: Successfully installed py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a
Stage: 0.37s. Install: 0.88s. Post-install: 0.52s. Total: 1.89s
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
==> Installing py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m [50/57]
==> No binary for py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m found: installing from source
==> Fetching 350b18f967.tar.gz
==> No patches needed for py-cython
==> py-cython: Executing phase: 'install'
==> py-cython: Successfully installed py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m
Stage: 1.07s. Install: 2m 49.80s. Post-install: 0.40s. Total: 2m 51.37s
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
[+] /opt/apps/spack/py-jinja2-3.1.2-wacpq7j
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro
==> Installing py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 [55/57]
==> No binary for py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 found: installing from source
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/7f/7fff3d3404ae8560533ac0e685db7acc02c4d8984faa4ac3d607096879fba2d1.tar.gz
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6
Stage: 0.07s. Install: 1.21s. Post-install: 0.39s. Total: 1.88s
[+] /opt/apps/spack/py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6
==> Installing py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp [56/57]
==> No binary for py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/pyerfa/pyerfa-2.0.1.1.tar.gz
==> No patches needed for py-pyerfa
==> py-pyerfa: Executing phase: 'install'
==> py-pyerfa: Successfully installed py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp
Stage: 0.93s. Install: 17.72s. Post-install: 0.33s. Total: 19.15s
[+] /opt/apps/spack/py-pyerfa-2.0.1.1-pkokp6u
==> Installing py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs [57/57]
==> No binary for py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> py-astropy: Successfully installed py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs
Stage: 1.16s. Install: 1m 3.34s. Post-install: 1.05s. Total: 1m 5.77s
[+] /opt/apps/spack/py-astropy-6.1.0-5brbkjn
* Updated to match with black formatting
Change the installer to take `([pkg], args)` in the constructor instead
of `[(pkg, args)]`. The reason is that certain arguments are global
settings, and the new API ensures that those arguments cannot be
different across different "build requests".
The `explicit` install arg is now a list of hashes, and the installer is
no longer responsible for determining what package is installed
explicitly. This way environment installs can simply pass the list of
environment roots, without them necessarily being explicit build
requests. For example an env with two roots [a, b], where b depends on
a, would not always cause spack install to mark b as explicit.
Notice that `overwrite` already took a list of hashes, this makes
`explicit` consistent.
`package.do_install(explicit=True)` continues to take a boolean.
* legion: add pip dependency for build
pip is needed for the build when the Legion Python binding is enabled
* legion: add variants for gc logging and system OpenMP use
This also removes the `cmake_cxx_flags` variable from `cmake_args()`
because that variable had no effect.
* [@spackbot] updating style on behalf of mpokorny
* legion: use spec.satisfies() in cmake_args()
e.g, replace use of '"+foo" in spec' with 'spec.satisfies("+foo")'
* legion: avoid configuring with multiple "-DLegion_REDOP_COMPLEX=ON" arguments
In the previous version, when both '+redop_complex' and '+bindings'
was set, two instances of "-DLegion_REDOP_COMPLEX=ON" arguments were
generated for cmake configuration.
* legion: fix value of "Legion_OUTPUT_LEVEL" for configuration
the previous version had no effect on setting the configuration value
---------
Co-authored-by: mpokorny <mpokorny@users.noreply.github.com>
The windows wrappers for basic functions like `os.symlink`,
`os.readlink` and `os.path.islink` in the `llnl.util.symlink` module
have bugs, and trigger more file system operations on non-windows than
they should.
This commit just binds `llnl.util.symlink.symlink = os.symlink` etc so
built-in functions are used on non-windows
`spack clean <spec>` will now resolve specs based on the active environment if one is active.
If an env is active but no matching spec is found, this will fall back on fully concretizing.
Before this PR, if Spack could see a possibility to reuse a spec that
doesn't match a strong preference, it would do so. After the PR, a
strong preference would take precedence.
`glibc` and `musl` provide a basic implementation of `iconv` (`iconv`,
`iconv_open`, `iconv_close`), but in practice the installation may be
missing the character encoding methods to make them usable. On Fedora
for example, users need to
```yum install glibc-gconv-extra```
to get the character encodings that `gettext` requires during configure,
namely EUC-JP. Users may not have permissions to install the missing
parts of glibc.
Since Spack can install `libiconv`, it is simpler to use that by
default.
* Spglib: add version 2.4.0
* dla-future-fortran: new package version 0.1.0
* [@spackbot] updating style on behalf of RMeli
* apply suggestion and add maintainer
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* [emacs] When installing on Mac OS, build and install the native
Emacs.app along with the standard executables.
* [emacs] Make the GUI build on Mac optional by adding "gui" variant
* Apply reviewer suggestion
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Add emacs version 29.3
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
avoid calling `spec.target` when None.
When an external compiler package has an `os` set but no `target` set, Spack
currently falls into a codepath that calls `spec.target` (which itself calls
`spec.architecture.target.Microarchitecture`) when `spec.architecture.target`
is None, throwing an error.
e.g.
```
packages:
gcc:
externals:
- spec: gcc@12.3.1 os=rhel7
prefix: /usr
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* glew: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-demos: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-glu: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* paraview: fix dependency on glew
* mesa: group dependency on when("+glx")
* Add missing dependency on libxml2
* paraview: remove the "osmesa" and "egl" variant
Instead, enforce consistency using the "gl" virtual that allows
only one provider.
* visit: remove osmesa variant
* Disable paraview in the aws-isc stacks
* data-vis-sdk: rework constrains to enforce front-ends
* e4s-power: remove redundant paraview
* Pipelines: update osmesa variants
* trilinos-catalyst-ioss-adapter: make gl a run dependency
Starting from 24.04, armpl-gcc will only have three packages files: dev, rpm, and macOS.
Only one version for gcc is provided, so the changes in the code reflect that the tar provided
for gcc is only one rather than many.
* Remove mesa18 and libosmesa
mesa18 was introduced in #19528 as a way to maintain the old
autotools build of mesa separate from the new meson build.
We could add a second build system to mesa, but since mesa18 has
been deprecated for a long time, we'll just remove it.
libosmesa was used to multiplex the gl provider between mesa18
and mesa, and is thus unecessary. Remove it to reduce complexity
in the graphical stack.
* Remove references to mesa18 and libosmesa
* vtk: rework dependency on gl and osmesa
* memsurfer: rework dependency on vtk
* visit: minimal fix to avoid having both osmesa and glx
* Use hdf5::hdf5 on Windows from Paraview CMake
This patch is already applied on VTK 9 or greater.
* Add comments stating that vtk and paraview patches are the same and should be modified in concert.
This fixes a bug occurring when two root specs need to select
old versions, and these versions have the same penalty in the
optimization. This sometimes caused an older version to be
preferred to a more recent one.
The issue was the omission of `PackageNode` in the optimization
tuple.
* rayleigh: new package
* Update var/spack/repos/builtin/packages/rayleigh/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* split edit into three methods
* add comments to clarify use of configure
* rayleigh: copyright year
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Use correct path separator in get_all_package_diffs for all platforms.
Ensures correct package change computation on Windows when pruning unchanged specs in Gitlab CI
* Update, fix serenity & serenity-libint
Version 1.6.1 of `serenity` is added, some dependencies are made more
explicit, some options are improved or fixed.
The url of `serenity-libint` is fixed. The old url is not reachable
anymore.
* Use upstream patch, modify cmake patch
* Update var/spack/repos/builtin/packages/serenity/package.py
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* heimdall: Astronomy software package. Requires dedisp and psrdada (included as part of this commit)
* Updated packages to align with Spack's style
Minor updates based on wdconinc's comments regarding Spack's style guide
* [@spackbot] updating style on behalf of aweaver1fandm
* Minor edits to fix copyright year and dedisp install
Fixed copyright year to be 2024 instead of 2023
Removed the overridden version of install and created a preinstall function to create the missing lib and inc directories, therefore allowing the default install to run
Here is output from the spack install of heimdall showing successful build with cuda. If necessary I do a spack clean and freshly install it
./spack install heimdall +cuda cuda_arch=86
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libiconv-1.17-enpmbhsi3kztebwmpclpub2afhlbr3gy
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/xz-5.4.1-pte76kujkezxb3laqse3o4sctlbygsaw
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zlib-1.2.13-utlfo5ltxz5v5bckirn5v3amtbxjdvwh
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/bzip2-1.0.8-x6navz7ucgfnb5xq7aelqmgd4zxsz5bs
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libmd-1.0.4-ncomhrodpdul4dm64o6b7426fhmc2u64
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/ncurses-6.4-c77h34rooycbzapxjvc27sg5td5jiwyb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zstd-1.5.5-eporpybumydxveg5rwtfzysrsu4eqzcv
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libffi-3.4.4-vskntokaojclfqxjfzkbyirkeogddbpx
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxcrypt-4.4.33-gtpn32p6mxztul3c3dxzqj7gvcyh555j
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/util-linux-uuid-2.38.1-g322a5peqjaad6gl5q64cdu4qo7kvw6o
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxml2-2.10.3-pza3kz2mtbncbbeim6rejfqkgftnf4rz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/openssl-1.1.1t-adbquvgg4qpc3vq6jynf44qzq3gfwrv5
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/pigz-2.7-e7mxj4ya2u4a6zb4hu64g7docujmkxeb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libbsd-0.11.7-wh4xleivbe7wndiqt5nsehzlfrccnjcg
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/readline-8.2-hhb647bwmbcj7iwpmtetbylninfm5rxf
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/cuda-11.7.1-osue2sx5rv7dgzhsmaemydpwhyribxng
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/tar-1.34-35f5gki2ycxmy5zd7zs5tsvp3xoszxum
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/expat-2.5.0-4gizyhhqklciuyrbyinq2tdggt73gds4
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gdbm-1.23-w3uzihtubj2iwv6es55fis6nt2q5zwlr
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/sqlite-3.40.1-m4ntzvuupsnbtkdfbz7oqpbjdlaffp2a
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/dedisp-1.0-pq6r3jnyxq6jzoygz3fp2e6jc2ojpvap
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gettext-0.21.1-hglzdeadmgkzjb76bmemt6dnulfkrpha
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/python-3.10.10-khu36qq4p2te7jf475ewr2h7egidekfl
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/psrdada-master-by4w6mrpcfnoihlhos2jcfo2roiyagaz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/heimdall-1.0-ohtdnltuhhejysshcert25h6nmuvluqp
* [@spackbot] updating style on behalf of aweaver1fandm
Restrict the hpcrun-fmt.txt patch to :2022. It's fixed in the code
after that, and in recent develop, some code paths have moved causing
this patch to fail.
This fixes an issue where ghcr, gitlab and possibly other container registries paginate tags by default, which violates the OCI spec v1.0, but is common practice (the spec was broken itself). After this commit, you can create build cache indices of > 100 specs on ghcr.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* pythia8: prefer 8.310
* [@spackbot] updating style on behalf of wdconinc
* pythia8: filter_file to remove sed n
* Revert "[@spackbot] updating style on behalf of wdconinc"
This reverts commit e2a3decaffbd3f464d1bd992025e1812df49f088.
* Revert "pythia8: prefer 8.310"
This reverts commit 568cb056b87129085e245d9dbef1732ee1c6c0aa.
* [@spackbot] updating style on behalf of wdconinc
* pythia8: comment for fix
* pythia8: fix style
* pythia8: filter_file with raw string because of escaped pipe
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
The release tarballs are not available anymore which means autoconf, automake and libtool are always needed.
The NVHPC specific patches don't make sense anymore being that the patched files are not distributed in the new tar files.
Add support for Gitlab CI on Windows
This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows
infrastructure).
* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Symlinks on Windows can use longpath prefixes (\\?\); these are fine
in the context of win32 API interactions but break numerous facets of
Spack behavior that rely on string parsing/matching (archiving,
binary distributions, tarball extraction, view regen, etc).
Spack's internal readlink method (llnl.util.symlink.readlink)
gracefully handles this by removing the prefix and otherwise behaving
exactly as os.readlink does, so we should prefer that in all cases.
* py-matplotlib: qualify when to do a post install
Older versions of py-matplotlib don't seem to have some of the
files that the post install step is trying to install.
Looks like the files first appeared in 3.6.0 and later.
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
* Change install paths for older matplotlib
---------
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add maintainer and fix linting
* allow for fewer deps in archive
* use meson for archive packages
* Fix version spec and f-string
* fix blas dependency links
* Add new release to spack
* Fix checksums for latest release
* Update version of iperf3 from 3.6 to 3.16
Spack currently only explicitly has version 3.6 of the iPerf3 package
(out of ESnet / LBNL). This makes the default the latest version of 3.16,
and adds some other versions (found in some Linux distros, for possible
compatibility purposes).
* iperf3: update to 3.17; update 3.6 hash for new url
* protobuf: update hash for patch needed when="@3.4:3.21"
* Revert "protobuf: update hash for patch needed when="@3.4:3.21""
This reverts commit 4d168d0b27.
* initial commit to enable tensorflow-2.14 for spack built ROCm
* fix style errors
* modify hipcc patch
* updates for rocm 6.1
* updates for tf-rocm-enhanced 2.16
* fix styling
* fix styling
* add patch for 2.16
* add patch for 2.16
* Update var/spack/repos/builtin/packages/py-tensorflow/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* update rocm enhanced version names
* changes for rocm-enhanced version name change
* fix styling
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix gromacs installation with SVE. Issue 44062
* [@spackbot] updating style on behalf of paolotricerri
* Remove `neoverse_n2` target
We have removed the neoverse_n2 target as its detection is more involved
compared to neoverse_v*.
Set MPI_C_COMPILER etc. env vars when building with fujitsu-mpi, which
override CMake logic. Without using these variables to explicitly
request the Fujitsu MPI wrappers, the builtin CMake logic is unwilling
to use these wrappers unless the Fujitsu compiler is used, but they
should be used for %clang as well.
This avoids patching CMake module files like in #16864, primarily to
avoid the possibility of altering behavior for specs that do not use
%fj or ^fujitsu-mpi.
* Add openfoam to aws-pcluster-neoverse_v1 stack
* Add more apps to aws-pcluster-x86_64_v4 stack
* Remove WRF while hdf5 cannot build in buildcache at the moment
* Update comment
* Add more apps for aws-pcluster-neoverse_v1 stack
* Remove apps that currently do not build
* Disable those packages that won't build
* Modify syntax such that correct cflags are used
* Changing syntax again to what works with other packages
* Fix overriding packages.yaml entry for gettext
* Use new `prefer` and `require:when` clauses to clarify intent
* Use newer spack version to install intel compiler
This removes the need for patches and makes sure the `prefer` directives in
`package.yaml` are understood.
* `prefer` not strong enough, need to set compilers
* Revert "Use newer spack version to install intel compiler"
This reverts commit ecb25a192c.
Cannot update the spack version to install intel compiler as this changes the
compiler hash but not the version. This leads to incompatible compiler paths. If
we update this spack version in the future make sure the compiler version also updates.
Tested-by: Stephen Sachs <stesachs@amazon.com>
* Remove `prefer` clause as it is not strong enough for our needs
This way we can safely go back to installing the intel compiler with an older
spack version.
* Prefer gcc or oneapi to build gettext
* Pin gettext version compatible with system glibc-headers
* relax gettext version requirement to enable later versions
* oneapi cannot build older gettext version
* octave: use pcre2 for @8:
* Add 'pcre2' variant to octave to control pcre vs. pcre2
* Update var/spack/repos/builtin/packages/octave/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alex Richert <alexander.richert@noaa.gov>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Apparently urllib can throw a range of different exceptions:
1. HTTPError
2. URLError with e.reason set to the actual exception
3. TimeoutError from getresponse, which is not wrapped
* New package: qpdf
* Update var/spack/repos/builtin/packages/qpdf/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Fix format of cmake_args
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* archive: relative links only
Ensure all links written into tarfiles generated from Spack prefixes do not contain symlinks pointing outside the prefix
* binary_distribution: limit extraction to prefix
Ensure files extracted from spackballs are not links pointing outside of the prefix
* Ensure rpaths are properly set on Windows
* hard error on extraction of absolute links
* refactor for non link-modifying approach
* Restore tarball extraction to original impl
* use custom readlink
* cleanup symlink module
* make lstrip
* flux-sched: set the version if the ver file is missing
problem: flux-sched needs a version, it normally gets this from a
release tarball or from git tags, but if using a source archive or a git
clone without tags the version is missing
solution: set the version through cmake based on the version spack sees
when the version file is missing
* Update var/spack/repos/builtin/packages/flux-sched/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Problem: the libsodium download endpoints are not reliable,
they fail multiple times a day for several of our automated
pipelines.
Solution: use the GitHub releases and branches.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
GCC C++ headers like cstdlib use `#include_next <stdlib.h>` to wrap libc
headers. We're using `-isystem` for libc, which puts those headers too
early in the search path. `-idirafter` fixes this so `include_next`
works.
* add darshan-3.4.5 packages, update URLs
* py-setuptools version switches for py-darshan
* more py-darshan test dependencies
* try a conditional importlib_resources dependency
Add the ability to include any number of (potentially nested) concrete environments, e.g.:
```yaml
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
or, from the CLI:
```console
$ spack env create myenv
$ spack -e myenv add python
$ spack -e myenv concretize
$ spack env create --include-concrete myenv included_env
```
The contents of included concrete environments' spack.lock files are
included in the environment's lock file at creation time. Any changes
to included concrete environments are only reflected after the environment
is re-concretized from the re-concretized included environments.
- [x] Concretize included envs
- [x] Save concrete specs in memory by hash
- [x] Add included envs to combined env's lock file
- [x] Add test
- [x] Update documentation
Co-authored-by: Kayla Butler <<butler59@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co
m>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Currently SPACK_COLOR=always is not respected in the build process on
macOS, because the global `_force_color` is re-evaluated in global scope
during module setup, where it is always `None`.
So, move global init bits from main.py to the module itself.
Some specs which were excluded from reuse,
are currently added back to the solve when
we traverse dependencies of other reusable
specs.
This fixes the issue by keeping track of what
we can explicitly reuse.
This commit adds a layer of indirection to improve build isolation with
and without external Python, as well as usability of environment views.
It adds `python-venv` as a dependency to all packages that `extends("python")`,
which has the following advantages:
1. Build isolation: only `PYTHONPATH` is considered in builds, not
user / system packages
2. Stable install layout: fixes the problem on Debian, RHEL and Fedora where
external / system python produces `bin/local` subdirs in Spack install prefixes.
3. Environment views are Python virtual environments (and if you add
`py-pip` things like `pip list` work)
Views work whether they're symlink, hardlink or copy type.
This commit additionally makes `spec["python"].command` return
`spec["python-venv"].command`. The rationale is that packages in repos we do
not own do not pass the underlying python to the build system, which could still
result in incorrectly computed install layouts.
Other attributes like `libs`, `headers` should be on `python` anyways and need no change.
Currently bootstrapping from source fails because clingo requires gnupg
requires clingo.
This commit stops eager bootstrapping. We don't need `patchelf` nor `gnupg`
generally. They're bootstrapped when needed.
This creates shared infrastructure for compiler packages to implement the
detailed search capabilities from the `spack compiler find` command for the
`spack external find` command.
After this commit, `spack compiler find` can be replaced with
`spack external find --tag compiler`, with the exception of mixed toolchains.
A named env cannot contain `.` and `/`.
So when a user runs `spack env create ./here` do not error but treat it
as `spack env create -d ./here`.
Also fix help string of `spack env create`, which seems to have been
copied from `activate` incorrectly.
Since reuse is the default now, `--reuse-deps` can be confusing, as it
technically does not imply roots are fresh.
So add `--fresh-roots`, which is also easier to discover when running
`spack concretize --fre<tab>`
* WIP NWChem with ARMCI-MPI and TCE optional
* rename armci-mpi to armcimpi
* add version for git
* add ARMCI-MPI support to NWChem package
EXTERNAL_ARMCI_PATH needs to be set to the ARMCI-MPI package install prefix.
I could not get it from spec["armcimpi"] but it worked to use the dependent build environment method to export a generic environmental variable to use instead.
* i suppose i can maintain NWChem as well
* check rejects option that dozens of packages use
i do not have time to fight with this nonsense
Run . share/spack/setup-env.sh
==> Error: armcimpi version 'master' has extra arguments: 'branch'
Valid arguments for a url fetcher are:
'url', 'sha256', 'md5', 'sha1', 'sha224', 'sha384', 'sha512', and 'checksum'
Error: Process completed with exit code 1.
* style
* ARMCI-MPI needs depends_on; the rest seems fixed
* fix ARMCI selection per review feedback from @zzzoom
https://github.com/spack/spack/pull/43883#discussion_r1585014147
* address reviewer feedback from @yizeyi18
https://github.com/spack/spack/pull/43883#discussion_r1587228084
elaborate on what +extratce does, in terms of the NWChem build environment variables,
some of which are documented on https://nwchemgit.github.io/TCE.html.
* style
* Update var/spack/repos/builtin/packages/nwchem/package.py
---------
Signed-off-by: Jeff Hammond <jeff.science@gmail.com>
Co-authored-by: Carlos Bederián <4043375+zzzoom@users.noreply.github.com>
* [TAU package] Update with +rocprofv2. Updated some tests.
pdated with +rocprofv2 flag. Only works with rocm-core >= 6.0.0
Can be tested with (Omnia):
spack install tau@master +rocm+rocprofv2 %gcc@11
Needs the last commit in our local repository, can be done by modifying the "git = " line, or waiting until the public one is updated.
In the case that tests cause issues when building TAU, there is the flag:
disable_tests = False
The rocm test is disabled by default, as the PR regarding tests loading dependencies is not solved (PR#43682).
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
* py-tensorflow: fix aarch64 build
* [@spackbot] updating style on behalf of aweits
* patch format
* change patch strategy, actually get the logic correct in
the patch
* only patch 2.16.1 onwards for aarch64
* __CUDACC__ not __NVCC__
* !(defined(__NVCC__) && defined(__CUDACC__))
---------
Co-authored-by: aweits <aweits@users.noreply.github.com>
We recently switched to using the new ReadTheDocs with "addons". That includes its own
analytics, which is nice, but we also want to continue using our GA4 analytics.
Adding GA4 is no longer supported by RTD, so we have to add it manually.
- [x] re-add the gtag to all pages, manually
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
PR #43701 is broken, preventing scorep from being installed. As explained in #43700 the issue is internal to scorep and can not be fixed in the package, at least by the method being attempted here.
Adds a pre-concretization check for the Windows SDK and WGL (Windows
GL) packages as non-buildable externals.
This is a redo of https://github.com/spack/spack/pull/43459, but makes
sure to modify the configuration scope outside of the bootstrap scope:
whichever is highest-precedence in the user's environment at the time
the concretization runs, which should either be an env scope or the
~ scope.
Adds pytest fixture mocking the check for WGL and WSDK as if they were
present.
This PR gives users finer control over which specs are reused during concretization.
The value of the `concretizer:reuse` config option now can take an object with the following properties:
- `roots`: true if reusing roots, false if reusing just dependencies
- `exclude`: list of constraints used to select reusable specs
- `include`: list of constraints used to select reusable specs
- `from`: allows to select the sources of reused specs
### Examples
#### Reuse only specs compiled with GCC
```yaml
concretizer:
reuse:
roots: true
include:
- "%gcc"
```
#### `openmpi` must be used from externals, and it must be the only external used
```yaml
concretizer:
reuse:
roots: true
from:
- type: local
exclude:
- "openmpi"
- type: buildcache
exclude:
- "openmpi"
- type: external
include:
- "openmpi"
```
With v2.10+, ADIOS added a campaign manager. This is auto-enabled
if SQLite3 is found.
Add explicit control for it now and disables it by default, to avoid
picking up system dependencies or bloating by default the ADIOS2
dependencies. Also, not yet fully mature to be used by default:
https://github.com/ornladios/ADIOS2/issues/4148
* pypi build of py-simpy
* [py-simpy] toml explicitly called out
* py-simpy: New version
* py-setuptools-scm: Added new version
* py-setuptools-scm: add url_for_version
Because versions @:7 have an underscore (setuptools_scm) in the URL, and
newer versions have a dash (setuptools-scm).
* py-setuptools-scm: fix flake8 line too long
* py-simpy: Fix hash
---------
Co-authored-by: Sid Pendelberry <sid@rit.edu>
Co-authored-by: Jen Herting <jen@herting.cc>
* PackageStillNeededError: add pkg that needs spec to exception msg
* PackageStillNeededError: f-string with short fmt and hash
* PackageStillNeededError: split long string
The old concretizer creates a cyclic graph when expanding virtuals for
`iconv`, which is a bug. This hack drops glibc and musl as possible
providers for `iconv` in the old concretizer to work around it.
* Added package to build Ollama
* Update package.py
Add license and documentation
* [@spackbot] updating style on behalf of teaguesterling
* We can now use OVERRIDE_GIT_DESCRIBE to set the version in DuckDB
* Update duckdb/package.py
- use f-string
- fix version specs to address inconsistencies
* Update package.py
Fix spec definitions further
* [@spackbot] updating style on behalf of teaguesterling
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The "DOTNET_CLI_TELEMETRY_OPTOUT" environment variable should be defined when using the product, not when installing it (the installation phase is just extract the files anyway).
Also use `~` instead of `-` to check for the variant and fix the second argument for `env.set` which should also be a string.
Add debug log for external detection tests. The debug log
is used to print which test is being executed.
Skip version audit on Windows where appropriate
* Update package.py to osg 1.119 and igtf 1.128
* Remove unnecessary white space
* Add missing comma
* change url to use git and use commit hash from repository instead of ssh256sum of tar.gz
* [@spackbot] updating style on behalf of rahmans1
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
* * Add initial tests-sos package
* Remove failing call of missing setup_compiler_environment from sos
package
* Add several variants for sos package
* [@spackbot] updating style on behalf of jack-morrison
We run `extend spack_flags_list SPACK_LDFLAGS` for `$mode in ld|ccld`.
That's problematic, cause `ccld` needs `-Wl,--flag` whereas `ld` needs
`--flag` directly. Only `-L` and `-l` are common to compiler & linker.
In all build systems `LDFLAGS` is for the compiler not the linker, cause
any linker flag `-x` can be passed as a compiler flag `-Wl,-x`, and there
are many compiler flags that affect the linker invocation, like `-fopenmp`,
`-fuse-ld=`, `-fsanitize=` etc.
So don't pass `LDFLAGS` to the linker directly.
This way users can set `ldflags: -Wl,--allow-shlib-undefined` in compilers.yaml
to work around an issue where the linker tries to resolve the `libcuda.so.1`
stub lib which cannot be located by design in `cuda`.
* Updates for rocm
Updated for rocm@6
Added conflict between rocprofiler and roctracer.
Request either +rocprofiler or +roctracer when +rocm. In this case, it automatically builds for one, instead of displaying the message.
Request +rocm when using either +rocprofiler or +roctracer. In this case, it automatically builds with +rocm, instead of displaying the message.
Disabled the tests. Will update them with the new test method.
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
Some packages can't be redistributed in source or binary form. We need an explicit way to say that in a package.
This adds a `redistribute()` directive so that package authors can write, e.g.:
```python
redistribute(source=False, binary=False)
```
You can also do this conditionally with `when=`, as with other directives, e.g.:
```python
# 12.0 and higher are proprietary
redistribute(source=False, binary=False, when="@12.0:")
# can't redistribute when we depend on some proprietary dependency
redistribute(source=False, binary=False, when="^proprietary-dependency")
```
To prevent Spack from adding either their sources or binaries to public mirrors and build caches. You can still unconditionally add things *if* you run either:
* `spack mirror create --private`
* `spack buildcache push --private`
But the default behavior for build caches is not to include non-redistributable packages in either mirrors or build caches. We have previously done this manually for our public buildcache, but with this we can start maintaining redistributability directly in packages.
Caveats: currently the default for `redistribute()` is `True` for both `source` and `binary`, and you can only set either of them to `False` via this directive.
- [x] add `redistribute()` directive
- [x] add `redistribute_source` and `redistribute_binary` class methods to `PackageBase`
- [x] add `--private` option to `spack mirror`
- [x] add `--private` option to `spack buildcache push`
- [x] test exclusion of packages from source mirror (both as a root and as a dependency)
- [x] test exclusion of packages from binary mirror (both as a root and as a dependency)
If there's no compiler we currently don't have any external libc for the solver.
This commit adds a fallback on libc from the current Python process, which works if it is dynamically linked.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
When looking at where we spend our time in solver setup, I noticed a fair bit of time is spent
in `Spec.format()`, and `Spec.format()` is a pretty old, slow, convoluted method.
This PR does a number of things:
- [x] Consolidate most of what was being done manually with a character loop and several
regexes into a single regex.
- [x] Precompile regexes where we keep them
- [x] Remove the `transform=` argument to `Spec.format()` which was only used in one
place in the code (modules) to uppercase env var names, but added a lot of complexity
- [x] Avoid escaping and colorizing specs unless necessary
- [x] Refactor a lot of the colorization logic to avoid unnecessary object construction
- [x] Add type hints and remove some spots in the code where we were using nonexistent
arguments to `format()`.
- [x] Add trivial cases to `__str__` in `VariantMap` and `VersionList` to avoid sorting
- [x] Avoid calling `isinstance()` in the main loop of `Spec.format()`
- [x] Don't bother constructing a `string` representation for the result of `_prev_version`
as it is only used for comparisons.
In my timings (on all the specs formatted in a solve of `hdf5`), this is over 2.67x faster than the
original `format()`, and it seems to reduce setup time by around a second (for `hdf5`).
The reverse provider lookup may have stale entries for deleted packages, which used to cause errors. It's hard to invalidate those cache entries, so this commit simply drops entries w/o invalidating the cache.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This commit differentiate linux from other platforms by
using libc compatibility as a criterion for deciding
which buildcaches / binaries can be reused. Other
platforms still use OS compatibility.
On linux a libc is injected by all compilers as an implicit
external, and the compatibility criterion is that a libc is
compatible with all other libcs with the same name and a
version that is lesser or equal.
Some concretization unit tests use libc when run on linux.
Some logic to detect what libc the c / cxx compilers use by default,
based on `-dynamic-linker`.
The function `compiler.default_libc()` returns a `Spec` of the form
`glibc@x.y` or `musl@x.y` with the `external_path` property set.
The idea is this can be injected as a dependency.
If we can't run the dynamic linker directly, fall back to `ldd` relative
to the prefix computed from `ld.so.`
In the future we may transform the database from a single JSON object to
a stream of JSON objects.
This paves the way for constant time writes and constant time rereads
when only O(1) changes are made. Currently both are linear time.
This commit gives just enough forward compat for Spack to produce a
friendly error when we would move to a stream of json objects, and a db
would look like this:
```json
{"database": {"version": "<something newer>"}}
```
* compiler wrapper: prioritize spack managed paths in search order
This commit partitions search paths of -L, -I (and -rpath) into three
groups, from highest priority to lowest:
1. Spack managed directories: these include absolute paths such as
stores and the stage dir, as well as all relative paths since they
are relative to a Spack owned dir
2. Non-system dirs: these are for externals that live in non-system
locations
3. System dirs: your typical `/usr/lib` etc.
It's very easy for Spack to known the prefixes it owns, it's much more
difficult to tell system dirs from non-system dirs. Before this commit
Spack tried to distinguish only system and non-system dirs, and failed
for very trivial cases like `/usr/lib/x/..` which comes up often, since
build systems sometimes copy search paths from `gcc -print-search-dirs`.
Potentially this implementation is even faster than the current state of
things, since a loop over paths is replaced with an eval'ed `case ...`.
* Trigger a pipeline
* Revert "Trigger a pipeline"
This reverts commit 5d7fa863de.
* remove redudant return statement
* Add conflicts for some blas implementations that don't build on
Windows (or with %msvc)
* Need to enclose CC/CXX variables in quotes in case those paths
have spaces, otherwise Meson runs into errors
* On Windows, Python dependencies now add <prefix>/Scripts to the
PATH (this is established as a standard in PEP 370)
* Later versions of oneAPI have moved, so update detection to find it
in both old and new location
* Remove reliance on ONEAPI_ROOT env variable when determining Fortran
compiler version for %msvc
* When finding a Fortran compiler for MSVC, there was logic enforcing
a maximum MSVC version for a given oneAPI Fortran version. This
mapping was out of date and excluding valid combinations, so has
been removed (the logic now just picks the latest available
oneAPI Fortran compiler for any given MSVC version).
On Windows, bootstrapping logic now searches for and adds the win-sdk
and wgl packages to the user's top scope as externals if they are not
present.
These packages are generally required to install most packages with
Spack on Windows, and are only available as externals, so it is
assumed that doing this automatically would be useful and avoid
a mandatory manual step for each new Spack instance.
Note this is the first case of bootstrapping logic modifying
configuration other than the bootstrap configuration.
* gxsview: compiles againts system qt and vtk on rhel8
* Update gxsview/package.py for blanks around operator
* Update gxsview/package.py import blank line
* Update gxsview/package.py for style
* Update gxsview/package.py checking vtk version
Score-P does not accept "--with-foo=yes", but only "--with-foo" or "--with-foo=some-valid-specific-choice-or-path". This keeps Spack from generating config flags that will cause Score-P to barf.
This adds some improvements to `spack find` output when in environments based
around some thoughts about what users want to know when they're in an env.
If you're working in an enviroment, you mostly care about:
* What are the roots
* Which ones are installed / not installed
* What's been added that still needs to be concretized
So, this PR adds a couple tweaks to display that information more clearly:
- [x] We now display install status next to every root. You can easily see
which are installed and which aren't.
- [x] When you run `spack find -l` in an env, the roots now show their concrete
hash (if they've been concretized). They previously would show `-------`
(b/c the root spec itself is abstract), but showing the concretized root's
hash is a lot more useful.
- [x] Newly added/unconcretized specs still show `-------`, which now makes more
sense, b/c they are not concretized.
- [x] There is a new option, `-r` / `--only-roots` to *only* show env roots if
you don't want to look at all the installed specs.
- [x] Roots in the installed spec list are now highlighted as bold. This is
actually an old feature from the first env implementation , but various
refactors had disabled it inadvertently.
Reduce incidence of spurious errors by:
* Ensuring we're passing the buffer by reference
* Get the correct short string size from Windows API instead of computing ourselves
* Ensure sufficient space for null terminator character
Add test for `windows_sfn`
Currently if you request pkg +example where example is a conditional
variant, and you have a pkg in the database for which the condition
did not hold (so no +example nor ~example), the solver would reuse it
regardless, not imposing +example.
The change rules out exactly one thing: variant_set without variant_value,
which in practice could only happen when not node_has_variant (i.e. when
under the current package.py rules the variant's when condition did not
trigger).
Add a "default" option that passes no option to configure. Existing
options changed in the MPICH 4.2.0 release, so update the package to
reflect those changes.
Currently, some of the tests in `spec_format` and `spec_semantics` fetch
the actual zlib repository when run, because they call `str()` on specs
like `zlib@foo/bar`, which at least currently requires a remote git clone
to resolve.
This doesn't change the behavior of git versions, but it uses our mock git
repo infrastructure and clones the `git-test` package instead of the *real*
URL from the mock `zlib` package.
This should speed up tests. We could probably refactor more so that the git
tests *all* use such a fixture, but the `checks` field that unfortunately
tightly couples the mock git repository and the `git_fetch` tests complicates
this. We could also consider *not* making `str()` resolve git versions, but
I did not dig into that here.
- [x] add a mock_git_test_package fixture that sets up a mock git repo *and*
monkeypatches the `git-test` package (like our git test packages do)
- [x] use fixture in `test_spec_format_path`
- [x] use fixture in `test_spec_format_path_posix`
- [x] use fixture in `test_spec_format_path_windows`
- [x] use fixture in `test_parse_single_spec`
Commit 330a9a7c9a aimed at preventing
generation of .cfg files when a given compiler does not exist
in the particular release. However the check does not
contain the full paths so it always fails resulting in empty
.cfg files. This commit fixes it.
* add new axom releases, sync changes between repos
* add new version of axom and add fallback depends for umpire/raja
* remove now redundant flags, add flags to cuda flags to flags output by cachedcmakepackage
Co-authored-by: white238 <white238@users.noreply.github.com>
* w3nco %oneapi cflags: append -Wno-error=implicit-function-declaration
* update flag for %apple-clang, %clang
* [@spackbot] updating style on behalf of eugeneswalker
Upon close inspection of clingo answer sets, in some cases we have "equivalent" (i.e. same hash for the concrete spec) duplicates that differ only because of virtual nodes that are added to the answer set, without any edge using them.
This commit adds a property `autopush` to mirrors. When true, every source build is immediately followed by a push to the build cache. This is useful in ephemeral environments such as CI / containers.
To enable autopush on existing build caches, use `spack mirror set --autopush <name>`. The same flag can be used in `spack mirror add`.
* Simplify config command and add BLAS/LAPACK location
* Use BLAS_ROOT and LAPACK_ROOT and disable use of system
package registry
* Adds location of BLAS_LIBRARIES and LAPACK_LIBRARIES to CMake
* Adds CMake variables to prevent picking up system installations
of BLAS/LAPACK. Fixes previous PR #43328 that was picking up
incorrect installations
* Adds versions 7.2.1
* libzip: add up to v1.10.1
- update homepage and change download url to GitHub
- change build system to CMake for releases starting with 1.4
* [@spackbot] updating style on behalf of aumuell
* libzip: fix urls
* [@spackbot] updating style on behalf of aumuell
* libzip: do not add versions from libzip.org
these are old, and urllib refuses to fetch them
* libzip: deprecate versions from libzip.org
urllib refuses to fetch them, only curl would work
---------
Co-authored-by: aumuell <aumuell@users.noreply.github.com>
* acts: new version 33.1.0
* actsvg: new version 0.4.41
* geomodel: new package
* [@spackbot] updating style on behalf of wdconinc
* acts: plugin_cmake_variant(geomodel)
* geomodel: with when(+visualization)
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
* Include recent change for Ubuntu
Select option -disable-no-pie-on-ubuntu for some Ubuntu systems
823971df01
* Added conflict for new variant
* Updated conflict version
* Added mention of Ubuntu to variant description
* Kokkos Kernels: adding missing TPLs and pre-conditions
Adding variants and dependencies for rocBLAS and rocSPARSE.
Also adding a "when=" close to the TPL variants that prevents
enabling the TPLs in versions of the library when it was not
yet available.
* Kokkos Kernels: remove comment for better format
* Kokkos Kernels: adding cusolver and rocsolver for at version 4.3.00
* Kokkos Ecosystem: updating packages for release 4.3.00
* Kokkos: adding arch for SG2042
* Removing sg2042 from spack_micro_arch_map
Removing it here and will work to add it in the proper generic spack location, likely: `spack/lib/spack/external/archspec/json/cpu/microarchitectures.json` ?
Allow reuse of specs that were built with compilers not in the current configuration. This means that specs from build caches don't need to have a matching compiler locally to be reused. Similarly when updating a distro. If a node needs to be built, only available compilers will be considered as candidates.
* add c++11 header to gold for compiler not defaulting to c++11
* glibc: add 2.39
* llvm: add 18.1.3
* fix#42314, remove cxx11 flag for llvm; should be controlled by cmake.
* modify patch
* llvm version
* add gmake version request
* yoda: new versions 1.9.9, 1.9.10, and 2.0.0
* [@spackbot] updating style on behalf of wdconinc
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
As in the original makefile "FFLAGS_PROMOTION = -fdefault-real-8
-fdefault-double-8" is only used when `precision=double`. This is the default
for the Spack package, so no change if `precision` is left unset.
* Generally use os.replace on Windows and Linux
* Windows behavior for os.replace differs when the destination exists
and is a symlink to a directory: on Linux the dst is replaced and
on Windows this fails - this PR makes Windows behave like Linux
(by deleting the dst before doing the rename unless src and dst
are the same)
* Nalu: updating Trilinos recipe a bit
Basic changes to build/install nalu properly using Spack.
Some more changes would be nice for instance adding an
option to build against Trilinos master or develop. Adding
a dependency on googletest to avoid the annoying build
failures in the unit-tests.
* Nalu: adding release 1.6.0
Nalu v1.6.0 can build cleanly against Trilinos 14.0.0 with the
proposed changes. The only other combo is master / master but
than one is "floating" as these branch evolve over time. When a
new Nalu comes out we might want to add another fixed version to
keep this recipes up to date!
the autotools build system does something funky which causes a link line
where gccs default link dirs are explicitly added and end up before the
-L from spack's libunwind, so that ultimately it links against system
libunwind.
the cmake build system does better.
* upgrade new versions
* style fix
* update jaxlib deps (not cuda and bazel yet)
* update jaxlib cuda versions
* update jaxlib cuda versions
* update jaxlib cuda versions
* chore: style fix
* Update package.py
* Update package.py
* fix: typo
* docs: add source for cuda version
* py-jaxlib 0.4.14 also doesn't build on ppc64le
* Add 0.4.26
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ParaView: Update version 5.12.0
Add 5.12.0 release
Update default to 5.12.0
* Add patch for building ParaView 5.12 with kits
* Drop VTKm from neoverse
* For avx build, the start address of values_ buffer in KernelParameters
is not a correct as it is computed based on 16-byte alignment.
* Style check error fix
* updating package.py for py-celery, py-kombu, py-amq
* added more py-kombu package versions
* fix copyrights and stype on py-kombu/package.py
* removed extra spaces
* added py-billiard 4.2.0 and added back the license('BSD-3-Clause')
* removed extra spaces in py-celery/package.py
* fixed py-amqp 2.4.0 sha; fixed py-celery's dependency of py-click (when version restrictions)
* more clean up on specifying version bounds
* Relax compiler and target mismatches
The mismatch occurs on an edge. Previously it was assigned
the parent priority, now it is assigned the child priority.
This should make reuse from buildcaches or store more likely,
since most mismatches will be counted with "reused" priority.
* Optimize version badness for runtimes at very low priority
We don't want to e.g. switch other attributes because we
cannot reuse an old installed runtime.
* Optimize runtime attributes at very low priority
This is such that the version of the runtime would
not influence whether we should reuse a spec.
Compiler mismatches are considered for runtimes,
to avoid situations where compiling foo%gcc@9
brings in gcc-runtime%gcc@13 if gcc@13 is among
the available compilers
* Exclude specs without runtimes from reuse
This should ensure that we do not reuse specs that
could be broken, as they expect the compiler to be
installed in a specific place.
The installer runs `get_dependent_ids`, which follows edges outside the
subdag that's being installed, so it returns a superset of the actual
dependents.
That's generally fine, except that it calls `s.package` on every
dependent, which triggers a package class to be instantiated, which is a
lot of work.
Instead, compute the package id from the spec, since that's all that's
used anyways and does not trigger *lots* of slow and redundant
instantiations of package objects.
If ONEAPI_ROOT is not set as an environment variable, the current approach will raise an error.
Instead we can compute the OneAPI_ROOT from the compiler paths like we do with vcvarsall.
The installation mechanism used on Linux to install py-pip (using pip
from the downloaded wheel to install the wheel) does not work on Windows.
This updates the installation of py-pip on Windows to download and
use a zipapp of a specific pip version in order to install the wheel
pip version that is requested.
`dpcpp` is deprecated by intel and has been superseded by `oneapi` compilers for a very long time.
---------
Co-authored-by: becker33 <becker33@users.noreply.github.com>
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* SEACAS, EXODUSII: New version; deprecate older versions; better variant descriptions
* [@spackbot] updating style on behalf of gsjaardema
* Fix long lines reported by flake8
---------
Co-authored-by: gsjaardema <gsjaardema@users.noreply.github.com>
* Add release 2.8.0
* Changing C compiler to hipcc
* Final release version
* Adding new cmake definition for rocm support
* Enabling rocm version support
* Update sha256
* Updating website URL
* Removing unnecessary C compiler spec
* Adding rocm-core dependency
* fixing rocm-core version
* fixing rocm-core version
* fixing style
* bugfix
* Add systemd
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* gobject-introspection: Correct glib versions
- The meson.build requirement that the glib version
is >= the gobject-introspective version is not in place
until v1.76.1.
- Prior to that, the requirement was glib >= 2.58.0.
- Bug introduced in acbf0d99c4, PR #42222.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* util-linux: add v2.39.3
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* py-natsort: add new versions
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: default systemd support to true
- Make the dependency sticky to force a failure
if systemd compilation fails, or force
the user to disable the option.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: Add initial multi-architecture support
- Restrict arch conflicts to 3.0.1
- Disable cpuid at configure time on non-x86_64 platforms.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: update docstrings
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* Add py-geopmdpy
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* Add geopm-runtime recipe
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
---------
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
This PR allows the user to specify a path to a custom cert file (or directory) in
Spack's config:
```yaml
# This is where custom certs for proxy/firewall are stored.
# It can be a path or environment variable. To match ssl env configuration
# the default is the environment variable SSL_CERT_FILE
ssl_certs: $SSL_CERT_FILE
```
`config:ssl_certs` can be a path to a file or a directory, or it can be and environment
variable that resolves to one of those. When it posts to something valid, Spack will
update the ssl context to include custom certs, and fetching via `urllib` and `curl`
will trust the provided certs.
This should resolve many issues with fetching behind corporate firewalls.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Alec Scott <alec@bcs.sh>
After #41373, where we stopped considering the source directory to be the stage for develop builds,
we resumed *deleting* the stage even after a successful build.
We don't want this for develop builds because developers need to iterate; we should keep the artifacts
unless they explicitly run `spack clean`.
Now:
- [x] Build artifacts for develop packages are not removed after a successful install
- [x] They are also not removed before an install starts, i.e. develop packages always
reuse prior artifacts, if available.
- [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
Prior FleCSI releases relied on commits on the control-replication branch (cr)
of Legion. That branch has now been merged into master and is part of the
24.03.0 Legion release.
* (py-)onnx: new version 1.14.{0,1}, 1.15.0
Notes on `onnx`:
- The C++ standard was changed to 14 in 1.15, so no more filter_file is needed. (The C++ standard has since changed to 17 in master.)
Notes on `py-onnx`:
- `py-pybind11` was an unlisted requirement in CMakeLists.txt since 1.3 or so (before earliest spack package).
* py-onnx: depends_on pybind11 with type link, not run
* py-onnx: depends_on py-setuptools@64:
Users requested an option to filter between local/upstream results in `spack find` output.
```
# default behavior, same as without --install-tree argument
$ spack find --install-tree all
# show only local results
$ spack find --install-tree local
# show results from all upstreams
$ spack find --install-tree upstream
# show results from a particular upstream or the local install_tree
$ spack find --install-tree /path/to/install/tree/root
```
---------
Co-authored-by: becker33 <becker33@users.noreply.github.com>
* sundials: add new version
* note previous default
* update when clause for removed options
---------
Co-authored-by: David J. Gardner <gardner48@llnl.gov>
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* exodusii -- refactor and bring up-to-date
* Add missed patch file
* [@spackbot] updating style on behalf of gsjaardema
* Apply seacas windows patch here also.
* Update url so old checksums valid; redo new checksums
---------
Co-authored-by: gsjaardema <gsjaardema@users.noreply.github.com>
* Initial commit to bump rocprofiler-dev to 6.0 and add aqlprofile recipe
* bump to 6.0.2 and extracting binaries from deb pkg
* fixes for hpctoolkit build errors
* add yum and zyp aqlprofile packages
* fix style issues
* Allow compilers to function across compatible OS's
* Add documentation in the default yaml
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
* py-libsonata: new package starting at 0.1.25
* align spack version with the git branch
* Fix min required versions / use spack packages instead of submodules
* Create devcontainer.json
* Ensure codespace can be setup for current branch
* fix: find compilers in site scope
* fix: use cloud_pipelines ubuntu20.04 image
* fix: spack config --scope site add
* fix: use develop, not develop-root mirror
This adds a dependency on pkg-config which in turn builds pkg-config
on pipelines using %onapi/%cce: update the pkg-config build to disable
specific warnings-as-errors from these compilers.
Co-authored-by: Reid Priedhorsky <1682574+reidpr@users.noreply.github.com>
* Add macos-14 as a runner (Apple M1)
* Mark a test xfail
We need to check later if this test needs modifications
on Apple Silicon chips.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Co-authored-by: alalazo <alalazo@users.noreply.github.com>
* buildcache sync: manifest-glob with arbitrary destination
The current implementation of the --manifest-glob is a bit restrictive
requiring the destination to be known by the generation stage of CI.
This allows specifying an arbitrary destination mirror URL.
* Add unit test for buildcache sync with manifest
* Fix test and arguments for manifest-glob with override destination
* Add testing path for unused mirror argument
* Remove a few compilers from static test data
These compilers were used only in a bunch of tests, so
they are added only there.
* Remove clang@3.3 from unit test configuration
* Parametrize compilers.yaml
* Remove specially named gcc from static data
The compilers are used in two tests
* Remove apple-clang and macOS compilers from static data
The compiler was used only in multimethod tests
* Remove clang@3.5 (compiler seems to be unused)
* Remove gcc@4.4.0 (compiler seems to be unused)
* Exclude x86_64 tests on other architectures
* Mark two tests as for clingo only
* Update version syntax in compilers.yaml
* Parametrize tcl tests on architectures
* Parametrize lmod tests on architectures
* Substitute gcc@4.5.0 with gcc@4.8.0 so it can be used on aarch64
* Fix a few issues with aarch64 and unit-tests
It's now possible to add config on the command line with `spack -c <CONFIG_VARS> ...`, but the new `command_line` scope isn't reflected in the help output for `--scope`:
```bash
> spack help config
...
--scope {defaults,system,site,user}[/PLATFORM] or env:ENVIRONMENT
configuration scope to read/modify
...
```
This PR adds:
- A new runtime for `%oneapi` compilers, called `intel-oneapi-runtime`
- Information to both `gcc-runtime` and `intel-oneapi-runtime`, to ensure
that we don't mix compilers using different soname for either `libgfortran`
or `libifcore`
To do so, the following internal mechanisms have been implemented:
- Possibility to inject virtual dependencies from the `runtime_constraints`
callback on packages
Information has been added to `gcc-runtime` to provide the correct soname
under different conditions on its `%gcc`.
Rules injected into the solver looks like:
```prolog
% Add a dependency on 'gfortran@5' for nodes compiled with gcc@=13.2.0 and using the 'fortran' language
attr("dependency_holds", node(ID, Package), "gfortran", "link") :-
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
attr("virtual_node", node(RuntimeID, "gfortran")) :-
attr("depends_on", node(ID, Package), ProviderNode, "link"),
provider(ProviderNode, node(RuntimeID, "gfortran")),
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
attr("node_version_satisfies", node(RuntimeID, "gfortran"), "5") :-
attr("depends_on", node(ID, Package), ProviderNode, "link"),
provider(ProviderNode, node(RuntimeID, "gfortran")),
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
```
The default url couldn't be the one with v0.0.0-aws since spack was
replacing v0.0.0-aws with v<version_number> for example, deleting the
-aws suffix. I used the url_for_version method to specify this suffix.
This adds support for prereleases. Alpha, beta and release candidate
suffixes are ordered in the intuitive way:
```
1.2.0-alpha < 1.2.0-alpha.1 < 1.2.0-beta.2 < 1.2.0-rc.3 < 1.2.0 < 1.2.0-xyz
```
Alpha, beta and rc prereleases are defined as follows: split the version
string into components like before (on delimiters and string boundaries).
If there's a string component `alpha`, `beta` or `rc` followed by an optional
numeric component at the end, then the version is prerelease.
So `1.2.0-alpha.1 == 1.2.0alpha1 == 1.2.0.alpha1` are all the same, as usual.
The strings `alpha`, `beta` and `rc` are chosen because they match semver,
they are sufficiently long to be unambiguous, and and all contain at least
one non-hex character so distinguish them from shasum/digest type suffixes.
The comparison key is now stored as `(release_tuple, prerelease_tuple)`, so in
the above example:
```
((1,2,0),(ALPHA,)) < ((1,2,0),(ALPHA,1)) < ((1,2,0),(BETA,2)) < ((1,2,0),(RC,3)) < ((1,2,0),(FINAL,)) < ((1,2,0,"xyz"), (FINAL,))
```
The version ranges `@1.2.0:` and `@:1.1` do *not* include prereleases of
`1.2.0`.
So for packaging, if the `1.2.0alpha` and `1.2.0` versions have the same constraints on
dependencies, it's best to write
```python
depends_on("x@1:", when="@1.2.0alpha:")
```
However, `@1.2:` does include `1.2.0alpha`. This is because Spack considers
`1.2 < 1.2.0` as distinct versions, with `1.2 < 1.2.0alpha < 1.2.0` as a consequence.
Alternatively, the above `depends_on` statement can thus be written
```python
depends_on("x@1:", when="@1.2:")
```
which can be useful too. A short-hand to include prereleases, but you
can still be explicit to exclude the prerelease by specifying the patch version
number.
### Concretization
Concretization uses a different version order than `<`. Prereleases are ordered
between final releases and develop versions. That way, users should not
have to set `preferred=True` on every final release if they add just one
prerelease to a package. The concretizer is unlikely to pick a prerelease when
final releases are possible.
### Limitations
1. You can't express a range that includes all alpha release but excludes all beta
releases. Only alternative is good old repeated nines: `@:1.2.0alpha99`.
2. The Python ecosystem defaults to `a`, `b`, `rc` strings, so translation of Python versions to
Spack versions requires expansion to `alpha`, `beta`, `rc`. It's mildly annoying, because
this means we may need to compute URLs differently (not done in this commit).
### Hash
Care is taken not to break hashes of versions that do not have a prerelease
suffix.
Generate CI scripts as powershell on Windows. This is intended to
output exactly the same bash scripts as before on Linux.
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
As the cmake build is triggered by scikit build, the usual spack option
for enabling tests had no effect and the heavy test suite ran all the time.
Used https://github.com/scikit-build/cmake-python-distributions/issues/172#issuecomment-890322263
to implement how to pass options to the actual `cmake` build.
I also excluded some tests that failed for me on alma9 (gcc 11.4.1),
so the rest of the test suite can be run.
Enable OpenBLAS's built-in CPU capability detection and kernel selection.
This allows run-time selection of the "best" kernels for the running CPU, rather
than what is specified at build time. For example, it allows OpenBLAS to use
AVX512 kernels when running on ZEN4, and built targeting the "ZEN" architecture.
Co-authored-by: Branden Moore <branden.moore@amd.com>
* Changes to re-enable aws-pcluster pipelines
- Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
- Install gcc from hash so there is no risk of building gcc from source in pipleine.
- `packages.yam` files are now part of the pipelines.
- No more eternal `postinstall.sh`. The necessary steps are in `setup=pcluster.sh` and will be version controlled within this repo.
- Re-enable pipelines.
* Add and
* Debugging output & mv skylake -> skylake_avx512
* Explicilty check for packages
* Handle case with no intel compiler
* compatibility when using setup-pcluster.sh on a pre-installed cluster.
* Disable palace as parser cannot read require clause at the moment
* ifort cannot build superlu in buildcache
`ifort` is unable to handle such long file names as used when cmake compiles
test programs inside build cache.
* Fix spack commit for intel compiler installation
* Need to fetch other commits before using them
* fix style
* Add TODO
* Update packages.yaml to not use 'compiler:', 'target:' or 'provider:'
Synchronize with changes in https://github.com/spack/spack-configs/blob/main/AWS/parallelcluster/
* Use Intel compiler from later version (orig commit no longer found)
* Use envsubst to deal with quoted newlines
This is cleaner than the `eval` command used.
* Need to fetch tags for checkout on version number
* Intel compiler needs to be from version that has compatible DB
* Install intel compiler with commit that has DB ver 7
* Decouple the intel compiler installation from current commit
- Use a completely different spack installation such that this current pipeline
commit remains untouched.
- Make the script suceed even if the compiler installation fails (e.g. because
the Database version has been updated)
- Make the install targets fall back to gcc in case the compiler did not install
correctly.
* Use generic target for x86_64_vX
There is no way to provision a skylake/icelake/zen runner. They are all in the
same pools under x86_64_v3 and x86_64_v4.
* Find the intel compiler in the current spack installation
* Remove SPACK_TARGET_ARCH
* Fix virtual package index & use package.yaml for intel compiler
* Use only one stack & pipeline per generic architecture
* Fix yaml format
* Cleanup typos
* Include fix for ifx.cfg to get the right gcc toolchain when linking
* [removeme] Adding timeout to debug hang in make (palace)
* Revert "[removeme] Adding timeout to debug hang in make (palace)"
This reverts commit fee8a01580489a4ea364368459e9353b46d0d7e2.
* palace x86_64_v4 gets stuck when compiling try newer oneapi
* Update comment
* Use the latest container image
* Update gcc_hashes to match new container
* Use only one tag providing tags per extends call
Also removed an unnecessary tag.
* Move generic setup script out of individual stack
* Cleanup from last commit
* Enable checking signature for packages available on the container
* Remove commented packages / Add comment for palace
* Enable openmpi@5 which needs pmix>3
* don't look for intel compiler on aarch64
* py-python-lsp-server: add v1.10.0
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Remove py-wheel from package
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Running a `spack-python` script like this:
```python
import spack
import multiprocessing
def echo(args):
print(args)
if __name__ == "__main__":
pool = multiprocessing.Pool(2)
pool.map(echo, range(10))
```
will fail in `develop` with an error like this:
```console
_pickle.PicklingError: Can't pickle <function echo at 0x104865820>: attribute lookup echo on __main__ failed
```
Python expects to be able to look up the method `echo` in `sys.path["__main__"]` in
subprocesses spawned by `multiprocessing`, but because we use `InteractiveConsole` to
run `spack python`, the executed file isn't considered to be the `__main__` module, and
lookups in subprocesses fail. We tried to fake this by setting `__name__` to `__main__`
in the `spack python` command, but that doesn't fix the fact that no `__main__` module
exists.
Another annoyance with `InteractiveConsole` is that `__file__` is not defined in the
main script scope, so you can't use it in your scripts.
We can use the [runpy.run_path()](https://docs.python.org/3/library/runpy.html#runpy.run_path) function,
which has been around since Python 3.2, to fix this.
- [x] Use `runpy` module to launch non-interactive `spack python` invocations
- [x] Only use `InteractiveConsole` for interactive `spack python`
Often in containers, the files we use to detect whether a cray system supports new features are not available.
Given that the cray containers only support the newer versions, and that these versions have been
around for a while at this point and few sites don't support them, this PR changes the logic for
detecting cray systems so that:
1. Don't even consider whether something is the `cray` platform if `opt/cray` is not in `MODULEPATH`
2. Only use the `cray` platform if we can read files in /opt/cray/pe and positively detect an older version
3. Otherwise, assume we're *not* on a cray (includes newer Cray PE's, which we treat as Linux)
Compilation with the old flags fails on PowerPC (power8le) due to syntax
errors in the output from the preprocessor. Compilation with the
extended set of flags works both on PowerPC and x86_64.
The correct set of flags was suggested from the berkeleygw developers:
https://groups.google.com/a/berkeleygw.org/g/help/c/ewi3RZgOyeE/m/jSIoe45PAgAJ
Uses spack-provided compiler prefix to call cpp when compiling berkeleygw with gcc.
Adds variants to turn off tests
Add variants for some missing TPL options
Add the variables required to build in ~shared
* Add pamgen to Trilinos as a variant to support SEACAS
This adds the ability to turn off and on pamgen as needed
through the variant interface for the Trilinos package.py.
Add changes for seacas package.py to build the appropriate
Trilinos variants.
Add zlib-api as depends_on instead of zlib directly for SEACAS
package.py
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Add variant typescript for py-jupyter-server@:1, which then requires npm/node. Patch the build system for ~typescript so that it doesn't find any npm/node installations and attempts to build the typescript extension even though it shouldn't
* Fix formatting in var/spack/repos/builtin/packages/py-jupyter-server/package.py
* Constrain typescript variant to py-jupyter-server versions 1.10.2:1
* with when not needed if variant doesn't exist for other versions
This should fix issue #43192
Basically, had issue where a make variable was set to the output
of a shell function which included cd commands, and then the value
of that variable used as a makefile target.
The cd commands in the shell function caused assorted informational
messages (e.g. "Entering directory ...") which got included in the
return of the shell function, corrupting the value of the variable.
The presence of colons in the corrupted value caused make to issue
"multiple target" erros.
This fix adds --no-print-directory flags to the calls to the
make function in the package's build method, which resolves the
issue above.
It is admittedly a crude fix, and will remove *all* informational
messages re directory changes, thereby potentially making it more
difficult to diagnose/debug future issues building this package.
However, I do not see a way for to turn off these messages in a
more surgical manner.
* mgard: don't restrict protobuf version more than necessary
successfully built:
mgard@2022-11-18 ^protobuf@3.{4,21,25}
mgard@2023-01-10 ^protobuf@3.{4,25}
mgard@2023-03-31 ^protobuf@3.{4,25}
compile failures:
mgard@2022-11-18 ^protobuf@3.3
mgard@2023-01-10 ^protobuf@3.3
mgard@2023-03-31 ^protobuf@3.3
* mgard: add conflicts to address CI errors
* mgard: conflict between cuda and abseil@20240116.1
compiling mgard+cuda with gcc@12.3.0 and nvcc from cuda@12.3.0 against
protobuf pulling in abseil-cpp@20240116.1 results in the errors reported
here: https://github.com/abseil/abseil-cpp/issues/1629
* py-shacl: new version, update dependencies
Also updates the dependencies py-prettytable and py-rdflib.
* review comments
* Update var/spack/repos/builtin/packages/py-pyshacl/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-poetry-core: add required 1.8.1
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
`jinja2` can be a costly import, and right now it happens at startup every time we run
Spack. This slows down `spack --print-shell-vars` a bit, which is needed by `setup-env.*sh`.
Patch allowing Clingo to build with VS22 has landed both in Spack
and Clingo upstream, update Spack's bootstrap constraints to handle
this.
Additionally, properly scope the patch application in the clingo
package to handle upstream patch.
Currently (outside of this PR) when you `spack develop` a path, this path is treated as the staging
directory (this means that for example all build artifacts are placed in the develop path).
This PR creates a separate staging directory for all `spack develop`ed builds. It looks like
```
# the stage root
/the-stage-root-for-all-spack-builds/
spack-stage-<hash>
# Spack packages inheriting CMakePackage put their build artifacts here
spack-build-<hash>/
```
Unlike non-develop builds, there is no `spack-src` directory, `source_path` is the provided `dev_path`.
Instead, separately, in the `dev_path`, we have:
```
/dev/path/for/foo/
build-{arch}-<hash> -> /the-stage-root-for-all-spack-builds/spack-stage-<hash>/
```
The main benefit of this is that build artifacts for out-of-source builds that are relative to
`Stage.path` are easily identified (and you can delete them with `spack clean`).
Other behavior added here:
- [x] A symlink is made from the `dev_path` to the stage directory. This symlink name incorporates
spec details, so that multiple Spack environments that develop the same path will not conflict
with one another
- [x] `spack cd` and `spack location` have added a `-c` shorthand for `--source-dir`
Spack builds can still change the develop path (in particular to keep track of applied patches),
and for in-source builds, this doesn't change much (although logs would not be written into
the develop path). Packages inheriting from `CMakePackage` should get this benefit
automatically though.
Add current versions of the 17 and 18 releases
Stop making it nearly impossible to compose this correctly with code built with gcc
Build for compatibility by default like we do in every other llvm package
The `patch()` directive can now be invoked with `reverse=True` to apply a patch in reverse.
This is useful for reverting commits that caused errors in projects, even if only the forward
patch is available, e.g. via a GitHub commit patch URL.
`patch(..., reverse=True)` runs `patch -R` behind the scenes. This is a POSIX option so we
can expect it to be available on the `patch` command.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
set the FC variable to the MPI Fortran compiler and also set the F90 variable
to the same compiler for versions 9.8 and up. FC needs to be set because the
configure script still uses FC.
* unmaintained packages: add new versions
* Fix parallel and numactl
* Revert numactl changes
* rollback lua-sol2 version
* Update alluxio version format
* py-psyclone and py-fparser new releases added
* py-fparser: add missing releases for py-psyclone
* py-psyclone: actioned @adamjstewart comments
* py-psyclone: removed py-pytest-pylint
* py-pytest-pylint: added package @ latest version
* py-pytest-pylint: reformatted
* Update var/spack/repos/builtin/packages/py-psyclone/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pytest-pylint: added build deps and runtime dep versions
* py-pytest-pylint: removed version from setuptools
* py-psyclone: add py-pytest-pylint test dep and alphabetize deps
* Update var/spack/repos/builtin/packages/py-pytest-pylint/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-psyclone: deps ordered
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
fixes#43097
Before this PR the behavior of mixins used together with
builders was to mask completely the callbacks defined from
the class coming later in the MRO.
Here we fix the behavior by accumulating all callbacks,
and de-duplicating them later.
* package/qgis: add new version
* improve Qsci.pro
* improve
* fix undefined symbol qsciprinter error
* add import test
* fix bug
* add version 3.36
* [@spackbot] updating style on behalf of Sinan81
* fix long line
* only run import test when +python
* first attempt at stand-alone test
* add TODO
---------
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
Co-authored-by: Sinan81 <Sinan81@users.noreply.github.com>
Co-authored-by: Sinan81 <Sinan@world>
* proj: correct CMake arg for shared build with proj older than 7.0.0
* Actually use new CMake arg
* Update var/spack/repos/builtin/packages/proj/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
This adds Spack packages for these Perl distributons:
- Bio::DB::EUtilities and its dependencies:
- Bio::ASN1::EntrezGene
- Bio::Cluster
- Bio::Variation
* Properly specify Curl builder interface for static vs shared curl
with NMake system to ensure all built curls export expected
symbols.
* Symlinks curl library build artifact to more idiomatic name for
FindCurl.cmake implementations and other NMake consumers.
* Deprecating py-pylint@2.3 as it cannot build with python@3.8:
* Style fix
* Removed versions because can't build with python@3.7
---------
Co-authored-by: Gava, Francesco <francesco.gava@mclaren.com>
* py_cheap_repr: add initial package.py
* Update var/spack/repos/builtin/packages/py-cheap-repr/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-cheap-repr: use pypi link instead
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Added new versions @1.5.1 and @1.4.1 (sfcgal moved from github to gitlab)
Placed restrictions on what versions of cgal are supported for different
cfcgal versions. These restrictions are based on what was found in the
version history at https://gitlab.com/sfcgal/SFCGAL/-/blob/v1.5.1/NEWS?ref_type=tags
as well as the CMakeLists.txt for different versions.
@1.4 and @1.5 seem to require a specific version of cgal based on CMakeLists.txt
Earlier versions (@1.3.8:1.3.10) claim to support cgal@4.3: (but Spack recipe
claims did not work until @4.7, so sticking with that as minimum). CMakeList.txt
suggests they support cgal@5 as well, but version history suggests otherwise.
* Expanding the duckdb package to fix the version number (required for external extensions to work) being pulled from git and have variants for the built-in extensions at build-time. This also changes the build system from CMakePackage to Makefile package (as advised from upstream).
* - Reorganized and cleaned up variants
- Updated the patch to work for 0.10.0+
- Removed (or made non-default) some unecessary variants
- Added ninja as the default generator
- Set up some shared library dependencies;
* enable tensorflow-2.11 support for ROCm
* add latest sha for mesa and limit the patches to older version.similar
changes in #37910 to enable gitlab-ci pass
* address review commemts
Remove dependency on `importlib_metadata` and `pkg_resources`, which can be problematic if the version in PYTHONPATH is incompatible with the interpreter Spack is running under.
* Kokkos Kernels: adding missing TPLs and pre-conditions
Adding variants and dependencies for rocBLAS and rocSPARSE.
Also adding a "when=" close to the TPL variants that prevents
enabling the TPLs in versions of the library when it was not
yet available.
* Kokkos Kernels: remove comment for better format
* Kokkos Kernels: fix issue with unpacking wrong number of args
After changing the tpls dictionary we need to update the unpacking
logic to catch the right number of outputs out of it!
* Kokkos Kernels: updating doc string for tpls var and using f-string
Improving comment a bit and switching to f-string for more readability.
When setup to do more testing will try to use f-string in the CMake
options generation part of the package.
* Style change
* sst-core now effectively depends on ncurses
* use --with-curses
* sst-core: update comment about ncurses
* should have curses for build, link, and run
Closes#43052.
Maybe moving the argument to the `find` subcommand is a good idea, but I
just wanted to get the docs fix out.
Co-authored-by: Patrice Peterson <patrice.peterson@itz.uni-halle.de>
This PR adds the ability to load spack extensions through `importlib.metadata` entry
points, in addition to the regular configuration variable.
It requires Python 3.8 or greater to be properly supported.
#42878 adds a post install filter of the netCDFConfig.cmake file
that replaces a valid CMake target on Windows with an invalid one.
Don't do this replacement on Windows.
* Updates for migraphx 6.0.0 & 6.0.2
* Style check error and audit check error fix
* Adding patch for half-include-directory
* The parameter GPU_TARGETS is used from 5.7 in migraphx
* Adding rocmlir dependency in migraphx and 6.0 updates in rocmlir
* Applying upcoming changes to make CK JIT optional and enable
compilation on Windows in order to build without ck dependency
* mpfr: missing dependency for version 4.0.1
mpfr 4.0.1 (like 4.0.2) needs autoconf-archive where it takes the
AX_PHREAD macro from
* autoconf-archive is also required for mpfr@4.0.0
New changes have been made to nvptx-tools that address dropping support
for sm_30 in later CUDA versions (12.0+).
Also refactor gcc to make nvptx-tools a dependency instead of a resource.
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Adds CHI::Driver::Memcached.
Modified perl-chi to make the tests work. Testing modules in perl-chi
were not loaded when testing CHI::Driver::Memcached, so added the "run"
type to these.
* ASP-based solver: improve reusing nodes with gcc-runtime
This PR skips emitting dependency constraints on "gcc-runtime",
for concrete specs that are considered for reuse.
Instead, an appropriate version of gcc-runtime is recomputed
considering also the concrete nodes from reused specs.
This ensures that root nodes in a DAG have always a runtime
that is at a version greater or equal than their dependencies.
* Add unit-test for view with multiple runtimes
* Select latest version of runtimes in views
* Construct result keeping track of latest
* Keep ordering stable, just in case
1) support for version @3.0
Unfortunately, download seems to require registration now
so using manual_download mechanism for @3:
2) copying from hdf-eos5 patch from @vanderwb to enable
use of Spack compiler wrappers instead of h4cc
3) Patching an issue in hdf-eos2 configure script. The
script will test for jpeg, libz libraries, succeed and
append HAVE_LIBJPEG=1, etc to confdefs.h, and then abort
because HAVE_LIBJPEG not set in running environment.
4) Add some LDFLAGS to build environment. Otherwise
seems to fail to build test script due to rpc dependence
in HDF4.
Adds Net::Server::SS::PreFork and its dependencies.
Installed OK with build-time tests. Added dependencies:
- Server::Starter
- Net::Server
- HTTP::Server::Simple
Adds Catalyst::Action::RenderView and its dependencies.
Installed OK with build-time tests. Added dependencies:
- Catalyst::Action::RenderView
- Catalyst::Action::RenderView
- Catalyst::Action::RenderView
* package/imagemagick add new version, improve
* confimed that build fails when libsm is missing on linux
---------
Co-authored-by: Sinan81 <Sinan@world>
* perl-chi and deps: new packages
Adds CHI and its dependencies.
Installed OK with build-time tests. Added dependencies:
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
- CHI
* Add license
@@ -220,6 +220,40 @@ section of the configuration:
.._binary_caches_oci:
---------------------------------
Automatic push to a build cache
---------------------------------
Sometimes it is convenient to push packages to a build cache as soon as they are installed. Spack can do this by setting autopush flag when adding a mirror:
..code-block::console
$ spack mirror add --autopush <name> <url or path>
Or the autopush flag can be set for an existing mirror:
..code-block::console
$ spack mirror set --autopush <name> # enable automatic push for an existing mirror
$ spack mirror set --no-autopush <name> # disable automatic push for an existing mirror
Then after installing a package it is automatically pushed to all mirrors with ``autopush: true``. The command
..code-block::console
$ spack install <package>
will have the same effect as
..code-block::console
$ spack install <package>
$ spack buildcache push <cache> <package> # for all caches with autopush: true
..note::
Packages are automatically pushed to a build cache only if they are built from source.
@@ -73,9 +73,12 @@ are six configuration scopes. From lowest to highest:
Spack instance per project) or for site-wide settings on a multi-user
machine (e.g., for a common Spack instance).
#.**plugin**: Read from a Python project's entry points. Settings here affect
all instances of Spack running with the same Python installation. This scope takes higher precedence than site, system, and default scopes.
#.**user**: Stored in the home directory: ``~/.spack/``. These settings
affect all instances of Spack and take higher precedence than site,
system, or defaults scopes.
system, plugin, or defaults scopes.
#.**custom**: Stored in a custom directory specified by ``--config-scope``.
If multiple scopes are listed on the command line, they are ordered
@@ -196,6 +199,45 @@ with MPICH. You can create different configuration scopes for use with
mpi: [mpich]
.._plugin-scopes:
^^^^^^^^^^^^^
Plugin scopes
^^^^^^^^^^^^^
..note::
Python version >= 3.8 is required to enable plugin configuration.
Spack can be made aware of configuration scopes that are installed as part of a python package. To do so, register a function that returns the scope's path to the ``"spack.config"`` entry point. Consider the Python package ``my_package`` that includes Spack configurations:
..code-block::console
my-package/
├── src
│ ├── my_package
│ │ ├── __init__.py
│ │ └── spack/
│ │ │ └── config.yaml
└── pyproject.toml
adding the following to ``my_package``'s ``pyproject.toml`` will make ``my_package``'s ``spack/`` configurations visible to Spack when ``my_package`` is installed:
..code-block::toml
[project.entry_points."spack.config"]
my_package="my_package:get_config_path"
The function ``my_package.get_extension_path`` in ``my_package/__init__.py`` might look like
@@ -111,3 +111,39 @@ The corresponding unit tests can be run giving the appropriate options to ``spac
(5 durations < 0.005s hidden. Use -vv to show these durations.)
=========================================== 5 passed in 5.06s ============================================
---------------------------------------
Registering Extensions via Entry Points
---------------------------------------
..note::
Python version >= 3.8 is required to register extensions via entry points.
Spack can be made aware of extensions that are installed as part of a python package. To do so, register a function that returns the extension path, or paths, to the ``"spack.extensions"`` entry point. Consider the Python package ``my_package`` that includes a Spack extension:
..code-block::console
my-package/
├── src
│ ├── my_package
│ │ └── __init__.py
│ └── spack-scripting/ # the spack extensions
└── pyproject.toml
adding the following to ``my_package``'s ``pyproject.toml`` will make the ``spack-scripting`` extension visible to Spack when ``my_package`` is installed:
..code-block::toml
[project.entry_points."spack.extenions"]
my_package="my_package:get_extension_path"
The function ``my_package.get_extension_path`` in ``my_package/__init__.py`` might look like
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
