* Update seacas package.py
Adding libcatalyst variant to seacas package
When seacas is installed with "seacas +libcatalyst"
then a dependency on the spack package "libcatalyst"
(which is catalyst api 2 from kitware) is added, and
the appropriage cmake variable for the catalyst TPL
is set. The mpi variant option in catalyst (i.e. build
with mpi or build without mpi) is passed on to
libcatalyst. The default of the libcatalyst variant
is false/off, so if seacas is installed without the
"+libcatalyst" in the spec it will behave exactly as
it did before the introduction of this variant.
* shortened line 202 to comply with < 100 characters per line style requirement
* py-httpx: New version
* [py-httpx] fix when for dependencies
* [py-httpx] organized dependencies
* [py-httpx] added version 0.27.2
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
* Automated deployment to update package flux-sched 2024-09-05
* flux-sched: add back check for run environment
* flux-sched: add conflict for gcc and clang above 0.37.0
---------
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
autotools packages with a configure script should generate the libtool
executable, there's no point in `depends_on("libtool", type="build")`.
the libtool executable in `<libtool prefix>/bin/libtool` is configured
for the wrong toolchain (libtools %compiler instead of the package's
%compiler).
Some package link to `libltdl.so`, which is fine, but had a wrong
dependency type.
See https://github.com/spack/spack/pull/46314#discussion_r1752940332.
This further simplifies `cxxstd` variant handling in `acts` by removing superfluous
version constraints from dependencies for `geant4` and `root`.
The version constraints in the loop are redundant with the conditional variant
values here:
```python
_cxxstd_values = (
conditional("14", when="@:0.8.1"),
conditional("17", when="@:35"),
conditional("20", when="@24:"),
)
_cxxstd_common = {
"values": _cxxstd_values,
"multi": False,
"description": "Use the specified C++ standard when building.",
}
variant("cxxstd", default="17", when="@:35", **_cxxstd_common)
variant("cxxstd", default="20", when="@36:", **_cxxstd_common)
```
So we can simplify the dependencies in the loop to:
```python
for _cxxstd in _cxxstd_values:
for _v in _cxxstd:
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +geant4")
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +fatras_geant4")
depends_on(f"root cxxstd={_v.value}", when=f"cxxstd={_v.value} +tgeo")
```
And avoid the potential for impossible variant expressions.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Openmpi provider statements were changed in #46102. The package change
was fine in and of itself, but apparently one of our tests depends on
the precise constraints used in those statements. I updated the test
to remove the checks for constraints that were removed.
In #44425, we add stricter variant audits that catch expressions that can never match.
This fixes 13 packages that had this type of issue.
Most packages had lingering spec expressions from before conditional variants with
`when=` statements were added. For example:
* Given `conflicts("+a", when="~b")`, if the package has since added
`variant("a", when="+b")`, the conflict is no longer needed, because
`a` and `b` will never exist together.
* Similarly, two packages that depended on `py-torch` depended on
`py-torch~cuda~cudnn`, which can't match because the `cudnn` variant
doesn't exist when `cuda` is disabled. Note that neither `+foo` or `~foo`
match (intentionally) if the `foo` variant doesn't exist.
* Some packages referred to impossible version/variant combinations, e.g.,
`ceed@1.0.0+mfem~petsc` when the `petsc` variant only exist at version `2`
or higher.
Some of these correct real issues (e.g. the `py-torch` dependencies would have never
worked). Others simply declutter old code in packages by making all constraints
consistent with version and variant updates.
The only one of these that I think is not all that useful is the one for `acts`,
where looping over `cxxstd` versions and package versions ends up adding some
constraints that are impossible. The additional dependencies could never have
happened and the code is more complicated with the needed extra constriant.
I think *probably* the best thing to do in `acts` is to just not to use a loop
and to write out the constraints explicitly, but maybe the code is easier to
maintain as written.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Update var/spack/repos/builtin/packages/fms/package.py: apply patch for fms@2023.03 to fix compiler options bug in cmake config, add variant shared and corresponding patch for fms@2024.02
* Fix fms package audit: use c9bba516ba.patch?full_index=1 instead of c9bba516ba.patch?full_index=1
* Update checksum of patch for fms@2023.03
* CUDA: support Grace Hopper 9.0a compute capability
* Fix other packages
* Add type annotations
* Support ancient Python versions
* isort
* spec -> self.spec
Co-authored-by: Andrew W Elble <aweits@rit.edu>
* [@spackbot] updating style on behalf of adamjstewart
---------
Co-authored-by: Andrew W Elble <aweits@rit.edu>
Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
fixes#46295
A proper solution would be a tag directive that accumulates tags
with the ones defined in base classes.
For the time being, rewrite them explicitly.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Boost:Adjust bootstrapping/b2 options as needed for Windows (the
bootstrapping phase sufficiently differs between Windows/Unix
that it is handled entirely within its own branch).
* Boost: Paths in user-config.jam should be POSIX, including on Windows
* Python: `.libs` for the Python package should return link libraries
on Windows. The libraries are also stored in a different directory.
The option config:install_missing_compilers is currently buggy,
and has been for a while. Remove it, since it won't be needed
when compilers are treated as dependencies.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fast-float: new package
* fast-float: add test dependency
* fast-float: fix doctest dependency type
* fast-float: convert deps to tuple
* fast-float: add v6.1.5 and v6.1.6
* fast-float: patch older versions to use find_package(doctest)
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
* Changed software pull location
The original package pulled a tagged release version from GitHub. That tagged version was created in 2022 and has not been updated since. It no longer runs because newer versions of numpy have removed deprecation warnings for several of their calls. The main branch for this repository has addressed these numpy issues as well as some other important fixes but no new release has been generated. Because of this and the apparent minimal development that now appears to be going on, it is probably best to always pull from the main branch
* [@spackbot] updating style on behalf of aweaver1fandm
* py-your: Changed software pull location
1. Restored original URL and version (0.6.7) as requested
2. Updated py-numpy dependency versions to be constrained based on the version of your. Because of numpy deprecations related to your version 0.6.7 need to ensure that the numpy version used is not 1.24 or greater because the depracations were removed starting with that version
* gptune: new test API
* gptune: cleanup; finish API changes; separate unrelated test parts
* gptune: standalone test cleanup with timeout constraints
* gptune: ensure stand-alone test bash failures terminate; enable in CI
* gptune: add directory to terminate_bash_failures
* gptune/stand-alone tests: use satisifes for checking variants
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add numactl 2.0.16-2.0.18
* Create link-with-latomic-if-needed-v2.0.16.patch
Add a link to libatomic, if needed, for numactl v2.0.16.
* Add some missing patches to v2.0.16
* Create numactl-2.0.18-syscall-NR-ppc64.patch
In short, we need numactl to set __NR_set_mempolicy_home_node on ppc64, if it's not already defined.
* Apply a necessary patch for v2.0.18 on PPC64
* Add libatomic patch for v2.0.16
`spack reindex` relies on projections from configuration to locate
installed specs and prefixes. This is problematic because config can
change over time, and we have reasons to do so when turning compilers
into depedencies (removing `{compiler.name}-{compiler.version}` from
projections)
This commit makes reindex recursively search for .spack/ metadirs.
* py-httpcore: Added new version
* [py-httpcore]
- added version 0.18.0
- restructured dependencies as everything has a when and
type/when ordering was all over the place
* [py-httpcore] ordered dependencies in the order listed in v1.0.5 pyproject.toml
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
When a package is running `setup_dependent_package` on a parent, ensure
that module variables like `spack_cc` are available. This was often
true prior to this commit, but externals were an exception.
---------
Co-authored-by: John Parent <john.parent@kitware.com>
* New package: py-monai
* Fixed linked issues with py-monai
* [py-monai] removed extra line
* [py-monai]
- New version 1.3.2
- ran black
- update copyright
* [py-monai] added license
* [py-monai]
- moved build only dependencies
- specified newer python requirements for newer versions
---------
Co-authored-by: vehrc <vehrc@rit.edu>
* Replace if ... in spec with spec.satisfies in f* and g* packages
* gromacs: ^amdfftw -> ^[virtuals=fftw-api] amdfftw
* flamemaster: add virtuals lapack for the amdlibflame dependency
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Allow flags from different sources (compilers, `require:`, command-line
specs, and `depends_on`) to be merged together, and enforce a consistent
order among them.
The order is based on the sources, e.g. flags on specs from the command
line always come last. Some flag order consistency issues are fixed:
1. Flags from `compilers.yaml` and the command line were always intra- and
inter-source order consistent.
2. Flags from dependents and packages.yaml (introduced via `require:`)
were not: for `-a -b` from one source and `-c` from another, the final
result might rearrange `-a -b`, and would also be inconsistent in terms
of whether `-c` came before or after.
(1) is/was handled by going back to the original source, i.e., flags are
retrieved directly from the command line spec rather than the solver.
(2) is addressed by:
* Keeping track of grouped flags in the solver
* Keeping track of flag sources in the solver on a per-flag basis
The latter info is used in this PR to enforce DAG ordering on flags
applied from multiple dependents to the same package, e.g., for this
graph:
```
a
/|\
b | c
\|/
d
```
If `a`, `b`, and `c` impose flags on `d`, the combined flags on `d` will
contain the flags of `a`, `b`, and `c` -- in that order.
Conflicting flags are allowed (e.g. -O2 and -O3). `Spec.satisifes()` has
been updated such that X satisfies Y as long as X has *at least* all of
the flags that Y has. This is also true in the solver constraints.
`.satisfies` does not account for how order can change behavior (so
`-O2 -O3` can satisfy `-O3 -O2`); it is expected that this can be
addressed later (e.g. by prohibiting flag conflicts).
`Spec.constrain` and `.intersects` have been updated to be consistent
with this new definition of `.satisfies`.
Spack can now bootstrap two new dependencies on Windows: GnuPG, and file.
These dependencies are modeled as a separate package, and they install a cross-compiled binary.
Details on how they binaries are built are in https://github.com/spack/windows-bootstrap-resources
This PR adds py-pybind11 versions 2.13.0, 2.13.1, 2.13.2, 2.13.3, and
2.13.4. It also adds a new conflict between gcc 14 and pybind versions
up to and including 2.13.1.
* Allow deprecating more than one property in config
This internal change allows the customization of errors
and warnings to be printed when deprecating a property.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fix
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Use a list comprehension for "issues"
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
fixes#40791
Currently stacks behave differently if used in unify:false
environments, which leads to inconsistencies during concretization.
For instance, we might have two abstract user specs that do not
intersect with each other map to the same concrete spec in the
environment. This is clearly wrong.
This PR removes the best effort expansion, so that user specs
are always applied strictly.
* gaudi: Specify boost components and add +fiber for v39
* gaudi: Limit fmt version to allow building master branch
* Make boost dependencies a bit more readable
* Remove patches for no longer existing versions
* Replace if ... in spec with spec.satisfies in d* and e* packages
* Use virtuals for different mpi implementations in esmf
* esmf: ^[virtuals=mpi] mpt
* extrae: ^[virtuals=mpi] intel-oneapi-mpi
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
This change aligns the build condition for parmetis with the
depends_on condition.
The current build condition of parmetis looks for "+parmetis" in
the spec which is not added by the depends_on as that adds
"^parmetis" instead.
* Adding addtional check for omptarget library for amdgpu in nvidia environment
* Avoiding registration of duplicate when built on cuda
* Adding hsa library path in LD_LIBRARY_PATH
* Correction in hsa prefix library path in LD_LIBRARY_PATH
==> Error: InstallError: For Trilinos@[master,develop], ^kokkos version in spec must match version in Trilinos source code. Specify ^kokkos@4.4.00 for trilinos@[master,develop] instead of ^kokkos@4.3.01.
* root: Add dependency on libglx
We have been trying to build the Acts package on MacOS, and in this
process we have been running into problems with the ROOT spec on that
operating system; the primary issue we are encountering is that the
compiler is unable to find the `GL/glx.h` header, which is part of glx.
It seems, therefore, that ROOT depends on libglx, but this is not
currently encoded in the spec. This commit ensures that ROOT depends on
the virtual libglx package when both the OpenCL and X11 variants are
enabled.
* Enable builtin glew on MacOS
* Allow `root+opengl+aqua~x` on macOS
dd4hep versions up to and including 1.27 had a conflict with root
versions starting from 6.31.1, as shown in
https://github.com/AIDASoft/DD4hep/issues/1210. This PR explicitly adds
that conflict to the spec.
* whizard: add a patch when using hepmc3 3.3.0 or newer
* whizard: comment with patch origin
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* podio: apply patch for gcc 14 builds
Podio versions after 0.17.0 but before 1.0.0 are broken when using gcc
14 because of a missing include, which is addressed in the podio pull
request at https://github.com/AIDASoft/podio/pull/600. This commit
patches pre-1.0.0 versions of Podio so they can be compiled with gcc 14,
which is important for building Acts.
* Style
* Style 2
* Fixes
* Add comment:
* Add sha256
This should help not selecting, by default, some niche implementation that are supposed to be externals.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This PR simplifies the code doing external spec detection by removing
the `DetectedPackage` class. Now, functions accepting or returning lists
of `DetectedPackage`, will accept or return list of specs.
Performance doesn't seem to change if we use `Spec.__reduce__` instead
of `DetectionPackage.__reduce__`.
- Honor compiler extra_rpaths (this build bypasses spack wrappers,
so the RPATHs are added explicitly as CMake options)
- Use root_cmakelists_dir instead of adding it to the command line
- Add BLT as a dependency, allowing different versions outside of what
is in the tarball
- Put a copy of host-config in the stage directory: this allows
examination of the host-config when a build fails (before, the
host-config was just stored in the install directory, which is
deleted by default on a failed build)
* Add conflict between NVSHMEM and cuFFTMp for GROMACS package
These don't work in the same build configuration.
* [@spackbot] updating style on behalf of mabraham
* Update package.py
Also constrain NVSHMEM appropriately
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
* Update package.py
* [@spackbot] updating style on behalf of mabraham
* Update package.py
* Update package.py
* Update package.py
* [@spackbot] updating style on behalf of mabraham
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
Change the signature of Spec.from_detection to set the
external prefix, and the external modules, if they are
present.
Delete "spack.package_prefs.spec_externals" since it
is unused.
This context manager was used to prevent detecting a platform
as "cray". Since now Cray machines are detected as linux, we can
remove the context manager.
This commit adds version 0.73.0 of the detray package. As this version
drops support for pre-C++20 standards, I had to update the `cxxstd`
variant logic.
Right now if you run `spack ci generate` you get a warning about CDash credentials even
if there's no CDash configuration specified. We should only warn if there was actually a
CDash config.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Source mirrors store entries by digest and add a human readable alias of the
form 'name-version'. If no digest is available, the alias is used as the primary
storage location.
Spack erroneously fetches by alias when the digest path does not exist. This is
problematic if `version(..., sha256=...)` changes in package.py, and the mirror
is populated with the old shasum. That would result in an error when a digest
is available, but in case of git versions with a modified commit sha, the wrong
sources would be fetched without error. With this PR, only the digest path is
used, not the alias, in case a digest is available. This is also a small performance
optimization, as the number of request is halved for mirrors that don't contain
the sources.
Further, for git sources the tag was used as a digest, but this is a moving
target. Only commit sha is used now.
Also whenever the alias already existed, Spack used to keep it in place when
updating the mirror cache, which means that aliases would always point to
outdated mirror entries whenever digests are modified. With this PR the alias
is moved in place.
Lastly, fix a recent regression where `Stage.disable_mirrors` disabled mirrors
but not the local download cache, which was the intention.
* VTK-m: Point to github mirror for source tarball
The gitlab.kitware.com source location seems to have intermittent
network issues. Switching the to mirror hosted at Github may alleviate
some of the timeouts.
* Update sha256 for GitHub tarballs
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
* davix: add versions 0.8.2-0.8.7 and dependencies
This commit adds new versions 0.8.2-0.8.7 of the davix package, and it
also improves the handling of embedded packages. Davix will try to build
libcurl from its own embedded version of that code, which doesn't mesh
well with Spack's design philosophy, so I've changed the CMake
configuration to disallow the builtin libcurl and use a Spack dependency
instead. Up to version 0.8.7, RapidJSON was also builtin, but version
0.8.7 allows users to specify that they want to use a pre-installed
version of RapidJSON, so this commit also adds that as a dependency for
versions 0.8.7:.
* Fix old versions
* acts: add v36.0.0, v36.1.0 and fixes
This commit makes several changes to the Acts repository, namely:
1. It adds versions 36.0.0 and 36.1.0.
2. It adds the traccc plugin and related dependencies.
3. It updates the version requirements of some dependencies.
4. It adds the Geant4 module of GeoModel.
5. It updates the C++ standard requirement.
6. It adds a new variant determining the scalar type to use.
This commit supercedes #45851. Thanks @jmcarcell for the version
updates; I have added you as co-author.
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Updates
* alphabetic cmake args
---------
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Add missing MultiMethodMeta metaclass in builders
and remove the Python 2 fallback option in favor of hard errors to catch
similar issues going forward.
The fallback option can cause about 10K stat calls due to use of
`realpath` in the inspect module, depending on how deep Spack itself is
nested in the file system, which is ... undesirable.
* code shuffling to avoid circular import
* more reshuffling
* move reserved variant names into variants module
Currently, the llvm-amdgpu package doesn't compile on MacOS, but it is
also not marked as a conflict. This causes problems because it seems
that Spack is very happy to pull in llvm-amdgpu as the default package
to satisfy any virtual libllvm dependency, which can cause dependent
specs to fail to install on MacOS. This commit marks a conflict between
this llvm package and the Darwin platform.
* root: set +webgui~http conflict from @6.28.12:
Currently, the ROOT spec correctly identifies a conflict between +webgui
and ~http, but this conflict is marked as affecting @6.29.00: only. As a
matter of fact, ROOT 6.28.12 is also affected. This commit, therefore,
updates the when clause on the conflict to @6.28.12:.
* Remove when clause entirely
* oops
* Adds an MSBuild system + Builder to the icu4c package
* Adds custom install method as MSBuild system does not vendor an
install target
* The cxxstd variant is not supported on Windows (there are no config
options you use to tell the build system what cxx standard to
build against), so the variant definition was updated to occur
everywhere except Windows
Also, this commit defines the c/cxx..._flag properties of the MSVC
compiler (although they are not used by `icu4c` and not strictly
necessary to bundle with this PR).
This commit adds a dependency on GeoModel 4.6.0 when the GeoModel plugin
is enabled. Note that the dependency is upgraded to 6.3.0 in Acts
36.1.0, but that will need to be covered in #45851.
* geomodel: add versions 5.1.0-6.4.0
This PR adds new versions 5.1.0 through 6.4.0 of the geomodel package.
It also updates the CMake configuration to use the `define_from_variant`
mechanism and it adds me as a maintainer.
* Undo cmake change
The avx2 version can be downloaded from the ORCA [forum](https://orcaforum.kofo.mpg.de/app.php/dlext/?view=detail&df_id=214#).
The version is named `avx2-6.0.0` (as opposed to the more
natural-looking `6.0.0-avx2`) to avoid the avx2 version shadowing the
non-avx2 one. Definitely open for better suggestion.
* hsakmt-roct: new test API
* hsakmt-roct: minor change to check_install script variable name
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parallel-netcdf: new test API
* parallel-netcdf: fix test args and tweak docstring and variables
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* WarpX: Python on pyAMReX
Long overdue update for WarpX: in 2024, we updated our Python
bindings to rely on the new pyAMReX package. This deprecates the old
`py-warpx` package and adds a new dependency and variant to WarpX.
Also deprecates old versions that we will not continue to support.
* Update Cloud/E4S Pipelines for WarpX
`py-warpx` is replaced by `warpx +python`
oneAPI does not support IPO/LTO: diable for `py-amrex` even though
pybind11 strongly encourages it.
* Add `+python` variant
* `esmf` package installs Python bindings when `+python` is set
Note: this does not inherit `PythonPackage`, which force an either/or
choice between the Makefile and Pip builder: it instantiates a
`PythonPipBuilder` as needed (when `+python` is set).
Paths over 260 characters in length are not handled by `shutil.rmtree`
unless they use the extended-length path syntax (using a prefix of
"\\?\").
This fixes an issue where stage cleaning fails when paths in a stage
exceed the normal 260-character limit.
This indicates that other parts of the codebase should be examined/
refactored to handle long paths.
* harfbuzz: enable freetype in MesonBuilder to facilitate depends_on("freetype")
* spack style fix
* freetype is defined as a depends_on(), so set as enabled in MesonBuilder rather than an option/flag/variant
* add back depends_on($lang) lines for new test api
* keep original order
Previously `spack buildcache push --only=package` errored in the OCI
case, but it's been requested that OCI can be used as pure storage w/o
the need for runnable container images.
This commit makes it so that
1. manifests refer only to runtime dependencies that were selected to be
pushed
2. failure to upload a blob among the selected specs does not prevent a
manifest/tag to be created for dependents: they just don't refer to
the missing blob as a layer/dependency
This fixes the following issues:
1. dependents of non-redistributable specs can now be pushed to oci
build caches without error
2. failure to upload one tarball does not cause cascading failures for
dependents whose tarballs do upload succesfully -- so it's better
best-effort behavior
3. for some people uploading with deps caused a massive amount of
fetches of their manifests (which certain registries count as a
download of an image, even though their layers are not fetched) --
being able to specify --only=package reduces the number of fetches.
Stage objects create mirrors ad-hoc from current config.
- There is no way to prevent mirrors from being used
- There is no way to restrict mirrors to source/binary, which is of
course context dependent.
- Stage is also used in build caches, where iterating over mirrors is
already implemented differently, and wouldn't work anyways cause it's
source only, and in particular it makes no sense for OCI build caches.
This commit:
1. Injects the sensible mirrors into the stage object from contexts
where it is relevant
2. Separates mirrors from cache, so that w/o mirrors download cache can
still be used
The test_foreground_background unit test has been marked
xfail for a while, meaning:
- Nobody looks at the results of the test
- It still runs every time
That test happens to hang frequently on some Apple M1 I have access to,
so here I mark it as skip.
Also went through other xfailing and skipped tests, and applied minor changes.
* Use absolute paths instead of https:// fake mirrors (this speed-up tests by avoiding requests)
* Add a fixture to gather in a single place code that is copy/pasted in a lot of tests
* General clean-up of tests and repeated code
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* hypre: get rid of depreciated run_test method
* hypre: use mkdirp directly
* hypre: use install() for ij for addition of permissions fix
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parsec: old to new test API
* parsec: restore stand-alone test subparts; preliminary test build fixes
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* n2p2: convert from old to new test API
* n2p2: Enhance stand-alone testing checks to reduce unnecessary processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* stand-alone test API update: self.cache_extra_test_sources(...) -> cache_extra_test_sources(self, ...)
* superlu: switch to new cache_extra_test_sources API
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add options for sparse checkout in GitFetcher
Newer versions of git have a beta feature called sparse checkout
that allow users to check out a portion of a large repo.
This feature will be ideal for monolithic repo projects that want to
model their infrastructure via spack. This PR implements an addition
to the GitFetcher that allows users to add a `git_sparse_paths`
attribute to package classes or versions which will then use sparse
checkout on those directories/files for the package.
* Style
* Split git clone into multiple functions
* Add sparse-checkout impl
* Internalize src clone functions
* Docs
* Adding sparse clone test
* Add test for partial clone
* [@spackbot] updating style on behalf of psakievich
* Small fixes
* Restore default branch status
* Fix attributes for package
* Update lib/spack/docs/packaging_guide.rst
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Extend unit test to multiple git versions
* style
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Feature update py-fortranformat
Add more recent versions of py-fortranformat. The currently included release (0.2.5) is from 2014. I've added the latest point release of each of the major versions from the last 4 years.
* update homepage
* magma: old to new test API
* magma: simplify stand-alone test method/part docstrings/purposes
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* initial commit to enable asan in remaining rocm packages
* remove os import
* add rocm-opencl rocm-dbgapi rocm-debug-agent
* add libclang path to LD_LIBRARY_PATH
* enable asan for rocfft
* add f-string and add +asan dependencies for hip and rocm-opencl
* add conflicts for centos7/8 and rhel 9
When sending Repo and RepoPath over to a child process,
we go through a marshaling procedure with pickle. The
default behavior for these classes is highly inefficient,
as it serializes a lot of specs that can just be
reconstructed on the other end of the pipe.
Here we write optimized procedures to __reduce__ both
classes.
* raja: new test API
* raja: tweak test method names and docstrings
* raja: restore running tests under proper directory
* raja: cleanup skiptest message and example call
* raja: Tweak expected outputs to match current
* raja: test_views -> test_stencil_offset_layout
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* mptensor: Changed skiptest, test name, and added docstring
* mptensor: make stand-alone test method name and docstring more specific
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add version 2.0.-2
* Newer version at the top
* [@spackbot] updating style on behalf of rahmans1
* py-htgettoken: reorder comments
* Use sha256sum instead of commit id for version 2.0-2 and above
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Extracted from #44419
This adds / modifies some external detection tests for compilers,
to reproduce cases that are currently tested in unit tests.
The unit tests will later be removed.
Extracted from #45638
When adding the "detectable" tag to a package class that has the
"tag" attribute inherited from a base class, we need to copy it to
avoid modifying the base class.
* kickoff attempt
* resource similar to fortran
* delete unused install_component_codeplay
* Adding conflict for versions <= 2022.2.1, moving install to package.py, adding sha256 for version 2024.2.1
* [@spackbot] updating style on behalf of kaanolgu
---------
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
"spack buildcache push" for partially installed environments pushes all it
can by default, and only dumps errors towards the end.
If --fail-fast is provided, error out before pushing anything if any
of the packages is uninstalled
oci build caches using parallel push now use futures to ensure pushing
goes in best-effort style.
* migraphx: Old to new test API
* migraphx: tweak name and docstring to be more descriptive
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* c: new test API
* gcc: provides('c')
* c: bugfix and simplification of the new stand-alone test method
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* fortran: new test API
* fortran: add provides to gcc package
* fortran: simplify stand-alone test processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* hypre-cmake: old to new test API
* hypre-cmake: update Makefile to use installed files
* hypre-cmake: make stand-alone test method name more specific
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
The old concretizer is still used to bootstrap clingo from source. If we switch to a DAG model
where compilers are treated as nodes, we need to either:
1. fix the old concretizer to support this (which is a lot of work and possibly research), or
2. bootstrap `clingo` without the old concretizer.
This PR takes the second approach and gets rid of the old concretizer code. To bootstrap
`clingo`, we store some concrete spec prototypes as JSON, select one according to the
coarse-grained system architecture, and tweak them according to the current host.
The old concretizer and related dead code are removed. In particular, this removes
`Spec.normalize()` and related methods, which were used in many unit-tests to set
up the test context. The tests have been updated not to use `normalize()`.
- [x] Bootstrap clingo concretization based on a JSON file
- [x] Bootstrap clingo *before* patchelf
- [x] Remove any use of the old concretizer, including:
* Remove only_clingo and only_original fixtures
* Remove _old_concretize and _new_concretize
* Remove _concretize_together_old
* Remove _concretize_together_new
* Remove any use of `SPACK_TEST_SOLVER`
* Simplify CI jobs
- [x] ensure bootstrapping `clingo` works on on Darwin and Windows
- [x] Raise an intelligible error when a compiler is missing
- [x] Ensure bootstrapping works on FreeBSD
- [x] remove normalize and related methods
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`setup-env.sh` is meant to be sourced, not executed directly.
By revoking execution permissions, users who accidentally execute
the script will receive an error instead of seeing no effect.
* Remove execution permission from `setup-env.sh` and friends
* Don't make output file executable in `spack commands --update-completion`
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* On Windows it is built with CMake, however CMake is built against
libuv, so libuv must depend on cmake+ownlibs to short circuit the
circular dependency
* libuv currently fetches the -dist source distribution of libuv for
certain versions because those versions contain a pre-generated
./configure script. However those distributions have all CMake
files removed, so they cannot be used to build on Windows.
Because the source distributions are different, this means the
checksums are different, and necessitates an additional version
declaration for each version we want to support with CMake.
* Update available versions
* fix style
* Reduce added versions to just a single new version as per review
* fix style
* Set dependency versions in line with pyproject and setup.py
* add new interp variant
* add wheel as required
* Add variant description
* change +interp default, actually add packaging this time
Fixes#43494
Add a set of environment variables SPACK_ALWAYS_CFLAGS (etc.) that
are always applied by the compiler wrapper.
Unlike SPACK_CFLAGS, for example, these will also be applied to
version checks (both SPACK_CFLAGS and SPACK_ALWAYS_CFLAGS will be
applied to the other invocation modes like ccld etc.).
Using this new functionality, the classic Intel and oneAPI compilers
are updated to pass compiler flags that disable warning messages
when newer versions are invoked via their older binary names
(these warnings were also generated for version checks, hence the
need for a new wrapper variable).
---------
Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
* namd:add compile options for ROCm
* Combine --rocm-prefix and its value in a single opts.extend call to ensure they remain ordered correctly and improve code robustness.
* Standardize the code and add ROCm conflicts
* add single_node_gpu conflict
---------
Co-authored-by: U-PALLAS\boudaoud <boudaoud@pc44.pallas.cines.fr>
When clang is installed I get `clang: Permission denied`.
Setting CLANG to the full path of clang fixes this.
Signed-off-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Fixes error:
```txt
3 errors found in build log:
61 ==> xmlto: Executing phase: 'build'
62 ==> [2023-10-26-09:48:35.425903] 'make' '-j16' 'V=1'
63 make all-am
64 make[1]: Entering directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
65 /bin/sh ./ylwrap xmlif/xmlif.l .c xmlif/xmlif.c -- /bin/sh /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing flex
66 /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing: line 81: flex: command not found
>> 67 WARNING: 'flex' is missing on your system.
68 You should only need it if you modified a '.l' file.
69 You may want to install the Fast Lexical Analyzer package:
70 <http://flex.sourceforge.net/>
>> 71 make[1]: *** [Makefile:757: xmlif/xmlif.c] Error 127
72 make[1]: Leaving directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
>> 73 make: *** [Makefile:584: all] Error 2
```
For example: spack install py-pyqt5 or py-pyqt6 would install pylupdate5 pyrcc5 and
pyuic5 into in python-venv's install prefix.
Fix https://github.com/spack/spack/issues/45359
* [py-configspace] fix dependency on cython
* py-cython not needed starting from 1.0.0
* added py-configspace 1.0.0 and 1.0.1
* py-configspace: fix style
* added py-configspace version 1.1.0
* added py-configspace version 1.1.1
* py-configspace: two more versions and new maintainer
* py-configspace: fixed typo
* py-configspace: added version 1.1.4
* Allow spec queries by `namespace`
Spack specs have "namespaces" that indicate what package repository they come from, but
there has not been a way to use the spec syntax to match one.
You can say things like this:
```console
spack find builtin.zlib
spack find myrepo.zlib
```
But, because namespaces are written as a dot-separated prefix on the name, you can't say
"find me all specs in namespace myrepo". The syntax doesn't allow it.
This PR allows you to specify anonymous specs with namespaces on the CLI. Specifically
you can do queries like this:
```console
spack find namespace=builtin
spack find namespace=myrepo
```
You can use this anywhere else you use spec syntax, e.g. in a config file to separate
installations based on what repo they came from:
```yaml
spack:
config:
install_tree:
root: $spack/opt/spack
projections:
namespace=myrepo: "myrepo_special_path/{name}-{hash}"
namespace=builtin: "builtin/{name}-{hash}"
```
This PR adds a special `namespace_if_anonymous` attribute to specs, which returns the
`namespace` if the spec has no name, otherwise it returns `None`. This allows us to
print the namespace for anonymous specs but to continue hiding it for most views, as
we've done so far.
This is implemented as a special case, but it's one that already exists, along with
`platform`, `os`, `target`, etc. This also reserves existing special case names for
variants so that users cannot define them in their package files. This change is
potentially breaking, but I do not think it will be common. There are no builtin
packages with a variant called `namespace`, and defining `os`, `target`, or `platform`
as a variant would've likely caused other problems if they were already being used.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* format: allow spaces in format specifiers
Key-value pair format specifiers can now contain spaces in the key. This allows us to
add spaces to format strings that are *only* present when the attribute formatted is not
``None``. Instead of writing:
```
{arch=architecture}
```
and special casing `arch=` like a sigil in `Spec.format()`, we can now write:
```
{ arch=architecture}
```
And the space is *only* printed when `architecture` is not `None`. This allows us to
remove the special case in `Spec.format()` for `arch=`.
Previously the only `key=` prefix allowed in format specifiers was `arch=`, but this PR
removes that requirement, and the `key=` part of a key-value specifier can be any name.
It does *not* have to correspond to the formatted attribute.
- [x] modify `SPEC_FORMAT_RE` to allow arbitrary keys in key-value pairs.
- [x] remove special case for `arch=` from `Spec.format()`.
- [x] modify format strings using `{arch=architecture}` to use `{ arch=architecture}`
- [x] add more tests for formatting
This PR saves other more complex attributes like compiler flags and their spacing for later.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
VC toolset versions 144 and 143 are both associated with the platform
toolset 143; this deviates from prior version choices made by the
MSVC devs; add a special case to report platform toolset version
as 143 when VC toolset version is >= 143 (this will need revisiting
for later releases).
The setSet tool (removed in version 10) has an optional dependency on readline.
The build script will use the system readline if not provided by Spack.
* dd4hep: restrict to podio@0 to prevent failures with podio@1
* acts: restrict to podio@0 to prevent failures with podio@1
* dd4hep: close when range for podio
* acts: close when range for podio
* acts: fix when range for podio
* acts: close when range for podio
* acts,dd4hep: ensure main/master still depends on podio
* Fix +rocm variant, to ensure correct dependencies on ROCm packages
and use of AMD LLVM
* Add a +pshm variant for comm=gasnet to enable fast shared-memory
comms between co-locales
* Add logic to ensure we get the native CXI libfabric network provider
on Cray EX
* Expand dependency type for package modules to encompass runtime
dependencies
* Factor logic for setting (LD_)LIBRARY_PATH and PKG_CONFIG_PATH of
runtime dependencies
* Workaround issue #44746 that causes a transitive dependency on lua
to break SLURM
* Disable nonfunctional checkChplDoc test
* Annotate some variants as conditional, to improve spack info output
and reduce confusion
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
`apple-libuuid` includes types that aren't available in other `uuid`
providers; this cause issues in consuming packages (e.g., py-matplotlib)
that use SDKs like CarbonCore.framework when they attempt to use
`util-linux-uuid` as a `uuid` provider on Mac OS.
Tweak `util-linux-uuid` to indicate that it does not provide `uuid`
on Mac OS.
This was missed in #45394 because we don't run unit tests for package PRs, and
`test_all_virtual_packages_have_default_providers`, which would've caught it, is a unit
test, not an audit.
- [x] add a default provider for `wasi-sdk` in `etc/spack/defaults/packages.yaml` (which
we require for all virtuals)
- [x] rework `test_all_virtual_packages_have_default_providers` as an audit called
`_ensure_all_virtual_packages_have_default_providers`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* pinentry: New API
* move code around
* added back version check
* Complete check_version refactor
* Honor original handling of guis (i.e., don't try if not there)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* upgrade apptainer: security fix
apptainer 1.3.1 has a high severity security issue: CVE-2024-3727
Upgrade to 1.3.2 or preferably 1.3.3
* added comment for 1.3.1
---------
Co-authored-by: Walter de Jong <walter.dejong@surf.nl>
* intel-oneapi-compilers: update description with current compilers
* Update var/spack/repos/builtin/packages/intel-oneapi-compilers/package.py
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* intel-oneapi-compilers: break docstring line
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* kokkos: new test API
* kokkos: added import llnl.util.lang as lang because earlier versions couldn't be installed without it
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* povray: new test API
* capture output and test name change
* povray: add v3.7.0.10, deprecate 3.7.0.8
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* umpire: old to new test format
* umpire: old to new test method and refactor
* indentation
* black reformat
* last minute syntax
* docstring and checks
* black format
* change test name
* method call correction
* Resolve problem with stand-alone tests (or examples) not loading library (Fixes#44960)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* sz: new test API
* fix typo; check installed executable; conform to subpart naming convention
* skip tests early if not installed; remove unnecessary "_sz" from test part names
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Library names on Windows are not typically prefixed with lib; the default
`.libs` implementation accounts for this, but `netlib-lapack` has a
custom implementation of `.libs` that did not account for this.
The `Executable` class emits a warning when you pass quoted arguments to it:
```
> spack pkg grep '"namespace"'
==> Warning: Quotes in command arguments can confuse scripts like configure.
The following arguments may cause problems when executed:
"namespace"
Quotes aren't needed because spack doesn't use a shell. Consider removing them.
If multiple levels of quotation are required, use `ignore_quotes=True`.
```
This is to warn new package authors who aren't used to calling build commands in python.
It's not useful for `spack pkg grep`, where we really are passing args on the command
line, and if we pass a quoted string, we probably meant to.
- [x] make `ignore_quotes` an instance variable, not just an argument to ``__call__`
- [x] set `ignore_quotes` to `True` in `spack pkg grep`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* package/linux-pam: dependencies
* Adding variants to linux-pam
* Updating linux-pam variants
* Fixing variants for linux-pam after testing
* clean up flag handling
* clean up terrible tabs
* cant use default_args for compiler dependencies
* Change selinux to off by default
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Adding new versions and compilation conflict for nodejs
* Update failed version for gcc14
* Updating conflicts notes for correctness/clarity/format
* Applying spack-ized versions of fix in https://github.com/nodejs/node/issues/52223 to adddress CI failures
* Update fix-old-glibc-random-headers.patch
* Update package.py
* Update fix-old-glibc-random-headers.patch
* Update fix-old-glibc-random-headers.patch
* Adding conflict for older glibc
* Fixing patch for older systems, need to undef
* Removing overly strict conflicts
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Several tests can be enabled on Windows with no changes to logic
(either the test logic or logic being tested)
* Test for `spack location` requires modification of the test logic,
but with a minor change can be enabled on Windows
using self.compiler.cc_pic_flag here results in these errors:
==> sqlite: Executing phase: 'install'
==> Error: AttributeError: 'AutotoolsBuilder' object has no attribute 'compiler'
change it to self.pkg.compiler.cc_pic_flag instead.
* imoroved-rdock: new test API
* Make test subpart names and or descriptions more accurate
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* e4s ci: enable some disabled specs
* comment out cp2k +cuda due to unsupported cuda_arch
* comment out dealii+cuda due to concretize error
* work through concretize errors
* e4s: comment out failing builds
* e4s stack: disabled non-building specs
* comment out failing specs
* comment out failing specs
* cleanup comments
* rust: rework external find to require both rustc and cargo
* rust: handle unable to parse version
* [@spackbot] updating style on behalf of wdconinc
* rust: not x or not y -> not (x and y)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: pick first rustc found
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: list comprehensions
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
* libmesh: add missing v1.7.1 release
* libmesh: avoid pulling in petsc if only +mpi or +metis is given
* libmesh: add shared variant
Co-authored-by: rbberger <rbberger@users.noreply.github.com>
The "use_store" context manager is used to swap the value
of a global variable (spack.store.STORE), while keeping
another global variable consistent (spack.config.CONFIG).
When doing that it tries to evaluate the previous value
of the store, if that was not done already. This is wrong,
since the configuration might be in an "intermediate" state
that was never meant to trigger side effects.
Remove that operation, and add a unit test to
prevent regressions.
Add language dependencies `c`, `cxx`, and `fortran`.
These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.
The `# generated` comment can be removed by package maintainers after
validating correctness.
* detray: new package
This commit adds the detray package, a detector description library for
HEP experiments that is designed to be GPU-friendly.
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* vasp: add new version 6.4.3 and rework package
* vasp: remove redundant cuda dep
* vasp: bump fftlib variant version restriction
* vasp: honor the still existing scalapack variant
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Since the the MetaPathFinder now owns a lazily constructed RepoPath object, we need to deserialize environments before the package that needs to be restored. Before we were relying on globals to be inconsistent in a way that let the entire process go.
The patch which enables icx/ifx compilers is only added for `wrf@4.4:`. This PR prints a useful message at concretization time instead of failing the installation later on.
Co-authored-by: stephenmsachs <stephenmsachs@users.noreply.github.com>
* slate: changed from old to new format
* make code tighter
* replace assert method
* SkipTest plus other cleanup
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* vecmem: add infrastructure for working with SYCL
The vecmem package uses an unorthodox build system where, instead of
expecting a SYCL-capable compiler in the `CXX` environment variable, it
expects one in `SYCLCXX`. It also needs the correct SYCL flags to be
set. This commit adds a custom build environment for the vecmem package
which allows it to be built in this way. I've also added an extra CMake
flag to ensure that the build system doesn't download any unwanted
dependencies.
* Update var/spack/repos/builtin/packages/vecmem/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
- Add explicit conflict on ppc64le for 12.5 and newer.
- Update/fix intel compiler conflict to reflect that intel@2021 is compatible
only since 11.4.1 and not since 11.1.1.
- Add intel compiler conflicts to reflect strict support matrix since
12.2.0.
As discussed in https://github.com/spack/spack/pull/44881#issuecomment-2218411735 a `spack install py-globus-cli` fails to concretize on an Ubuntu 22.04 under Windows WSL2 because of too strict of explicit dependencies.
Let's try to remove them here (since these are "just in case" and in all honesty should be handled by `py-globus-sdk` anyways).
* Now builds with latest fmt release (11.0.1)
* Missing array include in nem_spread
* Fix timestep conssitency check in file-per-rank case if one or more dbs have no timesteps.
If an external happens to be a provider of anything, the solver would
set its weight to 0, meaning that it is most preferred, even if
packages.yaml config disagrees.
That was done so that `spack external find mpich` would be sufficent to
pick it up as mpi provider.
That may have made sense for mpi specifically, but doesn't make sense
for other virtuals. For example `glibc` provides iconv, and is an
external by design, but it's better to use libiconv as a separate
package as a provider.
Therefore, drop this rule, and instead let users add config:
```
mpi:
require: [mpich]
```
or
```
mpi:
buildable: false
```
which is well-documented.
This change makes `spack external find --not-buildable` mark virtuals
provided by detected packages as non-buildable, so that it's sufficient
for users to let spack detect say mpich and have the concretizer pick it
up as mpi provider even when openmpi is "more preferred".
Consistently with ArmPL@24:, the include directory for acfl@24:
has changed to include. The change wants to update to this change
and distinguish the include path for releases previous to 24.04
and the future ones
* rocm-cmake: changed test format from old to new
* Rename cmake variable
* post-conflict resolution: remove remaining version check
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Added Release 0.0.4 and 0.0.5
* Changed requirement for gotcha
use gotcha 1.0.5 for 0.0.2 and 0.0.3
* Combine gotcha 1.0.7 for master and develop
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
in case of inheritance the static tags prop may be updated multiple
times, and it turns out builder classes magically inherit from
traditional package classes
While trying to build packages with the OneAPI compiler version 2024.1 I
ran into the following error, indicating that the compiler is unable to
find some necessary libraries:
```
/storage/Software/oneapi/2024.1/compiler/2024.1/bin/sycl-post-link: error
while loading shared libraries: libonnxruntime.1.12.22.721.so: cannot open
shared object file: No such file or directory
icpx: error: unable to execute command: No such file or directory
icpx: error: sycl-post-link command failed due to signal (use -v to see
invocation)
```
Indeed, `libonnxruntime.1.12.22.721.so` does come bundled with the
OneAPI compiler, but it is not available in the build environment by
default. In this commit, I update the custom environment created by
OneAPI to include the `lib/` directory in which these libraries reside
in the `LD_LIBRARY_PATH` environment variable.
In my enthusiasm to add dfelibs to Spack, I didn't realise that the
unit tests of dfelibs use Boost and, as such, Boost is required as a
testing dependency.
Originally if you had `x -> y -> z`, and an env with `x` in its speclist that is concretized but not installed, then `spack find -c y` would not show anything. This was intended: `spack find` has up-until-now only ever listed out installed specs (and `-c` was for adding a preamble section about roots).
This changes `spack find` so:
* `-c` makes it search through all concretized specs in the env (in a sense it is anticipated that a concretized environment would serve as a "speculative" DB and users may want to query it like they query the DB outside of envs)
* Adds a `-i/--install-status` option, equivalent to `-I` from `spack spec`
* Shows install status for either `-c` or `-i`
* As a side effect to prior point, `spack find -i` can now distinguish different installation states (upstream/external)
Examples:
```
$ spack find -r
==> In environment findtest
==> 1 root specs
- raja
==> 6 installed packages (not shown)
==> 12 concretized packages to be installed (not shown)
```
```
$ spack find
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
berkeley-db@18.1.40 bzip2@1.0.8 diffutils@3.10 gmake@4.4.1 gnuconfig@2022-09-17 libiconv@1.17
==> 6 installed packages
==> 12 concretized packages to be installed (show with `spack find -c`)
```
```
$ spack find -c
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
[+] berkeley-db@18.1.40 [+] bzip2@1.0.8 - cmake@3.29.4 [+] diffutils@3.10 [+] gmake@4.4.1 [+] libiconv@1.17 - nghttp2@1.62.0 - pkgconf@2.2.0 - readline@8.2
- blt@0.6.2 - camp@2024.02.1 - curl@8.7.1 - gdbm@1.23 [+] gnuconfig@2022-09-17 - ncurses@6.5 - perl@5.38.2 - raja@2024.02.2 - zlib-ng@2.1.6
==> 6 installed packages
==> 12 concretized packages to be installed
```
$ spack -E find
...
==> 82 installed packages
```
* Add py-datasets version 2.20.0
* Add dependency requirements for version 2.20 + refactor
* Add missing tqdm and requests versions and to install latest py-datasets
* Add missing python requirements for 2.8.0 and 2.20.0
Fix a bug where Spec.tree with cover=nodes reduces deptypes from all
in-edges, including from nodes not reachable from the root, which almost
always happens for concrete specs
`Spec.__getitem__` queries dependent edges, which almost always point to
nodes outside the sub-dag considered. It should only ever look at edges
being traversed.
When using a newer Clang for Kokkos than supported by a given CUDA version, the
CUDA compiler detection in Legion's CMake still needs to be passed
CMAKE_CUDA_FLAGS to pass the compiler check.
This modifies heuristic to decay to clingo default
over time. The hope is that this helps with specs
that have an optimal solution with a high penalty.
Let target and compiler heuristic decay too, do not
guess compiler
* dfelibs: new package
This commits adds the `dfelibs` package, which is an Acts project fork
of dfelibs.
* dfelibs: docstring typo
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Add MatrixSwitch package
Add libOMM package
Use libOMM as external in ELSI
Add include paths for all libraries
Fortran modules need one directory up (can't use libs.directories since they are just the include/ directories)
This addresses a few issues in the kentutils package:
- Issue #44372
- Cleaning up formattting and styles
- Removing old versions that are not avaialble anymore
- Removing newer versions that are also not available anymore
- The installer does not install the static libs
that are expected by packages that depend on kentutils
- There are conflicts and less-than-desirable dependencies
that can be addressed via variants
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Bumping up to ROCm 6.1.2
* Bump up rocmlir & amdsmi to 6.1.2
* Removing llvm dependency from amdsmi
* Removed redundant for loops and brought rocm-cmake in for loop
* Removing version check of deprecated versions
This adds spack.util.spack_yaml.anchorify, which takes a non-cyclic
dict/list structure, and replaces identical values with (back)
references to the first instance, so that yaml serialization will use
anchors.
`repr` is used to identify sub-dags, which in principle is quadratic
complexity in depth of the graph, but in practice the depth is O(1) so
this should not matter.
Then this is used in CI to reduce the size of generated YAML files to
30% of their original size.
* lammps: add new versions
* lammps: add directories for plugins
* lammps-example-plugin: new package
Example illustrating how to install a LAMMPS plugin, based on the
example part of the LAMMPS distribution.
If netlib-lapack is built with ~external-blas, it internally links
liblapack.so with libblas.so, meaning that whenever netlib-lapack is
used as a lapack provider, the package must also be a blas provider.
Conversely using netli-lapack as a blas provider does not imply that it
also must provide lapack, but nothing is lost disallowing that...
In practice people don't care about having their build dependencies reinstalled with say cflags=-O3 if that is what is set at the input spec, so restrict propagation to link/run deps only.
Also simplify the encoding in asp.
* Changed test method from old method to new method
* Corrected test name and added docstring
* Split test method into stand alone-tests
* Added SkipTest
* code refactor
* removed repeated code
* Remove exe from test part purpose
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve proj version selection to avoid too new of a proj with older api usage in vtk9
* Proj constraint range around 8.2.1
---------
Co-authored-by: John Parent <john.parent@kitware.com>
Fix the following on Windows:
* `spack load --list` (this printed 0 packages even if packages were
loaded)
* `spack unload <package>` (this said that the package is not loaded
even if it was)
Update unit tests for `spack load` to also run on Windows (specifically
for ".bat"). This involved refactoring a few tests to parameterize
based on whether the unit tests are being run on a Windows system
(and to account for batch syntax).
* Adding additional versions to plink2 and switching to tarballs to allow for better version detection in the future
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
The pcluster pipeline runs spack external find and autodetects the most ancient of zlib versions on the system: 1.2.7, release date May 3, 2012.
Turns out some packages require parts of the zlib api added within the last decade.
* update e4s stacks
* adios2 +rocm: disable kokkos due to spack issue #44832
* comment out mgard+cuda due to spack issue #44833
* comment out cabana, legion, arborx due to kokkos spack issue #44832
* comment out slepc, petsc due to petsc spack issue #44600
* comment out adios2+rocm due to kokkos rocm spack issue #44832
* comment out kokkos due to spack issue #44832
When we changed how to deal with errors in November,
we didn't realize that for an unconstrained choice
rule it is more important in the heuristic to guess
what is NOT in the answer set, since it will be the
majority of options.
Previously this was following automatically from what
was in the answer set, via `1 { ... } 1` cardinality
constraints.
Here we improve the heuristic and the solve time for specs.
* Patched CMakeLists.txt to find drm include at libdrm - archlinux
* Restricted usage patch to versions 34 onwards
* Update var/spack/repos/builtin/packages/rdma-core/libdrm.patch
---------
Co-authored-by: Mark Abraham <Mark.J.Abraham@gmail.com>
* Spglib: add version 2.4.0
* DLA-Future: fix +test option
* elsi
* elsi improvements
* [@spackbot] updating style on behalf of RMeli
* remove test variants
* fix ntpoly and and use externals as default
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* py-multiqc: updating to @1.21, adding new dependency py-pyyaml-env
* Correcting dependencies
* Drop matplotlib depends_on() for versions of multiqc not included in package.py
---------
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
* Allow zlib to find external installations.
* Apply suggestions from code review
* Fixed the determine_version function to loop over all of the potential
libraries that could be installed by zlib.
---------
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
* Update hohqmesh and feq-parse versions
Update feq-parse homepage to new documentation page
Update feq-parse license to be consistent with 2.2.2 release
---------
Co-authored-by: fluidnumerics-joe <fluidnumerics-joe@users.noreply.github.com>
This commit adds new versions of actsvg. I have also taken the liberty
of adding myself a maintainer as I work actively on the ACTS project and
its dependencies, and would like to start keeping the Spack specs more
up to date.
* e4s external rocm ci: bump rocm stack to v6.1.1
* comment out exago+rocm due to issue with raja@0.14.0 see spack issue #44593
* comment out adios2+rocm due to spack issue #44594
* comment out petsc+rocm due to spack issue #44600
* comment out sundials+rocm due to spack issue #44601
* comment out slepc+rocm due to petsc spack issue #44600
* comment out tau+rocm due to spack issue #44659
* comment out ecp-data-vis-sdk due to spack issue #44745
* packages: register rocm-core as external
* re-enable tau due to issue #44659 having been resolved
* use latest ci image: ecpe4s/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.1.1:2024.06.17
* comment out paraview due to spack issue #44745
* comment out ecp-data-vis-sdk +vtkm due to issue https://gitlab.spack.io/spack/spack/-/jobs/11632511
* Update for 2.0.1
* cloudpickle dependency is only 'run'
* Follow new formatting guidelines
* black wants trailing commas
* Simplified version ranges, as recommended by @tldahlgren
* vtk-m: Changed test method names and skipping non-applicable tests from old to new approach
-----
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* add cray detection taken from upcxx
* add CUDA/ROCm support
* add numerous pass-through options to Chapel build,
like gpu_mem_strategy, comm_substrate, etc.; all variants are
translated to analogous CHPL_* environment variables. As a side
effect, this defines a number of environment variables that are
not actually used by Chapel.
* Define LD_LIBRARY_PATH, LIBRARY_PATH, and PKG_CONFIG_PATH to
help programs built with Chapel properly locate needed runtime
dependencies
---------
Co-authored-by: bonachea <dobonachea@lbl.gov>
* Projsync needs curl
This fixes a proj compilation error from paraview
* tiff does not set TIFF_INCLUDE_DIR tested by proj
* TIFF patch exists for proj 9.2
* Add comments with location of proj patch
* Use TIFF patches for different cmake versions
* Use modules starting cmake 3.28 and variables before that
* Fix black style
* Simpler patches, add newer proj
* Remove duplicated flag
* Deprecate PROJ 4.8 and older
* Remove PRIVATE
* Deprecate PROJ 7.0.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Not sure why I had this here, as `/bin/sh` will find the first spack in `PATH` just like
`env`.
- [x] remove `/usr/bin/env` and avoid an extra process launch.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* omega-h: reformatted from old test method to new test method
* omega-h: added proper output checking
--------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The ``+cluster` variant requires there be an MPI family known in the
spec. When using externals it is easy to miss-configure this requirement
leading to a runtime exception which is not desirable. This converts the
exception to a package rule.
* xorg libs: new versions
* xorg pkgs: update homepage from cgit to gitlab
* xorgs pkgs: fix homepage since, yeah, I did edit those 122 files by hand...
* libxmu: don't need the .0 patch version here
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Fixed some issues with installation breaking due to mysql_client
* Remove debugging details
* Adding client_only to deps
* Adding client_only to variant and deps
* Undoingw client_only to variant and deps
* py-pyopengl: new simple package
* py-pyopengl: Fix typo in comment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pyopengl: add variants and dependencies
* py-pyopengl: build from source and improve variants
* py-pyopengl: use corrected freeglut libs
* py-pyopengl: update copyright
* py-pyopengl: remove duplicate conflict clause
* py-pyopengl: change dependency to link
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Core change: logic for extracting RPATHs from modules may return
`None`: filter this out of the set of RPATHs that is auto-generated
* Core change: `CachedCMakePackage` no longer adds ldflags to
`CMAKE_STATIC_LINKER_FLAGS`: generally these flags are not appropriate
for static linking (e.g. invocation of `ar`)
* [mfem] Add version 4.7
* [mfem] Add variant for precision (single/double). Enforce consistency
for precision amongst mfem and hypre/petsc/mumps dependencies
* [mfem] Add cxxstd (and related constraints preventing use of
old cxxstd values for newer versions of some dependencies)
* [hypre] In line with prior point, added support for specifying
precision
* [petsc] Add config option to avoid error when building against
`superlu-dist+rocm`
* [hiop] add proper `raja`/`umpire`/`camp` version constraints for
`hiop` versions 0.3.99-0.4.x; require `+raja` for `+rocm`, and
add dependency on `hiprand` for `+rocm`
* [butterflypack, mfem, strumpack, suite-sparse] Require
`CRAYLIBS_{target-family}` env var to be defined
* [suite-sparse] versions `@7.4:` changed install location of headers:
add symlink from old location to new location
* [zlib-ng] Fix error where shared libs were not successfully built for
`%cce@17` (the build did not fail, but the finished `zlib-ng%cce@17`
install did not have shared libs)
* suite-sparse library path can be lib or lib64
* fixing style check error
* Adding the missing import os
* Style check error fix
* Updating patch for 6.1.0 with correct lib path
* Preserve higher weight for CLI git ref versions
Currently the concretizer fails if you reuse a git ref version
that has already been installed but modify the spec at all.
See #38484 for futher diagnosis
The issue here is that since there is no established provenance for
these versions the highest weight they are currently assigned is
that of prior install. Re-use checks then fail because the weight of
the version is identical to the solver.
Ironically, these versions are given the highest weights possible when
specified on the CLI for the first time. They should only appear in a
DAG if they are an exact match or if the user specifies them at the CLI.
Therefore it makes sense to preserve their higher ordering.
Getting this right is critical to moving all branch based versions to a pinned
git-ref in the future.
* [@spackbot] updating style on behalf of psakievich
* Update lib/spack/spack/solver/asp.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Add provenance specific to git ref installs
* Sensitvity to name that I could not track down
* Add regression test
* Adjust test
* Add prefer standard unit-test
* Style
* Add required mock
* Format and mark
* Make unit-test case reproduce CLI investigation
* Remove unnecessary mock package
* [@spackbot] updating style on behalf of psakievich
* Use already developed fixture
* Add zlib-ng to mocks again
* Remove accidental adds
* Remove maintainer
* [@spackbot] updating style on behalf of psakievich
* Rename test file
* [@spackbot] updating style on behalf of psakievich
* Remove unused imports
* Update tests
* [@spackbot] updating style on behalf of psakievich
* Style
* Update lib/spack/spack/test/concretize.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Update solver rule
* Duplicate installed rules for installed_git_version
* Revert "Duplicate installed rules for installed_git_version"
This reverts commit 17223fc8d1.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
* libdistributed: renamed test to check_test
* libpressio-rmetric: renamed test method
* libpressio-tools: renamed test method
* Renamed build-time test methods for: libpressio-opt, libpressio-tthresh, opened
---------
Co-authored-by: Jackson Peter Lawrence <lawrence31@borax5.llnl.gov>
* Provide instructions for building a developmental version of lcov.
* style fix
* style fix
* Promote single version git URL to the package level.
* Formatting fix.
* verilator: Some environment varibles are no longer required
* Revert depends_on for flex back to the deafult case
* Use decorator, when, for setup_run_environment insdtead of if inside function
* libtiff: add v4.6.0 and default disable opengl (#44545)
* libtiff: Fix typo in CMake key
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: Broader description of OpenGL variant
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: reformat using spack style black recommendation
* libtiff: couple opengl flag with autotools
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There hasn't been a release in almost a decade, and build failures with
GCC 14 were reported. I don't think it makes sense to patch it, since
the project moved over to `util-linux`, and Spack's default provider is
that package. Better to just get rid of it in the next Spack release.
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/pcre-8.45-33dfhul
[+] /opt/apps/spack/libpciaccess-0.17-2qdxdjo
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/qhull-2020.2-klc7ewb
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/alsa-lib-1.2.3.2-a7icjdy
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/yasm-1.3.0-v4etmmm
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/unzip-6.0-mqftjtp
[+] /opt/apps/spack/libjpeg-turbo-3.0.0-vjvivme
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/libpng-1.6.39-kiuku4y
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/swig-4.0.2-fortran-z3sbnze
[+] /opt/apps/spack/binutils-2.42-vkjcvfr
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/hdf5-1.14.3-sbuiw6q
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/ffmpeg-6.1.1-2vhrbda
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/freetype-2.13.2-in4taxi
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/hwloc-2.9.1-fhoz6al
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/lua-5.3.6-47356ia
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/elfutils-0.190-uswzaiw
[+] /opt/apps/spack/py-pytz-2023.3-ojdlhrd
[+] /opt/apps/spack/py-cycler-0.11.0-b7mjvv7
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/llvm-14.0.6-3nosumn
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-six-1.16.0-iv6iv3q
[+] /opt/apps/spack/py-pybind11-2.12.0-5rvupjv
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-pyparsing-3.1.2-fq6imlt
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/py-llvmlite-0.41.1-qom3l5h
[+] /opt/apps/spack/py-astropy-4.0.1.post1-xbojixg
[+] /opt/apps/spack/py-python-dateutil-2.8.2-kzdfskc
[+] /opt/apps/spack/py-numexpr-2.8.4-fc5xc5s
[+] /opt/apps/spack/py-h5py-3.11.0-y6drk3j
[+] /opt/apps/spack/py-tifffile-2023.8.30-oof4try
[+] /opt/apps/spack/py-pygments-2.16.1-stgrccl
[+] /opt/apps/spack/py-mdurl-0.1.2-nqk43ry
[+] /opt/apps/spack/py-scipy-1.13.0-vxjgfov
[+] /opt/apps/spack/py-contourpy-1.0.7-jg5lhss
[+] /opt/apps/spack/py-pillow-10.3.0-ijh2cju
[+] /opt/apps/spack/py-kiwisolver-1.4.5-vdh5sq5
[+] /opt/apps/spack/py-fonttools-4.39.4-x5ydbox
[+] /opt/apps/spack/py-bottleneck-1.3.7-ztsm4lg
[+] /opt/apps/spack/py-lazy-loader-0.4-k7hgvka
[+] /opt/apps/spack/py-numba-0.58.1-hzvrjwj
[+] /opt/apps/spack/py-markdown-it-py-3.0.0-l4p4qv5
[+] /opt/apps/spack/py-imageio-2.34.0-z5hu4yc
[+] /opt/apps/spack/py-matplotlib-3.8.4-azq2fzm
[+] /opt/apps/spack/py-pandas-1.5.3-p3gnh6t
[+] /opt/apps/spack/py-rich-13.4.2-okhgwwz
[+] /opt/apps/spack/py-networkx-3.1-b54br7r
[+] /opt/apps/spack/py-scikit-image-0.23.2-cvyzh3t
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
#40773 introduced python-venv, which improved build isolation and avoids issues with,
e.g., `ubuntu`'s system python modifying `sysconfig` to include a (very unwanted)
`local` directory within the default install layout.
This addresses a few cases where #40773 removed functionality, without harming the
default cases where we use `python-venv`.
Traditionally, *every* view with `python` in it was essentially a virtual environment,
because we would copy the `python` interpreter and `os.py` into every view when linking.
We now rely on `python-venv` to do that, but only when it's used (i.e. new builds) and
only for packages that have an `extends("python")` directive.
This again makes every view with `python` in it a virtual environment, but only
if we're not already using a package like `python-venv`. This uses a different
mechanism from before -- instead of using the `virtualenv` trick of copying `python`
into the prefix, we instead create a `pyvenv.cfg` like `venv` (the more modern way
to do it).
This fixes two things:
1. If you already had an environment before Spack `v0.22` that worked, it would
stop working without a reconcretize and rebuild in `v0.22`, because we no longer
copy the python interpreter on link. Adding `pyvenv.cfg` fixes this in a more
modern way, so old views will keep working.
2. If you have an env that only includes python packages that use `depends_on("python")`
instead of `extends("python")`, those packages will now be importable as before,
though they won't have the same level of build isolation you'd get with `extends`
and `python-venv`.
* views: avoid making client code deal with link functions
Users of views and ViewDescriptors shouldn't have to deal with link functions -- they
should just say what type of linking they want.
- [x] views take a link_type, not a link function
- [x] views work out the link function from the link type
- [x] view descriptors and commands now just tell the view what they want.
* python: simplify logic for avoiding pyvenv.cfg in copy views
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
In #30882, we made Spack ignore `-Werror` calls so that it could more easily build
projects that inject `-Werror` into their builds. We did this by translating them to
`-Wno-error` in the compiler wrapper. However, some compilers (like `nvhpc`) do not
support `-Wno-error`. We need to exclude them from this feature until they do.
- [x] make a property on `PackageBase` for `keep_werror` that knows not to use it for
`nvhpc`.
- [x] update property so that it keeps only the specific `-Werror=...` args for newer nvhpc's,
which support `-Wno-error` but not `-Wno-error=...`
---------
Co-authored-by: William Mou <william.mou1024@gmail.com>
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
When Spack concretizes environments, it prints every (newly concretized) root spec
individually with all of its dependencies. For most reasonably sized environments, this
is too much output. This is true for three commands:
* `spack concretize` when concretizing an environment with newly added specs
* `spack install` when installing an environment with newly added specs
* `spack spec` with no arguments in an environment
The output dates back to before we had unified environments or nicer spec traversal
routines, and we can improve it.
This PR makes environment concretization output analogous to what we do for regular
specs. Just like `spack spec` for a single spec, we show all root specs with no
indentation, so you can easily see the specs you explicitly requested. Dependencies are
shown:
1. With indentation according to their depth in a breadth-first traversal starting at
the roots;
2. Only once if they appear on paths from multiple roots
So, the default is now consistent with `spack spec` for one spec--it's `--cover=nodes`.
i.e., if there are 100 specs in your environment, you'll get 100 lines of output.
If you want to see more details, you can do that with `spack spec` using the arguments
you're already familiar with. For example, if you wanted to see dependency types and
*all* dependencies, you could use `spack spec -l --cover=edges`. Or you could add
deptypes and namespaces with, e.g. `spack spec -ltN`.
With no arguments in an environment, `spack spec` concretizes (if necessary) and shows
the concretized environment. If you run `spack concretize` *first*, inspecting the
environment repeatedly with `spack spec` will be fast, as everything is already in the
`spack.lock` file.
- [x] factor most logic of `Spec.tree()` out of `Spec` class into `spack.spec.tree()`,
which can take multiple specs as roots.
- [x] make `Spec.tree()` call `spack.spec.tree()`
- [x] `spack.environment.display_specs()` now uses `spack.spec.tree()`
- [x] Update `spack concretize`
- [x] Update `spack install`
- [x] Update `spack spec` to call `spack.spec.tree()` for environments.
- [x] Continue to output specs individually for `spack spec` when using
`--yaml` or `--json`
* Apply patch from upstream to allow vtk compilation with %gcc 12 and 14.
* Fixes#44331
* fix soec usage
* fix compiler version range
* Finalize version range, switch to .diff file
---------
Co-authored-by: Chrismarsh <Chrismarsh@users.noreply.github.com>
* Now need to explicitly depend on libfabric for Cray EX
* Ensure build uses the selected CUDA and ROCm versions
* Correct spelling on `configure --with-ldflags`
* Patch a defect regarding `configure --with-ldflags`
* Default to Spack's copy of GASNet-EX, which is newer than embedded
e2fsprogs: adding version 1.47.0 and fuse2fs variant
fuse-overlayfs: adding version 1.13
squashfuse: adding version 0.5.1 and 0.5.2
e2fsprogs: fixing audit errors
apptainer: expanding deps to cover more versions, fix binary_path logic.
apptainer: add go version deps, fix e2fsprogs patch sums, rework dep paths
fuse-overlayfs: tightening up the libfuse dep
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* util-macros: ensure url_for_version only used for older versions
* util-macros: use url.substitute_version after xz -> bz2
* util-macros: mv url_for_version down the file
`BuildcacheBootstrapper` uses `Spec.intersects` to match specs needed
for bootstrapping against the binary cache. The specs were not
sufficiently-detailed to prevent matching e.g. cached binaries for
Mac OS on Windows; this commit adds the platform to each requested
bootstrap spec to prevent that.
* initial commit for RoCm-6.1.1 release
* fix style issues
* update the version for rocmlir and rocm-cmake
* restrict the patch for 6.0.0 release
* fix build failure for hipsolver-6.1.1 release
* update the hipblaslt for rocm-6.1.1 release
* add the patch for hipsparselt for 6.1.1 rel
Remove support for `cray` as a separate platform.
Any platform previously detected as `cray` is now detected as `linux`.
Users who still need platform=cray have to stick to Spack 0.22
Fixes a bug in the concretizer where specs depending on a host
incompatible libc would be used. This bug triggers when nothing is
built.
In the case where everything is reused, there is no libc provider from
the perspective of the solver, there is only compatible_libc. This
commit ensures that we require a host compatible libc on any reused
spec, additionally to requiring compat with the chosen libc provider.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Patch to add hsa prefix/include path:wq
* Style check error fix
* The patch need to be applied to 2024.01.stable. ROCm 6.0 requires hpctoolkit 2024.01.1 or later
* Fixing style chec error
* Adding a py4j variant that requires Java via spack to avoid situations where a system doesn't have Java and py4j expects it
* Adding new versions of py-pyspark
* Adding a new variant to require java (via py4j) and clean up dependency handling
* Adding myself as a maintainer for py-pyspark and py-py4j
* Fix overlooked version bump in py4j
* Version bump to meet py-spark expectations
* Version bump to add latest compatibile version with pyspark
* Matching py-grpcio bump
* Adding variants and dependents for pyspark
* Adding runtime deps
* Changing default java requirement. I'm not sure this is the right call
* Changing py4j with java dependency handling
* Fix style
* Update package.py fix unnecessary f-strinh
* Make +java the default for both
* Fix nested deps
* Revert styles after default change
* Added new versions and GCC 14 compatbility conflicts
* Added new versions and compatibility conflicts for gcc 14
* Added new versions paired to arrow (for gcc14 compat)
* Update py-protobuf compiler conflict
* Update depends to match
See https://github.com/grpc/grpc/blob/master/src/python/grpcio_status/setup.py
* Updating dependencies and conflicts for py-googleapis-common-protos. Added new version to avoid future issues
* Remove upper bound version on py-protobuf and add default_args
* Adding new versions and updating dependencies back to versions 1.35.0
* Updating oldest numpy deps
* Fixing merge
* bit more cleaniness for var/spack/repos/builtin/packages/py-googleapis-common-protos/package.py
* Adding latest matching version of py-grpcio and py-grpcio-status
* Update package.py
https://github.com/spack/spack/pull/44263#discussion_r1612317943
* Update dependencies
* Adding additional versions for dependent packages. Deprecated two versions: 1.16 is old, built for python ~3.6, and does not build for 3.8. 1.52.0 was removed from pypi
* Revert py-grpcio-tools changes. Will include in separate PR
* Adding patches and constraints to get 1.48 to build as it's a dependency that is called out for some other packages
* Updating to account for yanked packages for dependencies
* Fix style
* Update sha256 for py-grpcio v0.16.0 to reflect change
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* py-astropy: Add version 6.1
Added build info for version 6.1 in py-astropy. Requires a new additional package, astropy-iers-data which has been included as py-astropy-iers-data to match with spack's general naming conventions.
Below is the output of the spack install showing successful build fro version 6.1 and the new py-astropy-iers-data package
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-pyerfa-2.0.0.1-kyfazhs
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/git-2.45.1-tuc5jnb
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
==> Installing py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj [52/55]
==> No binary for py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj found: installing from source
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj
Stage: 1.74s. Install: 0.93s. Post-install: 0.52s. Total: 3.36s
[+] /opt/apps/spack/py-astropy-iers-data-main-ukchsfz
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-qdhxchg
==> Installing py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo [55/55]
==> No binary for py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> Warning: Module file /opt/modulefiles/spack/Core/py-astropy/6.1.0.lua exists and will not be overwritten
==> py-astropy: Successfully installed py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo
Stage: 1.29s. Install: 1m 5.77s. Post-install: 0.60s. Total: 1m 7.94s
* Removed extra-whitespace
A blank line had white space, removed the white space
* Additional formatting changes for black
* Additional package updates
Based on previous recommendations updated py-astropy and py-astropy-iers-data packages. Also added a new version to py-pyerfa package to match the 6.1.0 dependencies better
Of importance in these updates, I did add pypi and version info to py-astropy-iers-data. Originally I had argued that this package updates quite frequently (on a weekly basis) and so it did not make sense to include pypi/versions and we should instead use the non-version git-repo structure based on the master branch. However, when I tried to build the package py-setuptools-scm errored out when trying to build the package:
/opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro/lib/python3.10/site-packages/setuptools_scm/git.py:163: UserWarning: "/tmp/root/spack-stage/spack-stage-py-astropy-iers-data-main-iw2mdzlukb37mkcbcozjjefjoefw2eyp/spack-src" is shallow and may cause errors
I believe this is due to the download file/stage directory not containing the version and instead including the branch. I changed the package to use versions instead and it worked just fine as shown below. In addition, when I had done some preliminary testing, the package installed fine using the non-version master branch. When I checked, that installation it used py-setuptools-scm@7.1 while in this installation run it used a much more recent 8.0.4 so it is possible that somewhere between scm7.1 and 8.0.04 something changed that caused this error to show up. Since the setuptools-scm package has something to do extracting package versions, I imagine it's some kind of mismatch issue
Output from build:
spack install py-astropy
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/git-2.45.1-zu6qkoc
[+] /opt/apps/spack/py-markupsafe-2.1.3-isgtki6
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
==> Installing py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a [49/57]
==> No binary for py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a found: installing from source
==> Fetching ac8a6fe91c.tar.gz
==> No patches needed for py-extension-helpers
==> py-extension-helpers: Executing phase: 'install'
==> py-extension-helpers: Successfully installed py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a
Stage: 0.37s. Install: 0.88s. Post-install: 0.52s. Total: 1.89s
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
==> Installing py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m [50/57]
==> No binary for py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m found: installing from source
==> Fetching 350b18f967.tar.gz
==> No patches needed for py-cython
==> py-cython: Executing phase: 'install'
==> py-cython: Successfully installed py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m
Stage: 1.07s. Install: 2m 49.80s. Post-install: 0.40s. Total: 2m 51.37s
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
[+] /opt/apps/spack/py-jinja2-3.1.2-wacpq7j
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro
==> Installing py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 [55/57]
==> No binary for py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 found: installing from source
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/7f/7fff3d3404ae8560533ac0e685db7acc02c4d8984faa4ac3d607096879fba2d1.tar.gz
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6
Stage: 0.07s. Install: 1.21s. Post-install: 0.39s. Total: 1.88s
[+] /opt/apps/spack/py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6
==> Installing py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp [56/57]
==> No binary for py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/pyerfa/pyerfa-2.0.1.1.tar.gz
==> No patches needed for py-pyerfa
==> py-pyerfa: Executing phase: 'install'
==> py-pyerfa: Successfully installed py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp
Stage: 0.93s. Install: 17.72s. Post-install: 0.33s. Total: 19.15s
[+] /opt/apps/spack/py-pyerfa-2.0.1.1-pkokp6u
==> Installing py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs [57/57]
==> No binary for py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> py-astropy: Successfully installed py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs
Stage: 1.16s. Install: 1m 3.34s. Post-install: 1.05s. Total: 1m 5.77s
[+] /opt/apps/spack/py-astropy-6.1.0-5brbkjn
* Updated to match with black formatting
Change the installer to take `([pkg], args)` in the constructor instead
of `[(pkg, args)]`. The reason is that certain arguments are global
settings, and the new API ensures that those arguments cannot be
different across different "build requests".
The `explicit` install arg is now a list of hashes, and the installer is
no longer responsible for determining what package is installed
explicitly. This way environment installs can simply pass the list of
environment roots, without them necessarily being explicit build
requests. For example an env with two roots [a, b], where b depends on
a, would not always cause spack install to mark b as explicit.
Notice that `overwrite` already took a list of hashes, this makes
`explicit` consistent.
`package.do_install(explicit=True)` continues to take a boolean.
* legion: add pip dependency for build
pip is needed for the build when the Legion Python binding is enabled
* legion: add variants for gc logging and system OpenMP use
This also removes the `cmake_cxx_flags` variable from `cmake_args()`
because that variable had no effect.
* [@spackbot] updating style on behalf of mpokorny
* legion: use spec.satisfies() in cmake_args()
e.g, replace use of '"+foo" in spec' with 'spec.satisfies("+foo")'
* legion: avoid configuring with multiple "-DLegion_REDOP_COMPLEX=ON" arguments
In the previous version, when both '+redop_complex' and '+bindings'
was set, two instances of "-DLegion_REDOP_COMPLEX=ON" arguments were
generated for cmake configuration.
* legion: fix value of "Legion_OUTPUT_LEVEL" for configuration
the previous version had no effect on setting the configuration value
---------
Co-authored-by: mpokorny <mpokorny@users.noreply.github.com>
The windows wrappers for basic functions like `os.symlink`,
`os.readlink` and `os.path.islink` in the `llnl.util.symlink` module
have bugs, and trigger more file system operations on non-windows than
they should.
This commit just binds `llnl.util.symlink.symlink = os.symlink` etc so
built-in functions are used on non-windows
`spack clean <spec>` will now resolve specs based on the active environment if one is active.
If an env is active but no matching spec is found, this will fall back on fully concretizing.
Before this PR, if Spack could see a possibility to reuse a spec that
doesn't match a strong preference, it would do so. After the PR, a
strong preference would take precedence.
`glibc` and `musl` provide a basic implementation of `iconv` (`iconv`,
`iconv_open`, `iconv_close`), but in practice the installation may be
missing the character encoding methods to make them usable. On Fedora
for example, users need to
```yum install glibc-gconv-extra```
to get the character encodings that `gettext` requires during configure,
namely EUC-JP. Users may not have permissions to install the missing
parts of glibc.
Since Spack can install `libiconv`, it is simpler to use that by
default.
* Spglib: add version 2.4.0
* dla-future-fortran: new package version 0.1.0
* [@spackbot] updating style on behalf of RMeli
* apply suggestion and add maintainer
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* [emacs] When installing on Mac OS, build and install the native
Emacs.app along with the standard executables.
* [emacs] Make the GUI build on Mac optional by adding "gui" variant
* Apply reviewer suggestion
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Add emacs version 29.3
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
avoid calling `spec.target` when None.
When an external compiler package has an `os` set but no `target` set, Spack
currently falls into a codepath that calls `spec.target` (which itself calls
`spec.architecture.target.Microarchitecture`) when `spec.architecture.target`
is None, throwing an error.
e.g.
```
packages:
gcc:
externals:
- spec: gcc@12.3.1 os=rhel7
prefix: /usr
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* glew: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-demos: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-glu: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* paraview: fix dependency on glew
* mesa: group dependency on when("+glx")
* Add missing dependency on libxml2
* paraview: remove the "osmesa" and "egl" variant
Instead, enforce consistency using the "gl" virtual that allows
only one provider.
* visit: remove osmesa variant
* Disable paraview in the aws-isc stacks
* data-vis-sdk: rework constrains to enforce front-ends
* e4s-power: remove redundant paraview
* Pipelines: update osmesa variants
* trilinos-catalyst-ioss-adapter: make gl a run dependency
Starting from 24.04, armpl-gcc will only have three packages files: dev, rpm, and macOS.
Only one version for gcc is provided, so the changes in the code reflect that the tar provided
for gcc is only one rather than many.
* Remove mesa18 and libosmesa
mesa18 was introduced in #19528 as a way to maintain the old
autotools build of mesa separate from the new meson build.
We could add a second build system to mesa, but since mesa18 has
been deprecated for a long time, we'll just remove it.
libosmesa was used to multiplex the gl provider between mesa18
and mesa, and is thus unecessary. Remove it to reduce complexity
in the graphical stack.
* Remove references to mesa18 and libosmesa
* vtk: rework dependency on gl and osmesa
* memsurfer: rework dependency on vtk
* visit: minimal fix to avoid having both osmesa and glx
* Use hdf5::hdf5 on Windows from Paraview CMake
This patch is already applied on VTK 9 or greater.
* Add comments stating that vtk and paraview patches are the same and should be modified in concert.
This fixes a bug occurring when two root specs need to select
old versions, and these versions have the same penalty in the
optimization. This sometimes caused an older version to be
preferred to a more recent one.
The issue was the omission of `PackageNode` in the optimization
tuple.
* rayleigh: new package
* Update var/spack/repos/builtin/packages/rayleigh/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* split edit into three methods
* add comments to clarify use of configure
* rayleigh: copyright year
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Use correct path separator in get_all_package_diffs for all platforms.
Ensures correct package change computation on Windows when pruning unchanged specs in Gitlab CI
* Update, fix serenity & serenity-libint
Version 1.6.1 of `serenity` is added, some dependencies are made more
explicit, some options are improved or fixed.
The url of `serenity-libint` is fixed. The old url is not reachable
anymore.
* Use upstream patch, modify cmake patch
* Update var/spack/repos/builtin/packages/serenity/package.py
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* heimdall: Astronomy software package. Requires dedisp and psrdada (included as part of this commit)
* Updated packages to align with Spack's style
Minor updates based on wdconinc's comments regarding Spack's style guide
* [@spackbot] updating style on behalf of aweaver1fandm
* Minor edits to fix copyright year and dedisp install
Fixed copyright year to be 2024 instead of 2023
Removed the overridden version of install and created a preinstall function to create the missing lib and inc directories, therefore allowing the default install to run
Here is output from the spack install of heimdall showing successful build with cuda. If necessary I do a spack clean and freshly install it
./spack install heimdall +cuda cuda_arch=86
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libiconv-1.17-enpmbhsi3kztebwmpclpub2afhlbr3gy
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/xz-5.4.1-pte76kujkezxb3laqse3o4sctlbygsaw
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zlib-1.2.13-utlfo5ltxz5v5bckirn5v3amtbxjdvwh
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/bzip2-1.0.8-x6navz7ucgfnb5xq7aelqmgd4zxsz5bs
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libmd-1.0.4-ncomhrodpdul4dm64o6b7426fhmc2u64
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/ncurses-6.4-c77h34rooycbzapxjvc27sg5td5jiwyb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zstd-1.5.5-eporpybumydxveg5rwtfzysrsu4eqzcv
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libffi-3.4.4-vskntokaojclfqxjfzkbyirkeogddbpx
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxcrypt-4.4.33-gtpn32p6mxztul3c3dxzqj7gvcyh555j
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/util-linux-uuid-2.38.1-g322a5peqjaad6gl5q64cdu4qo7kvw6o
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxml2-2.10.3-pza3kz2mtbncbbeim6rejfqkgftnf4rz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/openssl-1.1.1t-adbquvgg4qpc3vq6jynf44qzq3gfwrv5
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/pigz-2.7-e7mxj4ya2u4a6zb4hu64g7docujmkxeb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libbsd-0.11.7-wh4xleivbe7wndiqt5nsehzlfrccnjcg
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/readline-8.2-hhb647bwmbcj7iwpmtetbylninfm5rxf
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/cuda-11.7.1-osue2sx5rv7dgzhsmaemydpwhyribxng
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/tar-1.34-35f5gki2ycxmy5zd7zs5tsvp3xoszxum
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/expat-2.5.0-4gizyhhqklciuyrbyinq2tdggt73gds4
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gdbm-1.23-w3uzihtubj2iwv6es55fis6nt2q5zwlr
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/sqlite-3.40.1-m4ntzvuupsnbtkdfbz7oqpbjdlaffp2a
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/dedisp-1.0-pq6r3jnyxq6jzoygz3fp2e6jc2ojpvap
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gettext-0.21.1-hglzdeadmgkzjb76bmemt6dnulfkrpha
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/python-3.10.10-khu36qq4p2te7jf475ewr2h7egidekfl
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/psrdada-master-by4w6mrpcfnoihlhos2jcfo2roiyagaz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/heimdall-1.0-ohtdnltuhhejysshcert25h6nmuvluqp
* [@spackbot] updating style on behalf of aweaver1fandm
Restrict the hpcrun-fmt.txt patch to :2022. It's fixed in the code
after that, and in recent develop, some code paths have moved causing
this patch to fail.
This fixes an issue where ghcr, gitlab and possibly other container registries paginate tags by default, which violates the OCI spec v1.0, but is common practice (the spec was broken itself). After this commit, you can create build cache indices of > 100 specs on ghcr.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* pythia8: prefer 8.310
* [@spackbot] updating style on behalf of wdconinc
* pythia8: filter_file to remove sed n
* Revert "[@spackbot] updating style on behalf of wdconinc"
This reverts commit e2a3decaffbd3f464d1bd992025e1812df49f088.
* Revert "pythia8: prefer 8.310"
This reverts commit 568cb056b87129085e245d9dbef1732ee1c6c0aa.
* [@spackbot] updating style on behalf of wdconinc
* pythia8: comment for fix
* pythia8: fix style
* pythia8: filter_file with raw string because of escaped pipe
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
The release tarballs are not available anymore which means autoconf, automake and libtool are always needed.
The NVHPC specific patches don't make sense anymore being that the patched files are not distributed in the new tar files.
Add support for Gitlab CI on Windows
This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows
infrastructure).
* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Symlinks on Windows can use longpath prefixes (\\?\); these are fine
in the context of win32 API interactions but break numerous facets of
Spack behavior that rely on string parsing/matching (archiving,
binary distributions, tarball extraction, view regen, etc).
Spack's internal readlink method (llnl.util.symlink.readlink)
gracefully handles this by removing the prefix and otherwise behaving
exactly as os.readlink does, so we should prefer that in all cases.
* py-matplotlib: qualify when to do a post install
Older versions of py-matplotlib don't seem to have some of the
files that the post install step is trying to install.
Looks like the files first appeared in 3.6.0 and later.
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
* Change install paths for older matplotlib
---------
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add maintainer and fix linting
* allow for fewer deps in archive
* use meson for archive packages
* Fix version spec and f-string
* fix blas dependency links
* Add new release to spack
* Fix checksums for latest release
* Update version of iperf3 from 3.6 to 3.16
Spack currently only explicitly has version 3.6 of the iPerf3 package
(out of ESnet / LBNL). This makes the default the latest version of 3.16,
and adds some other versions (found in some Linux distros, for possible
compatibility purposes).
* iperf3: update to 3.17; update 3.6 hash for new url
* protobuf: update hash for patch needed when="@3.4:3.21"
* Revert "protobuf: update hash for patch needed when="@3.4:3.21""
This reverts commit 4d168d0b27.
* initial commit to enable tensorflow-2.14 for spack built ROCm
* fix style errors
* modify hipcc patch
* updates for rocm 6.1
* updates for tf-rocm-enhanced 2.16
* fix styling
* fix styling
* add patch for 2.16
* add patch for 2.16
* Update var/spack/repos/builtin/packages/py-tensorflow/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* update rocm enhanced version names
* changes for rocm-enhanced version name change
* fix styling
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix gromacs installation with SVE. Issue 44062
* [@spackbot] updating style on behalf of paolotricerri
* Remove `neoverse_n2` target
We have removed the neoverse_n2 target as its detection is more involved
compared to neoverse_v*.
Set MPI_C_COMPILER etc. env vars when building with fujitsu-mpi, which
override CMake logic. Without using these variables to explicitly
request the Fujitsu MPI wrappers, the builtin CMake logic is unwilling
to use these wrappers unless the Fujitsu compiler is used, but they
should be used for %clang as well.
This avoids patching CMake module files like in #16864, primarily to
avoid the possibility of altering behavior for specs that do not use
%fj or ^fujitsu-mpi.
* Add openfoam to aws-pcluster-neoverse_v1 stack
* Add more apps to aws-pcluster-x86_64_v4 stack
* Remove WRF while hdf5 cannot build in buildcache at the moment
* Update comment
* Add more apps for aws-pcluster-neoverse_v1 stack
* Remove apps that currently do not build
* Disable those packages that won't build
* Modify syntax such that correct cflags are used
* Changing syntax again to what works with other packages
* Fix overriding packages.yaml entry for gettext
* Use new `prefer` and `require:when` clauses to clarify intent
* Use newer spack version to install intel compiler
This removes the need for patches and makes sure the `prefer` directives in
`package.yaml` are understood.
* `prefer` not strong enough, need to set compilers
* Revert "Use newer spack version to install intel compiler"
This reverts commit ecb25a192c.
Cannot update the spack version to install intel compiler as this changes the
compiler hash but not the version. This leads to incompatible compiler paths. If
we update this spack version in the future make sure the compiler version also updates.
Tested-by: Stephen Sachs <stesachs@amazon.com>
* Remove `prefer` clause as it is not strong enough for our needs
This way we can safely go back to installing the intel compiler with an older
spack version.
* Prefer gcc or oneapi to build gettext
* Pin gettext version compatible with system glibc-headers
* relax gettext version requirement to enable later versions
* oneapi cannot build older gettext version
* octave: use pcre2 for @8:
* Add 'pcre2' variant to octave to control pcre vs. pcre2
* Update var/spack/repos/builtin/packages/octave/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alex Richert <alexander.richert@noaa.gov>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Apparently urllib can throw a range of different exceptions:
1. HTTPError
2. URLError with e.reason set to the actual exception
3. TimeoutError from getresponse, which is not wrapped
* New package: qpdf
* Update var/spack/repos/builtin/packages/qpdf/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Fix format of cmake_args
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* archive: relative links only
Ensure all links written into tarfiles generated from Spack prefixes do not contain symlinks pointing outside the prefix
* binary_distribution: limit extraction to prefix
Ensure files extracted from spackballs are not links pointing outside of the prefix
* Ensure rpaths are properly set on Windows
* hard error on extraction of absolute links
* refactor for non link-modifying approach
* Restore tarball extraction to original impl
* use custom readlink
* cleanup symlink module
* make lstrip
* flux-sched: set the version if the ver file is missing
problem: flux-sched needs a version, it normally gets this from a
release tarball or from git tags, but if using a source archive or a git
clone without tags the version is missing
solution: set the version through cmake based on the version spack sees
when the version file is missing
* Update var/spack/repos/builtin/packages/flux-sched/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Problem: the libsodium download endpoints are not reliable,
they fail multiple times a day for several of our automated
pipelines.
Solution: use the GitHub releases and branches.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
GCC C++ headers like cstdlib use `#include_next <stdlib.h>` to wrap libc
headers. We're using `-isystem` for libc, which puts those headers too
early in the search path. `-idirafter` fixes this so `include_next`
works.
* add darshan-3.4.5 packages, update URLs
* py-setuptools version switches for py-darshan
* more py-darshan test dependencies
* try a conditional importlib_resources dependency
Add the ability to include any number of (potentially nested) concrete environments, e.g.:
```yaml
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
or, from the CLI:
```console
$ spack env create myenv
$ spack -e myenv add python
$ spack -e myenv concretize
$ spack env create --include-concrete myenv included_env
```
The contents of included concrete environments' spack.lock files are
included in the environment's lock file at creation time. Any changes
to included concrete environments are only reflected after the environment
is re-concretized from the re-concretized included environments.
- [x] Concretize included envs
- [x] Save concrete specs in memory by hash
- [x] Add included envs to combined env's lock file
- [x] Add test
- [x] Update documentation
Co-authored-by: Kayla Butler <<butler59@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co
m>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Currently SPACK_COLOR=always is not respected in the build process on
macOS, because the global `_force_color` is re-evaluated in global scope
during module setup, where it is always `None`.
So, move global init bits from main.py to the module itself.
Some specs which were excluded from reuse,
are currently added back to the solve when
we traverse dependencies of other reusable
specs.
This fixes the issue by keeping track of what
we can explicitly reuse.
This commit adds a layer of indirection to improve build isolation with
and without external Python, as well as usability of environment views.
It adds `python-venv` as a dependency to all packages that `extends("python")`,
which has the following advantages:
1. Build isolation: only `PYTHONPATH` is considered in builds, not
user / system packages
2. Stable install layout: fixes the problem on Debian, RHEL and Fedora where
external / system python produces `bin/local` subdirs in Spack install prefixes.
3. Environment views are Python virtual environments (and if you add
`py-pip` things like `pip list` work)
Views work whether they're symlink, hardlink or copy type.
This commit additionally makes `spec["python"].command` return
`spec["python-venv"].command`. The rationale is that packages in repos we do
not own do not pass the underlying python to the build system, which could still
result in incorrectly computed install layouts.
Other attributes like `libs`, `headers` should be on `python` anyways and need no change.
Currently bootstrapping from source fails because clingo requires gnupg
requires clingo.
This commit stops eager bootstrapping. We don't need `patchelf` nor `gnupg`
generally. They're bootstrapped when needed.
This creates shared infrastructure for compiler packages to implement the
detailed search capabilities from the `spack compiler find` command for the
`spack external find` command.
After this commit, `spack compiler find` can be replaced with
`spack external find --tag compiler`, with the exception of mixed toolchains.
A named env cannot contain `.` and `/`.
So when a user runs `spack env create ./here` do not error but treat it
as `spack env create -d ./here`.
Also fix help string of `spack env create`, which seems to have been
copied from `activate` incorrectly.
Since reuse is the default now, `--reuse-deps` can be confusing, as it
technically does not imply roots are fresh.
So add `--fresh-roots`, which is also easier to discover when running
`spack concretize --fre<tab>`
* WIP NWChem with ARMCI-MPI and TCE optional
* rename armci-mpi to armcimpi
* add version for git
* add ARMCI-MPI support to NWChem package
EXTERNAL_ARMCI_PATH needs to be set to the ARMCI-MPI package install prefix.
I could not get it from spec["armcimpi"] but it worked to use the dependent build environment method to export a generic environmental variable to use instead.
* i suppose i can maintain NWChem as well
* check rejects option that dozens of packages use
i do not have time to fight with this nonsense
Run . share/spack/setup-env.sh
==> Error: armcimpi version 'master' has extra arguments: 'branch'
Valid arguments for a url fetcher are:
'url', 'sha256', 'md5', 'sha1', 'sha224', 'sha384', 'sha512', and 'checksum'
Error: Process completed with exit code 1.
* style
* ARMCI-MPI needs depends_on; the rest seems fixed
* fix ARMCI selection per review feedback from @zzzoom
https://github.com/spack/spack/pull/43883#discussion_r1585014147
* address reviewer feedback from @yizeyi18
https://github.com/spack/spack/pull/43883#discussion_r1587228084
elaborate on what +extratce does, in terms of the NWChem build environment variables,
some of which are documented on https://nwchemgit.github.io/TCE.html.
* style
* Update var/spack/repos/builtin/packages/nwchem/package.py
---------
Signed-off-by: Jeff Hammond <jeff.science@gmail.com>
Co-authored-by: Carlos Bederián <4043375+zzzoom@users.noreply.github.com>
* [TAU package] Update with +rocprofv2. Updated some tests.
pdated with +rocprofv2 flag. Only works with rocm-core >= 6.0.0
Can be tested with (Omnia):
spack install tau@master +rocm+rocprofv2 %gcc@11
Needs the last commit in our local repository, can be done by modifying the "git = " line, or waiting until the public one is updated.
In the case that tests cause issues when building TAU, there is the flag:
disable_tests = False
The rocm test is disabled by default, as the PR regarding tests loading dependencies is not solved (PR#43682).
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
* py-tensorflow: fix aarch64 build
* [@spackbot] updating style on behalf of aweits
* patch format
* change patch strategy, actually get the logic correct in
the patch
* only patch 2.16.1 onwards for aarch64
* __CUDACC__ not __NVCC__
* !(defined(__NVCC__) && defined(__CUDACC__))
---------
Co-authored-by: aweits <aweits@users.noreply.github.com>
We recently switched to using the new ReadTheDocs with "addons". That includes its own
analytics, which is nice, but we also want to continue using our GA4 analytics.
Adding GA4 is no longer supported by RTD, so we have to add it manually.
- [x] re-add the gtag to all pages, manually
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
PR #43701 is broken, preventing scorep from being installed. As explained in #43700 the issue is internal to scorep and can not be fixed in the package, at least by the method being attempted here.
Adds a pre-concretization check for the Windows SDK and WGL (Windows
GL) packages as non-buildable externals.
This is a redo of https://github.com/spack/spack/pull/43459, but makes
sure to modify the configuration scope outside of the bootstrap scope:
whichever is highest-precedence in the user's environment at the time
the concretization runs, which should either be an env scope or the
~ scope.
Adds pytest fixture mocking the check for WGL and WSDK as if they were
present.
This PR gives users finer control over which specs are reused during concretization.
The value of the `concretizer:reuse` config option now can take an object with the following properties:
- `roots`: true if reusing roots, false if reusing just dependencies
- `exclude`: list of constraints used to select reusable specs
- `include`: list of constraints used to select reusable specs
- `from`: allows to select the sources of reused specs
### Examples
#### Reuse only specs compiled with GCC
```yaml
concretizer:
reuse:
roots: true
include:
- "%gcc"
```
#### `openmpi` must be used from externals, and it must be the only external used
```yaml
concretizer:
reuse:
roots: true
from:
- type: local
exclude:
- "openmpi"
- type: buildcache
exclude:
- "openmpi"
- type: external
include:
- "openmpi"
```
With v2.10+, ADIOS added a campaign manager. This is auto-enabled
if SQLite3 is found.
Add explicit control for it now and disables it by default, to avoid
picking up system dependencies or bloating by default the ADIOS2
dependencies. Also, not yet fully mature to be used by default:
https://github.com/ornladios/ADIOS2/issues/4148
* pypi build of py-simpy
* [py-simpy] toml explicitly called out
* py-simpy: New version
* py-setuptools-scm: Added new version
* py-setuptools-scm: add url_for_version
Because versions @:7 have an underscore (setuptools_scm) in the URL, and
newer versions have a dash (setuptools-scm).
* py-setuptools-scm: fix flake8 line too long
* py-simpy: Fix hash
---------
Co-authored-by: Sid Pendelberry <sid@rit.edu>
Co-authored-by: Jen Herting <jen@herting.cc>
* PackageStillNeededError: add pkg that needs spec to exception msg
* PackageStillNeededError: f-string with short fmt and hash
* PackageStillNeededError: split long string
The old concretizer creates a cyclic graph when expanding virtuals for
`iconv`, which is a bug. This hack drops glibc and musl as possible
providers for `iconv` in the old concretizer to work around it.
* Added package to build Ollama
* Update package.py
Add license and documentation
* [@spackbot] updating style on behalf of teaguesterling
* We can now use OVERRIDE_GIT_DESCRIBE to set the version in DuckDB
* Update duckdb/package.py
- use f-string
- fix version specs to address inconsistencies
* Update package.py
Fix spec definitions further
* [@spackbot] updating style on behalf of teaguesterling
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The "DOTNET_CLI_TELEMETRY_OPTOUT" environment variable should be defined when using the product, not when installing it (the installation phase is just extract the files anyway).
Also use `~` instead of `-` to check for the variant and fix the second argument for `env.set` which should also be a string.
Add debug log for external detection tests. The debug log
is used to print which test is being executed.
Skip version audit on Windows where appropriate
* Update package.py to osg 1.119 and igtf 1.128
* Remove unnecessary white space
* Add missing comma
* change url to use git and use commit hash from repository instead of ssh256sum of tar.gz
* [@spackbot] updating style on behalf of rahmans1
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
* * Add initial tests-sos package
* Remove failing call of missing setup_compiler_environment from sos
package
* Add several variants for sos package
* [@spackbot] updating style on behalf of jack-morrison
We run `extend spack_flags_list SPACK_LDFLAGS` for `$mode in ld|ccld`.
That's problematic, cause `ccld` needs `-Wl,--flag` whereas `ld` needs
`--flag` directly. Only `-L` and `-l` are common to compiler & linker.
In all build systems `LDFLAGS` is for the compiler not the linker, cause
any linker flag `-x` can be passed as a compiler flag `-Wl,-x`, and there
are many compiler flags that affect the linker invocation, like `-fopenmp`,
`-fuse-ld=`, `-fsanitize=` etc.
So don't pass `LDFLAGS` to the linker directly.
This way users can set `ldflags: -Wl,--allow-shlib-undefined` in compilers.yaml
to work around an issue where the linker tries to resolve the `libcuda.so.1`
stub lib which cannot be located by design in `cuda`.
* Updates for rocm
Updated for rocm@6
Added conflict between rocprofiler and roctracer.
Request either +rocprofiler or +roctracer when +rocm. In this case, it automatically builds for one, instead of displaying the message.
Request +rocm when using either +rocprofiler or +roctracer. In this case, it automatically builds with +rocm, instead of displaying the message.
Disabled the tests. Will update them with the new test method.
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
Some packages can't be redistributed in source or binary form. We need an explicit way to say that in a package.
This adds a `redistribute()` directive so that package authors can write, e.g.:
```python
redistribute(source=False, binary=False)
```
You can also do this conditionally with `when=`, as with other directives, e.g.:
```python
# 12.0 and higher are proprietary
redistribute(source=False, binary=False, when="@12.0:")
# can't redistribute when we depend on some proprietary dependency
redistribute(source=False, binary=False, when="^proprietary-dependency")
```
To prevent Spack from adding either their sources or binaries to public mirrors and build caches. You can still unconditionally add things *if* you run either:
* `spack mirror create --private`
* `spack buildcache push --private`
But the default behavior for build caches is not to include non-redistributable packages in either mirrors or build caches. We have previously done this manually for our public buildcache, but with this we can start maintaining redistributability directly in packages.
Caveats: currently the default for `redistribute()` is `True` for both `source` and `binary`, and you can only set either of them to `False` via this directive.
- [x] add `redistribute()` directive
- [x] add `redistribute_source` and `redistribute_binary` class methods to `PackageBase`
- [x] add `--private` option to `spack mirror`
- [x] add `--private` option to `spack buildcache push`
- [x] test exclusion of packages from source mirror (both as a root and as a dependency)
- [x] test exclusion of packages from binary mirror (both as a root and as a dependency)
If there's no compiler we currently don't have any external libc for the solver.
This commit adds a fallback on libc from the current Python process, which works if it is dynamically linked.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
When looking at where we spend our time in solver setup, I noticed a fair bit of time is spent
in `Spec.format()`, and `Spec.format()` is a pretty old, slow, convoluted method.
This PR does a number of things:
- [x] Consolidate most of what was being done manually with a character loop and several
regexes into a single regex.
- [x] Precompile regexes where we keep them
- [x] Remove the `transform=` argument to `Spec.format()` which was only used in one
place in the code (modules) to uppercase env var names, but added a lot of complexity
- [x] Avoid escaping and colorizing specs unless necessary
- [x] Refactor a lot of the colorization logic to avoid unnecessary object construction
- [x] Add type hints and remove some spots in the code where we were using nonexistent
arguments to `format()`.
- [x] Add trivial cases to `__str__` in `VariantMap` and `VersionList` to avoid sorting
- [x] Avoid calling `isinstance()` in the main loop of `Spec.format()`
- [x] Don't bother constructing a `string` representation for the result of `_prev_version`
as it is only used for comparisons.
In my timings (on all the specs formatted in a solve of `hdf5`), this is over 2.67x faster than the
original `format()`, and it seems to reduce setup time by around a second (for `hdf5`).
The reverse provider lookup may have stale entries for deleted packages, which used to cause errors. It's hard to invalidate those cache entries, so this commit simply drops entries w/o invalidating the cache.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This commit differentiate linux from other platforms by
using libc compatibility as a criterion for deciding
which buildcaches / binaries can be reused. Other
platforms still use OS compatibility.
On linux a libc is injected by all compilers as an implicit
external, and the compatibility criterion is that a libc is
compatible with all other libcs with the same name and a
version that is lesser or equal.
Some concretization unit tests use libc when run on linux.
Some logic to detect what libc the c / cxx compilers use by default,
based on `-dynamic-linker`.
The function `compiler.default_libc()` returns a `Spec` of the form
`glibc@x.y` or `musl@x.y` with the `external_path` property set.
The idea is this can be injected as a dependency.
If we can't run the dynamic linker directly, fall back to `ldd` relative
to the prefix computed from `ld.so.`
In the future we may transform the database from a single JSON object to
a stream of JSON objects.
This paves the way for constant time writes and constant time rereads
when only O(1) changes are made. Currently both are linear time.
This commit gives just enough forward compat for Spack to produce a
friendly error when we would move to a stream of json objects, and a db
would look like this:
```json
{"database": {"version": "<something newer>"}}
```
* compiler wrapper: prioritize spack managed paths in search order
This commit partitions search paths of -L, -I (and -rpath) into three
groups, from highest priority to lowest:
1. Spack managed directories: these include absolute paths such as
stores and the stage dir, as well as all relative paths since they
are relative to a Spack owned dir
2. Non-system dirs: these are for externals that live in non-system
locations
3. System dirs: your typical `/usr/lib` etc.
It's very easy for Spack to known the prefixes it owns, it's much more
difficult to tell system dirs from non-system dirs. Before this commit
Spack tried to distinguish only system and non-system dirs, and failed
for very trivial cases like `/usr/lib/x/..` which comes up often, since
build systems sometimes copy search paths from `gcc -print-search-dirs`.
Potentially this implementation is even faster than the current state of
things, since a loop over paths is replaced with an eval'ed `case ...`.
* Trigger a pipeline
* Revert "Trigger a pipeline"
This reverts commit 5d7fa863de.
* remove redudant return statement
* Add conflicts for some blas implementations that don't build on
Windows (or with %msvc)
* Need to enclose CC/CXX variables in quotes in case those paths
have spaces, otherwise Meson runs into errors
* On Windows, Python dependencies now add <prefix>/Scripts to the
PATH (this is established as a standard in PEP 370)
* Later versions of oneAPI have moved, so update detection to find it
in both old and new location
* Remove reliance on ONEAPI_ROOT env variable when determining Fortran
compiler version for %msvc
* When finding a Fortran compiler for MSVC, there was logic enforcing
a maximum MSVC version for a given oneAPI Fortran version. This
mapping was out of date and excluding valid combinations, so has
been removed (the logic now just picks the latest available
oneAPI Fortran compiler for any given MSVC version).
On Windows, bootstrapping logic now searches for and adds the win-sdk
and wgl packages to the user's top scope as externals if they are not
present.
These packages are generally required to install most packages with
Spack on Windows, and are only available as externals, so it is
assumed that doing this automatically would be useful and avoid
a mandatory manual step for each new Spack instance.
Note this is the first case of bootstrapping logic modifying
configuration other than the bootstrap configuration.
* gxsview: compiles againts system qt and vtk on rhel8
* Update gxsview/package.py for blanks around operator
* Update gxsview/package.py import blank line
* Update gxsview/package.py for style
* Update gxsview/package.py checking vtk version
Score-P does not accept "--with-foo=yes", but only "--with-foo" or "--with-foo=some-valid-specific-choice-or-path". This keeps Spack from generating config flags that will cause Score-P to barf.
This adds some improvements to `spack find` output when in environments based
around some thoughts about what users want to know when they're in an env.
If you're working in an enviroment, you mostly care about:
* What are the roots
* Which ones are installed / not installed
* What's been added that still needs to be concretized
So, this PR adds a couple tweaks to display that information more clearly:
- [x] We now display install status next to every root. You can easily see
which are installed and which aren't.
- [x] When you run `spack find -l` in an env, the roots now show their concrete
hash (if they've been concretized). They previously would show `-------`
(b/c the root spec itself is abstract), but showing the concretized root's
hash is a lot more useful.
- [x] Newly added/unconcretized specs still show `-------`, which now makes more
sense, b/c they are not concretized.
- [x] There is a new option, `-r` / `--only-roots` to *only* show env roots if
you don't want to look at all the installed specs.
- [x] Roots in the installed spec list are now highlighted as bold. This is
actually an old feature from the first env implementation , but various
refactors had disabled it inadvertently.
Reduce incidence of spurious errors by:
* Ensuring we're passing the buffer by reference
* Get the correct short string size from Windows API instead of computing ourselves
* Ensure sufficient space for null terminator character
Add test for `windows_sfn`
Currently if you request pkg +example where example is a conditional
variant, and you have a pkg in the database for which the condition
did not hold (so no +example nor ~example), the solver would reuse it
regardless, not imposing +example.
The change rules out exactly one thing: variant_set without variant_value,
which in practice could only happen when not node_has_variant (i.e. when
under the current package.py rules the variant's when condition did not
trigger).
Add a "default" option that passes no option to configure. Existing
options changed in the MPICH 4.2.0 release, so update the package to
reflect those changes.
Currently, some of the tests in `spec_format` and `spec_semantics` fetch
the actual zlib repository when run, because they call `str()` on specs
like `zlib@foo/bar`, which at least currently requires a remote git clone
to resolve.
This doesn't change the behavior of git versions, but it uses our mock git
repo infrastructure and clones the `git-test` package instead of the *real*
URL from the mock `zlib` package.
This should speed up tests. We could probably refactor more so that the git
tests *all* use such a fixture, but the `checks` field that unfortunately
tightly couples the mock git repository and the `git_fetch` tests complicates
this. We could also consider *not* making `str()` resolve git versions, but
I did not dig into that here.
- [x] add a mock_git_test_package fixture that sets up a mock git repo *and*
monkeypatches the `git-test` package (like our git test packages do)
- [x] use fixture in `test_spec_format_path`
- [x] use fixture in `test_spec_format_path_posix`
- [x] use fixture in `test_spec_format_path_windows`
- [x] use fixture in `test_parse_single_spec`
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
