Currently, a few packages using kokkos rely on
kokkos itself monkeypatching its own spec to
provide some attribute.
In this commit we change this attribute to be
defined on the package, and never be monkeypatched.
Currently, when we setup the ASP problem for `clingo`, we don't take into account the configuration. This results in setting up ASP problems that are larger than necessary, with possibly redundant information, and higher concretization times.
This PR tries to improve things by adding an opt-in feature that computes the _possible dependencies_ of a solve taking also into account the current configuration, and avoids adding possible dependencies that we are certain can't be in the final solution.
The feature can be activated with:
```yaml
concretizer:
static_analysis: true
```
Examples of simple rules to discard dependencies are:
- Dependencies that are not buildable, and for which no binary is present (e.g. `cray-mpich` etc. on non Cray systems)
- Dependencies that are not for the current platform (e.g. `msmpi` on non Windows platforms)
- Conditional dependencies that cannot be activated, because of some user requirement (e.g. `cuda` etc. if the user requires `~cuda` in configuration)
- Virtual providers that cannot be used, because of a requirement on a virtual
The speed-up these rules seem to give depends on the use case at hand, but if the configuration is updated properly, they are noticeable.
Since in cases where there is no rule to exclude packages upfront, reuse is active, and this option is activated, it's possible to see some minor slow down, the feature has been added as opt-in, so it's turned off by default.
* Bump up the version for rocm-6.3.2 release
* rocm-openmp-extras update and style correction
* Updating mivisionx, omniperf, rccl & rocprofiler-systems
* Updating hipsparselt & rocm-opencl
* rocprofiler-systems on gcc-13 and rvs commit instead of patch
* Updated rocjpeg & rocm-examples for 6.3.2
* ROCPROFSYS_BUILD_DYNINST & DYNINST_BUILD_TBB are required only with gcc-13
---------
Co-authored-by: afzpatel <122491982+afzpatel@users.noreply.github.com>
Current location is within the Spack prefix, which causes builds to
pollute VCS with stage artifacts and generally inflates the Spack
install prefix.
This PR moves it to the user cache location now that we can
consistently support paths with spaces on Windows.
With CFLAGS, the code path in the amdfftw build system will bypass the logic around AMD_ARCH.
---------
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
* salt: Add v0.3.0 of SALT
This version contains important bug fixes for building and parsing
projects containing Fortran
* salt: Be more explicit about dependency types
- llvm+clang+flang is needed at build, link and runtime for the
correct operation of SALT
- Testing with llvm@master ( llvm > 19.x ) shows that SALT is
currently incompatible with the latest llvm API so an updated salt
will be required when LLVM 20 is released
`relocate_links` warns when the target is absolute and not matched by
any prefix from the prefix to prefix map.
This can lead to false positives, cause the prefix to prefix map does
not contain trivial/identity entries whenever a package is installed to
its original location.
Since relocate_links is the odd one out there (we don't warn about
similar issues with rpaths, etc), just remove the warning.
Codecov needs to see the token secret when uploading, so we have to
add this line to the workflow YAML:
```yaml
with:
token: ${{ secrets.CODECOV_TOKEN }}
```
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Remove variable from cmake.py
#48775 left a dangling variable that was not caught in CI but by the eyes of @haampie. Restructure variable to local method.
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
* Update lib/spack/spack/build_systems/cmake.py
* Update lib/spack/spack/build_systems/cmake.py
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword
* Revert "Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword"
This reverts commit faf9b8ce85.
* Apply workaround for oneAPI compiler
* style problem resolved
* use spec.satisfies syntax
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Add ruff configuration to `pyproject.toml`.
This allows `ruff format` in the Spack repository to format all the files we care about,
with our line length of 99, the exceptions we already put in place, and excluding things
we don't auto-format, like vendored dependencies.
Right now it'll reformat 175 or so files, but only slightly, in places where `ruff` differs from
`black`. For the most part I like the ruff format decisions better than `black`, but none of
the changes seem too severe.
This does not change `spack style` -- I figure that can come later but this at least will
let people start playing with `ruff`.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
These are some changes that `ruff check --fix` would make that the current
`spack style` also agrees with. Make the changes now so that the `ruff`
change is less disruptive.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
The import-check action now presents problematic import statements
introduced by the PR better.
The idea is roughly:
* Let (V₁, E₁) be the graph of modules as vertices and import statements
as edges before the change
* Let (V₂, E₂) be the graph after the code change, which is typically a small
perturbation of (V₁, E₁).
* X₁ = FAS(V₁, E₁) is the feedback arc set before (a minimal set of edges to
delete to make it acyclic)
* X₂ = FAS(V₂, E₂ ∖ X₁) is the feedback arc set after deletion of the minimal
set of edges that made the old graph acyclic.
* X₃ = FAS(V₂, E₂) is the feedback arc set after
Previously I displayed X₁ and X₃ and users had to diff themselves.
Now, I'm showing X₂, which is a small set, typically directly related to
code changes.
However, it can be that a small code change adding say 2 problematic imports
creates a completely different solution X₃ that only requires deletion of just 1
different import. In that case the user is informed that they can potentially do
less work.
So for PR #48784 the output is now:
> The overall number of problematic import statements increased by 1 from 31 to 32.
> This is likely a direct consequence of the following import statements:
>
> ```
> spack/config imports: spack.spec, spack.util.path, spack.util.remote_file_cache
> ```
>
> However, instead of removing 3 import statements, it is sufficient to remove only 1
> import statement from the following list:
>
> ```
> spack/concretize imports: spack.bootstrap, spack.solver.asp
> spack/environment imports: spack.bootstrap, spack.environment
> spack/fetch_strategy imports: spack.version.git_ref_lookup
> spack/install_test imports: spack.build_environment, spack.package_base
> spack/modules imports: spack.modules
> spack/platforms imports: spack.config
> spack/relocate imports: spack.bootstrap
> spack/repo imports: spack.package_base, spack.patch, spack.tag
> spack/spec imports: spack.binary_distribution, spack.compiler, spack.compilers, spack.concretize, spack.environment, spack.hash_types, spack.provider_index, spack.repo, spack.spec_parser, spack.store, spack.traverse, spack.variant, spack.version.git_ref_lookup
> spack/subprocess_context imports: spack.environment
> spack/util/gpg imports: spack.bootstrap
> spack/util/package_hash imports: spack.package_base
> spack/util/path imports: spack.config, spack.environment
> spack/util/remote_file_cache imports: spack.util.web
> ```
from which the user can figure out that
`spack/util/remote_file_cache imports: spack.util.web` is the "bottleneck" now.
* Add versions 2 and 3 of py-sphinx-rtd-theme.
Allow for versions of py-sphinx greater than 6.
Fix the Python version for older versions that depend on distutils.
Get the py-docutils dependency from the py-sphinx recipe.
* Depend purely on the py-docutils dependency in py-sphinx.
* More refined dependency versioning.
* Fixed versioning for py-sphinx and py-docutils.
Currently, environments created from manifest files with relative includes result in broken
references to config files.
This PR modifies `spack env create` to create local copies in the new environment of any local
config files from relative paths in the environment manifest passed as an init file.
This PR does not change the behavior if the include is an absolute path or if the include is from
a relative path outside the environment directory, but it does warn about missing relative includes if
they are inside the environment directory.
Includes regression test and short blurb in docs.
* Added salt variant to tau
* Update package.py
* [@spackbot] updating style on behalf of wspear
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* Create SALT package.py
Added a package for the SALT Source AnaLysis Toolkit
@zbeekman
* [@spackbot] updating style on behalf of wspear
* Update package.py
Line wrap
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* py-mdit-py-plugins: Add new versions 0.3.5, 0.4.2
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* py-myst-parser: Add new versions 0.19.0 to 4.0.0
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* hpctoolkit: Add +docs variant and manpages
This commit unconditionally enables manpages for the HPCToolkit tools.
The new `+docs` variant enables additional documentation, specifically
the user's manual. Both require new build-time dependencies.
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
---------
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* package api: drop wildcard re-export
To ensure package repos are forward/backward compatibility with Spack,
we should explicitly export all symbols we want to expose in the public
package API, and drop `from spack.something import *` because
removal/addition to the public API will go unnoticed.
Also `llnl.util.filesystem` has some methods that shouldn't be exposed
in the package API, so better to enumerate a subset explicitly.
* remove flatten_dependencies / install_dependency_symlinks
* JAX: add v0.4.34
* Disable search for clang
* Update CUDA flags
* Add py-jax 0.4.33, comment out until py-jaxlib 0.4.33 is also released
* Fix GCC build
* Try TF_NVCC_CLANG
* py-jax: add v0.4.34
* jax no longer has separate tags for jaxlib
* Install compiled wheel
* Join path before glob
* Wheel is in spack stage, not tmp path
* Add 0.4.35
* Add newer versions
* Build system has been refactored yet again
* Drop clang
* Fix build with source tarball, rocm support
* Support GCC
* Remove clang-specific compiler flags
* enable_cuda flag was removed
* Fix logic
* py-jax: add v0.4.38
* Add patch to fix GCC support
* Patch no longer needed
* Skip patching, directly pass flags
* New flags
* Remove unused import
* Patch changed
* Use older version of patch
* Newer patch
* Add CUDA symlink
* Symlink more directories
* Recursive symlink
* Import function
* Recursive search
* Undo cuda changes
* Add v0.5.0
* I quit
* Improve definition of a few placeholder packages
These packages are placeholders for vendor provided software,
that is not buildable, and should be declared as external.
* ibm-java: remove package, as asked by maintainer
Regressed in #47126
Spack was not interpreting mirrors using relative path with respect to the
metadata directory.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* test_no_matching_compiler_specs: does not need mock_low_high_config,
since mutable_config is already used at class level
* bindist.py: setup a configuration that doesn't super-impose builtin.mock
over builtin
* builder.py: use a mock configuration for the tests
This commit adds version 6.8.0 of GeoModel. As far as I can tell from
the change notes, there are no changes required to the build
configuration or dependencies.
This adds a new configuration section called `env_vars:` that can be set in an environment.
It looks very similar to the existing `environment:` section that can be added to `modules.yaml`,
but it is global for an entire spack environment. It's called `env_vars:` to deconflate it with spack
environments (the term was too overloaded).
The syntax looks like this:
```yaml
spack:
specs:
- cmake%gcc
env_vars:
set:
ENVAR_SET_IN_ENV_LOAD: "True"
```
Any of our standard environment modifications can be added to the `env_vars` section, e.g.
`prepend_path:`, `unset:`, `append_path:`, etc. Operations in `env_vars:` are performed
on `spack env activate` and undone on `spack env deactivate`.
* Reduce the size of outputted go built binaries
* Remove unused import from go package
* go: remove comment from setup dependents build env
* Add back missing imports after rebase
* Backward compat with Python 3.9 for socket.timeout
* Forward compat with Python [unknown] as HTTPResponse.geturl is deprecated
* Catch timeout etc from .read()
* Some minor simplifications: json.load(...) takes file object in binary mode.
* Fix CDash code which does error handling wrong: non-2XX responses raise.
The conflict is too strict and prevents usable combinations from being
concretized. Some packages depend on pika, but do not include pika
headers in HIP device code files, and can thus be compiled even with the
bad combination of HIP and GCC versions.
* py-awkward: add dependency on py-importlib-metadata and py-fsspec
* Implement suggested constraint and dependency type changes
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
On macOS you cannot unconditionally pass `-rpath`, it can conflict with
`-r`.
mpicc is not the place to inject rpaths to compiler runtime
libraries. That's up to the compiler's config files (spec files in gcc,
config files in llvm).
Also remove some patches that are redundant in newer versions of
openmpi.
Should be sufficient to set CONFIG_SITE to /dev/null before running configure to
prevent any config site file to be loaded, which currently causes non-determinism in builds.
Setting the variable CONFIG_SITE prevents configure files to check $prefix/share/config.site,
$prefix/etc/config.site, $ac_default_prefix/share/config.site, $ac_default_prefix/etc/config.site
so we don't have to patch a block of code.
* Address quoting issue that casuses dev_paths containing @ symbols to parse as versions
* Fix additional location were dev_path was imporperly constructed
The dev_path must be quoted to avoid parsing issues when
paths contain '@' symbols. Also add tests to catch
regression of this behavior
* Format to fix line length
* fix failing tests
* Fix whitespace error
* Add binary_compatibility fixture to test
* Change string formatting to avoid multiline f string
* Update lib/spack/spack/test/concretization/core.py
* Add tmate debug session
* Revert "Add tmate debug session"
This reverts commit 24e2f77e3c.
* Move test and refactor to use env methods
* Update lib/spack/spack/test/cmd/develop.py
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
Since macOS 15 `ld -single_module` warns with a deprecation message,
which makes configure scripts believe the flag is unsupported. That
in turn triggers a code path where `archive_cmds` is set to
```
$CC -r -keep_private_externs -nostdlib ... -dynamiclib
```
instead of just
```
$CC -dynamiclib ...
```
This code path was meant to trigger only on ancient macOS <= 14.4 where
libtool had to add `-single_module`, which is the default since macos
14.4, and is now apparently deprecated because the flag is a no-op for
more than 15 years.
The wrong `archive_cmds` causes actual problems combined with a bug in
OpenMPI's compiler wrapper (`CC=mpicc`), which appends `-rpath` flags,
which cause an error when combined with the `-r` flag added by the
autotools.
Spack's compiler wrapper doesn't do this, but it's likely there are
other compiler wrappers out there that are not aware that `-r` and
`-rpath` cannot be combined.
The fix is to change defaults: `lt_cv_apple_cc_single_mod=yes`.
* BF+ENH: Add wxwidgets 3.2.6 and py-wxpython 4.2.2
Improves compat with newer Python (>3.9) and numpy.
Fix error during configure step (even on at least some older
versions) by including 'pkgconfig' as a build dep so gtk+ is found.
* BF: Make wxpython use spack built wxwidgets instead of rebuilding
* Update var/spack/repos/builtin/packages/py-wxpython/package.py
Avoid using too new version of py-setuptools as license file format is currently not compatible.
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* for config: let `syaml_dict` inherit from `dict` instead of `OrderedDict`. `syaml_dict` now only exists as a mutable wrapper for yaml related metadata.
* for spec serialization / hashing: use `dict` directly
This is possible since we only support cpython 3.6+ in which dicts are ordered.
This improves performance of hash computation a bit. For a larger spec I'm getting 9.22ms instead of 11.9ms, so 22.5% reduction in runtime.
* OSError.errno apparently is int | None, but already part of __str__ anyway so drop
* fix disambiguate_spec_from_hashes
* List[Spec].sort() complains, ignore it
* fix typing of KnownCompiler, and use it in asp.py
Adds packages for the Arkouda server (`arkouda`) and Arkouda python client (`py-arkouda`).
Arkouda server and client are divided into separate packages to allow for them to be
installed independently of one another.
Future work remains to add a `+dev` variant to `py-arkouda`, but that will require additional
supporting packages made available through spack (e.g. for `py-pytest-env`
Modifications:
* Fix a severe bug where a spliced spec has the same hash as the original build spec
* Removed CacheManager in favor of install_mockery. The latter sets up a temporary store, and removes it at the end
of the test.
* Deleted a couple of helper functions e.g. _has_dependencies
* Checked that SolverException is raised, rather than Exception (more specific)
* Extended, and renamed the splicing_setup fixture (now called install_specs)
* Added more specific assertions in each test
One test is currently flaky, due to some instability when sorting multiple specs. It's currently marked xfail
The methods spack.spec.Spec.concretize and spack.spec.Spec.concretized
are deprecated in favor of spack.concretize.concretize_one.
This will resolve a circular dependency between the spack.spec and
spack.concretize in the next Spack release.
* add python bindings variant to chapel
* fix chpl_home, update chapel frontend rpath in venv
* fix chpl frontend shared lib rpath hack
* patch chapel env var line length limit
* patch chapel python shared lib location by chapel version
* unhack chpl frontend lib rpath
* fix postinstall main version number file path
* update chapel version number to 2.3
* use chapel releases instead of source tarballs, remove dead code
* Chapel 2.3 adds support for LLVM 19
* Bundled LLVM always requires CMake 3.20:
* Apply 2.3 patch for LLVM include search path
Fixes build errors for `chapel@2.3+rocm` of the form:
```
compiler/llvm/llvmDebug.cpp:174:9: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
compiler/llvm/llvmDebug.cpp:254:15: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
```
* fix style
* Fix misreporting of test_hello failures
`test_part` was aliasing the enclosing test name, leading to confusing and incorrect reporting on test failure.
* Ensure `libxml2` optional dep of `hwloc` is also added to `PKG_CONFIG_PATH` in the run env
* patch chplCheck for GPU + multilocale configs, install docs
* Adjust patch for checkChplInstall
* Ensure `CHPL_DEVELOPER` is unset for `~developer`
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
* Improve support using external modules with zsh
Spack integrates with external modules by launching a python subprocess
to scrape the path from the module command. In zsh, subshells do not
inheret functions defined in the parent shell (only environment
variables). This breaks the module function in module_cmd.py.
As a workaround, source the module commands using $MODULESHOME prior to
running the module command.
* Fix formatting
* Fix flake error
* Add guard around sourcing module file
* Add improved unit testing to module src command
* Correct style
* Another attempt at style
* formatting again
* formatting again
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
* bdsim: update to point to the new location in github and add 1.7.7
* Remove the C++ standard patch
* Remove the cmake_args and add a comment instead
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
`spack find` and other commands that read the DB will fail if the databse is in a
read-only location, because `FailureTracker` and `Database` can try to create their
parent directories when they are constructed, even if the DB root is read-only.
Instead, we should only write to the filesystem when needed.
- [x] don't create parent directories in constructors
- [x] have write operations create parents if needed
- [x] in tests, `chmod` databases to be read-only unless absolutely needed.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Adding the PACE library to the LAMMPS build
* Adding the PACE library for ML-PACE in LAMMPS
---------
Co-authored-by: Richard Berger <rberger@lanl.gov>
* Add py-protobuf@4.24.4 (needed for py-cylc-flow@8.3.6).
* Add py-cylc-flow@8.3.6 and enable png output when creating graphs
by requesting variant pangocairo for graphviz dependency.
* Add corresponding versions of py-metomi-rose@2.3.2,
py-cylc-rose@1.4.2, py-cylc-uiserver@1.4.2.
* Add myself as maintainer to all the cylc-related packages.
This PR allows using the subscript notation directly on packages. The
intent is to reduce the boilerplate needed to retrieve package
properties from nodes other than root.
New year, new versions of the ACTS dependencies. This time, we have a
new version of detray, a new version of algebra plugins, as well as a
new version of ACTS itself.
* Add type-hints to `spack.util.executable.Executable`
* Add type-hint to input
* Use overload, and remove assertions at calling sites
* Bump mypy to v1.11.2 (working locally), Python to 3.13
* root: Restrict patch range
* root: Set minimum gcc version for cxxstd=20
* root: fix gcc range when cxxstd 20
Co-authored-by: Paul Gessinger <hello@paulgessinger.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* add parse tool, json, and load json
bug fix
add variants and conflictions for g4tendl
* correct format
* correct format
* update version mapping
* remove 1.0
* Update var/spack/repos/builtin/packages/geant4-data/package.py
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* add options
* depends on it
* fix typo
---------
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* e4s cray rhel ci stack: re-enable and update for new cpe, should fix cray libsci issue
* only run e4s-cray-rhel stack
* Mkae Autotools build_system point at correct build_directory
* remove selective enable of cray-rhel stacks
* restore SPACK_CI_DISABLE_STACKS
* use dot prefix to hide cray-sles jobs instead of comment-out
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* edm4hep: Add lower clang version bounds
EDM4hep 0.99.1 introduced usage of consteval. While this is technically
supported in clang versions below 17, the implementation seems to be
incomplete and fail compilation of EDM4hep 0.99.1 and up.
* Update var/spack/repos/builtin/packages/edm4hep/package.py
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
---------
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
* py-py-spy: add 0.4.0
* py-py-spy: port from Package to CargoPackage
* CargoPackage: respect make_jobs
* rust: respect make_jobs during build and install
* spack style
* CargoBuilder: fix make_jobs syntax
* CargoBuilder: don't write to $HOME, use stage dir
* rivet: add version 3.1.11 with some back-port fixes
Some back-ported fixes from the 4.x series. Most importantly the
possibility to build against HepMC 3.3.0.
* yoda: Add version 1.9.11
* Add a conflict with hepmc=2
* Unify hepmc@3.3.0 conflicts into one statement
* Remove conflict and update conditional hepmc variant
* Use caret to signify conflict with dependency
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Make conflict and msg more specific
* Add dedicated version dependency to yoda
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* gromacs: remove dependency on Fortran
Fortran was removed from GROMACS core since 4.6. There are a few contrib
files around, but they are not built anyway.
Also fix a couple typos.
* [@spackbot] updating style on behalf of al42and
---------
Co-authored-by: al42and <al42and@users.noreply.github.com>
* zoltan: Ignore errors about incompatible pointer type with gcc@14
In gcc 14 -Werror=incompatible-pointer-types is now the default.
Silence it until it gets fixed in zoltan upstream.
* zoltan: Fix linking error when built with ifx
* mpich: gather in a single place env modifications needed by mpich derivatives
MPICH, and its derivatives, share a lot of copy/paste code to setup the
environment during the different stages of the package life-cycle.
This commit gathers the common modifications in a single place (a mixin class),
living in the Mpich package, and makes derivatives import, and reuse, it.
* Fix docs for Python < 3.13
f"Unable to find the {missing_packages}, please install these packages \
viatheVisual Studio installer \
beforeproceedingwithSpackorprovidethepathtoanonstandard install with \
'spack external find --path'"
f"Unable to find the {missing_packages}, please install these packages via the Visual "
"Studioinstaller before proceeding with Spack or provide the path to a non standard "
"installwith'spack external find --path'"
)
# wgl/sdk are not required for bootstrapping Spack, but
# are required for building anything non trivial
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
