Current location is within the Spack prefix, which causes builds to
pollute VCS with stage artifacts and generally inflates the Spack
install prefix.
This PR moves it to the user cache location now that we can
consistently support paths with spaces on Windows.
With CFLAGS, the code path in the amdfftw build system will bypass the logic around AMD_ARCH.
---------
Co-authored-by: vijay kallesh <Vijay-teekinavar.Kallesh@amd.com>
* salt: Add v0.3.0 of SALT
This version contains important bug fixes for building and parsing
projects containing Fortran
* salt: Be more explicit about dependency types
- llvm+clang+flang is needed at build, link and runtime for the
correct operation of SALT
- Testing with llvm@master ( llvm > 19.x ) shows that SALT is
currently incompatible with the latest llvm API so an updated salt
will be required when LLVM 20 is released
`relocate_links` warns when the target is absolute and not matched by
any prefix from the prefix to prefix map.
This can lead to false positives, cause the prefix to prefix map does
not contain trivial/identity entries whenever a package is installed to
its original location.
Since relocate_links is the odd one out there (we don't warn about
similar issues with rpaths, etc), just remove the warning.
Codecov needs to see the token secret when uploading, so we have to
add this line to the workflow YAML:
```yaml
with:
token: ${{ secrets.CODECOV_TOKEN }}
```
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Remove variable from cmake.py
#48775 left a dangling variable that was not caught in CI but by the eyes of @haampie. Restructure variable to local method.
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
* Update lib/spack/spack/build_systems/cmake.py
* Update lib/spack/spack/build_systems/cmake.py
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword
* Revert "Fix upcxx problem with a template argument list is expected after a name prefixed by the template keyword"
This reverts commit faf9b8ce85.
* Apply workaround for oneAPI compiler
* style problem resolved
* use spec.satisfies syntax
---------
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Add ruff configuration to `pyproject.toml`.
This allows `ruff format` in the Spack repository to format all the files we care about,
with our line length of 99, the exceptions we already put in place, and excluding things
we don't auto-format, like vendored dependencies.
Right now it'll reformat 175 or so files, but only slightly, in places where `ruff` differs from
`black`. For the most part I like the ruff format decisions better than `black`, but none of
the changes seem too severe.
This does not change `spack style` -- I figure that can come later but this at least will
let people start playing with `ruff`.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
These are some changes that `ruff check --fix` would make that the current
`spack style` also agrees with. Make the changes now so that the `ruff`
change is less disruptive.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
The import-check action now presents problematic import statements
introduced by the PR better.
The idea is roughly:
* Let (V₁, E₁) be the graph of modules as vertices and import statements
as edges before the change
* Let (V₂, E₂) be the graph after the code change, which is typically a small
perturbation of (V₁, E₁).
* X₁ = FAS(V₁, E₁) is the feedback arc set before (a minimal set of edges to
delete to make it acyclic)
* X₂ = FAS(V₂, E₂ ∖ X₁) is the feedback arc set after deletion of the minimal
set of edges that made the old graph acyclic.
* X₃ = FAS(V₂, E₂) is the feedback arc set after
Previously I displayed X₁ and X₃ and users had to diff themselves.
Now, I'm showing X₂, which is a small set, typically directly related to
code changes.
However, it can be that a small code change adding say 2 problematic imports
creates a completely different solution X₃ that only requires deletion of just 1
different import. In that case the user is informed that they can potentially do
less work.
So for PR #48784 the output is now:
> The overall number of problematic import statements increased by 1 from 31 to 32.
> This is likely a direct consequence of the following import statements:
>
> ```
> spack/config imports: spack.spec, spack.util.path, spack.util.remote_file_cache
> ```
>
> However, instead of removing 3 import statements, it is sufficient to remove only 1
> import statement from the following list:
>
> ```
> spack/concretize imports: spack.bootstrap, spack.solver.asp
> spack/environment imports: spack.bootstrap, spack.environment
> spack/fetch_strategy imports: spack.version.git_ref_lookup
> spack/install_test imports: spack.build_environment, spack.package_base
> spack/modules imports: spack.modules
> spack/platforms imports: spack.config
> spack/relocate imports: spack.bootstrap
> spack/repo imports: spack.package_base, spack.patch, spack.tag
> spack/spec imports: spack.binary_distribution, spack.compiler, spack.compilers, spack.concretize, spack.environment, spack.hash_types, spack.provider_index, spack.repo, spack.spec_parser, spack.store, spack.traverse, spack.variant, spack.version.git_ref_lookup
> spack/subprocess_context imports: spack.environment
> spack/util/gpg imports: spack.bootstrap
> spack/util/package_hash imports: spack.package_base
> spack/util/path imports: spack.config, spack.environment
> spack/util/remote_file_cache imports: spack.util.web
> ```
from which the user can figure out that
`spack/util/remote_file_cache imports: spack.util.web` is the "bottleneck" now.
* Add versions 2 and 3 of py-sphinx-rtd-theme.
Allow for versions of py-sphinx greater than 6.
Fix the Python version for older versions that depend on distutils.
Get the py-docutils dependency from the py-sphinx recipe.
* Depend purely on the py-docutils dependency in py-sphinx.
* More refined dependency versioning.
* Fixed versioning for py-sphinx and py-docutils.
Currently, environments created from manifest files with relative includes result in broken
references to config files.
This PR modifies `spack env create` to create local copies in the new environment of any local
config files from relative paths in the environment manifest passed as an init file.
This PR does not change the behavior if the include is an absolute path or if the include is from
a relative path outside the environment directory, but it does warn about missing relative includes if
they are inside the environment directory.
Includes regression test and short blurb in docs.
* Added salt variant to tau
* Update package.py
* [@spackbot] updating style on behalf of wspear
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* Create SALT package.py
Added a package for the SALT Source AnaLysis Toolkit
@zbeekman
* [@spackbot] updating style on behalf of wspear
* Update package.py
Line wrap
---------
Co-authored-by: wspear <wspear@users.noreply.github.com>
* py-mdit-py-plugins: Add new versions 0.3.5, 0.4.2
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* py-myst-parser: Add new versions 0.19.0 to 4.0.0
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* hpctoolkit: Add +docs variant and manpages
This commit unconditionally enables manpages for the HPCToolkit tools.
The new `+docs` variant enables additional documentation, specifically
the user's manual. Both require new build-time dependencies.
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
---------
Signed-off-by: Jonathon Anderson <anderson.jonathonm@gmail.com>
* package api: drop wildcard re-export
To ensure package repos are forward/backward compatibility with Spack,
we should explicitly export all symbols we want to expose in the public
package API, and drop `from spack.something import *` because
removal/addition to the public API will go unnoticed.
Also `llnl.util.filesystem` has some methods that shouldn't be exposed
in the package API, so better to enumerate a subset explicitly.
* remove flatten_dependencies / install_dependency_symlinks
* JAX: add v0.4.34
* Disable search for clang
* Update CUDA flags
* Add py-jax 0.4.33, comment out until py-jaxlib 0.4.33 is also released
* Fix GCC build
* Try TF_NVCC_CLANG
* py-jax: add v0.4.34
* jax no longer has separate tags for jaxlib
* Install compiled wheel
* Join path before glob
* Wheel is in spack stage, not tmp path
* Add 0.4.35
* Add newer versions
* Build system has been refactored yet again
* Drop clang
* Fix build with source tarball, rocm support
* Support GCC
* Remove clang-specific compiler flags
* enable_cuda flag was removed
* Fix logic
* py-jax: add v0.4.38
* Add patch to fix GCC support
* Patch no longer needed
* Skip patching, directly pass flags
* New flags
* Remove unused import
* Patch changed
* Use older version of patch
* Newer patch
* Add CUDA symlink
* Symlink more directories
* Recursive symlink
* Import function
* Recursive search
* Undo cuda changes
* Add v0.5.0
* I quit
* Improve definition of a few placeholder packages
These packages are placeholders for vendor provided software,
that is not buildable, and should be declared as external.
* ibm-java: remove package, as asked by maintainer
Regressed in #47126
Spack was not interpreting mirrors using relative path with respect to the
metadata directory.
---------
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* test_no_matching_compiler_specs: does not need mock_low_high_config,
since mutable_config is already used at class level
* bindist.py: setup a configuration that doesn't super-impose builtin.mock
over builtin
* builder.py: use a mock configuration for the tests
This commit adds version 6.8.0 of GeoModel. As far as I can tell from
the change notes, there are no changes required to the build
configuration or dependencies.
This adds a new configuration section called `env_vars:` that can be set in an environment.
It looks very similar to the existing `environment:` section that can be added to `modules.yaml`,
but it is global for an entire spack environment. It's called `env_vars:` to deconflate it with spack
environments (the term was too overloaded).
The syntax looks like this:
```yaml
spack:
specs:
- cmake%gcc
env_vars:
set:
ENVAR_SET_IN_ENV_LOAD: "True"
```
Any of our standard environment modifications can be added to the `env_vars` section, e.g.
`prepend_path:`, `unset:`, `append_path:`, etc. Operations in `env_vars:` are performed
on `spack env activate` and undone on `spack env deactivate`.
* Reduce the size of outputted go built binaries
* Remove unused import from go package
* go: remove comment from setup dependents build env
* Add back missing imports after rebase
* Backward compat with Python 3.9 for socket.timeout
* Forward compat with Python [unknown] as HTTPResponse.geturl is deprecated
* Catch timeout etc from .read()
* Some minor simplifications: json.load(...) takes file object in binary mode.
* Fix CDash code which does error handling wrong: non-2XX responses raise.
The conflict is too strict and prevents usable combinations from being
concretized. Some packages depend on pika, but do not include pika
headers in HIP device code files, and can thus be compiled even with the
bad combination of HIP and GCC versions.
* py-awkward: add dependency on py-importlib-metadata and py-fsspec
* Implement suggested constraint and dependency type changes
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
On macOS you cannot unconditionally pass `-rpath`, it can conflict with
`-r`.
mpicc is not the place to inject rpaths to compiler runtime
libraries. That's up to the compiler's config files (spec files in gcc,
config files in llvm).
Also remove some patches that are redundant in newer versions of
openmpi.
Should be sufficient to set CONFIG_SITE to /dev/null before running configure to
prevent any config site file to be loaded, which currently causes non-determinism in builds.
Setting the variable CONFIG_SITE prevents configure files to check $prefix/share/config.site,
$prefix/etc/config.site, $ac_default_prefix/share/config.site, $ac_default_prefix/etc/config.site
so we don't have to patch a block of code.
* Address quoting issue that casuses dev_paths containing @ symbols to parse as versions
* Fix additional location were dev_path was imporperly constructed
The dev_path must be quoted to avoid parsing issues when
paths contain '@' symbols. Also add tests to catch
regression of this behavior
* Format to fix line length
* fix failing tests
* Fix whitespace error
* Add binary_compatibility fixture to test
* Change string formatting to avoid multiline f string
* Update lib/spack/spack/test/concretization/core.py
* Add tmate debug session
* Revert "Add tmate debug session"
This reverts commit 24e2f77e3c.
* Move test and refactor to use env methods
* Update lib/spack/spack/test/cmd/develop.py
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
Since macOS 15 `ld -single_module` warns with a deprecation message,
which makes configure scripts believe the flag is unsupported. That
in turn triggers a code path where `archive_cmds` is set to
```
$CC -r -keep_private_externs -nostdlib ... -dynamiclib
```
instead of just
```
$CC -dynamiclib ...
```
This code path was meant to trigger only on ancient macOS <= 14.4 where
libtool had to add `-single_module`, which is the default since macos
14.4, and is now apparently deprecated because the flag is a no-op for
more than 15 years.
The wrong `archive_cmds` causes actual problems combined with a bug in
OpenMPI's compiler wrapper (`CC=mpicc`), which appends `-rpath` flags,
which cause an error when combined with the `-r` flag added by the
autotools.
Spack's compiler wrapper doesn't do this, but it's likely there are
other compiler wrappers out there that are not aware that `-r` and
`-rpath` cannot be combined.
The fix is to change defaults: `lt_cv_apple_cc_single_mod=yes`.
* BF+ENH: Add wxwidgets 3.2.6 and py-wxpython 4.2.2
Improves compat with newer Python (>3.9) and numpy.
Fix error during configure step (even on at least some older
versions) by including 'pkgconfig' as a build dep so gtk+ is found.
* BF: Make wxpython use spack built wxwidgets instead of rebuilding
* Update var/spack/repos/builtin/packages/py-wxpython/package.py
Avoid using too new version of py-setuptools as license file format is currently not compatible.
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* for config: let `syaml_dict` inherit from `dict` instead of `OrderedDict`. `syaml_dict` now only exists as a mutable wrapper for yaml related metadata.
* for spec serialization / hashing: use `dict` directly
This is possible since we only support cpython 3.6+ in which dicts are ordered.
This improves performance of hash computation a bit. For a larger spec I'm getting 9.22ms instead of 11.9ms, so 22.5% reduction in runtime.
* OSError.errno apparently is int | None, but already part of __str__ anyway so drop
* fix disambiguate_spec_from_hashes
* List[Spec].sort() complains, ignore it
* fix typing of KnownCompiler, and use it in asp.py
Adds packages for the Arkouda server (`arkouda`) and Arkouda python client (`py-arkouda`).
Arkouda server and client are divided into separate packages to allow for them to be
installed independently of one another.
Future work remains to add a `+dev` variant to `py-arkouda`, but that will require additional
supporting packages made available through spack (e.g. for `py-pytest-env`
Modifications:
* Fix a severe bug where a spliced spec has the same hash as the original build spec
* Removed CacheManager in favor of install_mockery. The latter sets up a temporary store, and removes it at the end
of the test.
* Deleted a couple of helper functions e.g. _has_dependencies
* Checked that SolverException is raised, rather than Exception (more specific)
* Extended, and renamed the splicing_setup fixture (now called install_specs)
* Added more specific assertions in each test
One test is currently flaky, due to some instability when sorting multiple specs. It's currently marked xfail
The methods spack.spec.Spec.concretize and spack.spec.Spec.concretized
are deprecated in favor of spack.concretize.concretize_one.
This will resolve a circular dependency between the spack.spec and
spack.concretize in the next Spack release.
* add python bindings variant to chapel
* fix chpl_home, update chapel frontend rpath in venv
* fix chpl frontend shared lib rpath hack
* patch chapel env var line length limit
* patch chapel python shared lib location by chapel version
* unhack chpl frontend lib rpath
* fix postinstall main version number file path
* update chapel version number to 2.3
* use chapel releases instead of source tarballs, remove dead code
* Chapel 2.3 adds support for LLVM 19
* Bundled LLVM always requires CMake 3.20:
* Apply 2.3 patch for LLVM include search path
Fixes build errors for `chapel@2.3+rocm` of the form:
```
compiler/llvm/llvmDebug.cpp:174:9: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
compiler/llvm/llvmDebug.cpp:254:15: error: cannot convert 'const char*' to 'llvm::dwarf::MemorySpace'
```
* fix style
* Fix misreporting of test_hello failures
`test_part` was aliasing the enclosing test name, leading to confusing and incorrect reporting on test failure.
* Ensure `libxml2` optional dep of `hwloc` is also added to `PKG_CONFIG_PATH` in the run env
* patch chplCheck for GPU + multilocale configs, install docs
* Adjust patch for checkChplInstall
* Ensure `CHPL_DEVELOPER` is unset for `~developer`
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
* Improve support using external modules with zsh
Spack integrates with external modules by launching a python subprocess
to scrape the path from the module command. In zsh, subshells do not
inheret functions defined in the parent shell (only environment
variables). This breaks the module function in module_cmd.py.
As a workaround, source the module commands using $MODULESHOME prior to
running the module command.
* Fix formatting
* Fix flake error
* Add guard around sourcing module file
* Add improved unit testing to module src command
* Correct style
* Another attempt at style
* formatting again
* formatting again
---------
Co-authored-by: psakievich <psakiev@sandia.gov>
* bdsim: update to point to the new location in github and add 1.7.7
* Remove the C++ standard patch
* Remove the cmake_args and add a comment instead
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
`spack find` and other commands that read the DB will fail if the databse is in a
read-only location, because `FailureTracker` and `Database` can try to create their
parent directories when they are constructed, even if the DB root is read-only.
Instead, we should only write to the filesystem when needed.
- [x] don't create parent directories in constructors
- [x] have write operations create parents if needed
- [x] in tests, `chmod` databases to be read-only unless absolutely needed.
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Adding the PACE library to the LAMMPS build
* Adding the PACE library for ML-PACE in LAMMPS
---------
Co-authored-by: Richard Berger <rberger@lanl.gov>
* Add py-protobuf@4.24.4 (needed for py-cylc-flow@8.3.6).
* Add py-cylc-flow@8.3.6 and enable png output when creating graphs
by requesting variant pangocairo for graphviz dependency.
* Add corresponding versions of py-metomi-rose@2.3.2,
py-cylc-rose@1.4.2, py-cylc-uiserver@1.4.2.
* Add myself as maintainer to all the cylc-related packages.
This PR allows using the subscript notation directly on packages. The
intent is to reduce the boilerplate needed to retrieve package
properties from nodes other than root.
New year, new versions of the ACTS dependencies. This time, we have a
new version of detray, a new version of algebra plugins, as well as a
new version of ACTS itself.
* Add type-hints to `spack.util.executable.Executable`
* Add type-hint to input
* Use overload, and remove assertions at calling sites
* Bump mypy to v1.11.2 (working locally), Python to 3.13
* root: Restrict patch range
* root: Set minimum gcc version for cxxstd=20
* root: fix gcc range when cxxstd 20
Co-authored-by: Paul Gessinger <hello@paulgessinger.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* add parse tool, json, and load json
bug fix
add variants and conflictions for g4tendl
* correct format
* correct format
* update version mapping
* remove 1.0
* Update var/spack/repos/builtin/packages/geant4-data/package.py
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* add options
* depends on it
* fix typo
---------
Co-authored-by: Seth R. Johnson <johnsonsr@ornl.gov>
* e4s cray rhel ci stack: re-enable and update for new cpe, should fix cray libsci issue
* only run e4s-cray-rhel stack
* Mkae Autotools build_system point at correct build_directory
* remove selective enable of cray-rhel stacks
* restore SPACK_CI_DISABLE_STACKS
* use dot prefix to hide cray-sles jobs instead of comment-out
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
* edm4hep: Add lower clang version bounds
EDM4hep 0.99.1 introduced usage of consteval. While this is technically
supported in clang versions below 17, the implementation seems to be
incomplete and fail compilation of EDM4hep 0.99.1 and up.
* Update var/spack/repos/builtin/packages/edm4hep/package.py
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
---------
Co-authored-by: Thomas Madlener <thomas.madlener@desy.de>
* py-py-spy: add 0.4.0
* py-py-spy: port from Package to CargoPackage
* CargoPackage: respect make_jobs
* rust: respect make_jobs during build and install
* spack style
* CargoBuilder: fix make_jobs syntax
* CargoBuilder: don't write to $HOME, use stage dir
* rivet: add version 3.1.11 with some back-port fixes
Some back-ported fixes from the 4.x series. Most importantly the
possibility to build against HepMC 3.3.0.
* yoda: Add version 1.9.11
* Add a conflict with hepmc=2
* Unify hepmc@3.3.0 conflicts into one statement
* Remove conflict and update conditional hepmc variant
* Use caret to signify conflict with dependency
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Make conflict and msg more specific
* Add dedicated version dependency to yoda
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* gromacs: remove dependency on Fortran
Fortran was removed from GROMACS core since 4.6. There are a few contrib
files around, but they are not built anyway.
Also fix a couple typos.
* [@spackbot] updating style on behalf of al42and
---------
Co-authored-by: al42and <al42and@users.noreply.github.com>
* zoltan: Ignore errors about incompatible pointer type with gcc@14
In gcc 14 -Werror=incompatible-pointer-types is now the default.
Silence it until it gets fixed in zoltan upstream.
* zoltan: Fix linking error when built with ifx
* mpich: gather in a single place env modifications needed by mpich derivatives
MPICH, and its derivatives, share a lot of copy/paste code to setup the
environment during the different stages of the package life-cycle.
This commit gathers the common modifications in a single place (a mixin class),
living in the Mpich package, and makes derivatives import, and reuse, it.
* Fix docs for Python < 3.13
* openloops: Add cmodel to user config file to override setting added in 2.1.2.
* openloops: Change cmodel from small to large following author recommendation.
A few changes to tarball creation (for build caches):
- do not run file to distinguish binary from text
- file is slow, even when running it in a batched fashion -- it usually reads all bytes and has slow logic to categorize specific types
- we don't need a highly detailed file categorization; a crude categorization of elf, mach-o, text suffices.
detecting elf and mach-o is straightforward and cheap
- detecting utf-8 (and with that ascii) is highly accurate: false positive rate decays exponentially as file size increases. Further it's not only the most common encoding, but the most common file type in package prefixes.
iso-8859-1 is cheaply (but heuristically) detected too, and sufficiently accurate after binaries and utf-8 files are classified earlier
- remove file as a dependency of Spack in general, which makes Spack itself easier to install
- detect file type and need to relocate as part of creating the tarball, which is more cache friendly and thus faster
mpicxx_shared_libraries seems a relic of #1550, and is
not currently used by any builtin package.
Thus, cleanup the recipes, and avoid monkey-patching
spec objects.
Python >= 3.13 does not have the crypt variant anymore. Still no matter
if the test for crypt succeeds (which it can on Fedora providing its own
crypt module for Python 3.13) or fails, it will add +crypt or ~crypt,
which both fail because the variant only exists until Python 3.12.
Co-authored-by: Richard Berger <rberger@lanl.gov>
* silo: variant python needs python
* Dependency to Python did not resolve the "Python.h" header not being found
Added the -I path to Python header to the compiler.
Having silo depend on python was not sufficient to get the path to python.
Maybe there is a smarter way to do that, but this one works.
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/silo/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* thepeg: Correctly specify rivet version for dependency
* herwig3: Add latest version 7.3.0
* thepeg: Make sure to have consistent hepmc version
rivet and thepeg need to have the same hepmc version otherwise things
will not compile
* libzip links with external libs found on host feelpp/spack#6
* add maintainers
* fix style
* use multi-build system
/cc @wdconinc
* fix style
* rm space and rename variant bz2 to bzip2
/cc @wdconinc
* fix variant name for bzip2
* zstd is supported in libzip@:1.8
* fix style
* fix style
* fix style
* rm deprecated version and versions that cannot be found easily
use only cmake from now on
* fix style
* fix style
* use variant when option for zstd
* Update var/spack/repos/builtin/packages/libzip/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* fix style
/cc @wdconinc
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
`kcov` was removed in Ubuntu 24.04, and it is no longer
installable via `apt` in our CI images. Instal it via
Linuxbrew instead, at least until it comes back to Ubuntu.
`subversion` is also not installed on ubuntu 24 by default,
so we have to install it manually.
- [x] Add linuxbrew to linux tests
- [x] Install `kcov` with brew
- [x] Install subversion with `apt`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Fix silent error when reporting builds to CDash
CDash has a 191 char maximum for build names. When this
is exceeded, CDash silently fails to correctly process the
reported XML. This truncates CDash build names to 190 chars
and emits a warning indicating it is doing so to prevent
such errors from occuring.
* test/reporters.py: add unittest for buildname len issue
* test/reporters.py: rename cdash buildname test
* ci/common.py: fix syntax causing breaking test
It appears that the CDash reporter is expecting a string
as the buildname.
* Update lib/spack/spack/reporters/cdash.py
Fix warning message to reflect actual issue.
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* ci/common.py: fix function call to actually call function
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: psakievich <psakiev@sandia.gov>
* py-typer: add version 0.15.1 and "standard" optional dependencies
* py-typer: remove variant that only exists in source, not sdist. Remove trailing .0 from versions.
* Improve variant robustness for dd4hep and edm4hep
Now variants won't be "false" if there's a typo.
* Use libs instead of manual prefix paths
* Improve cmake for another hep package
* Fix variant use and style
* Use directories for ODD
* r-rlas is a dependency for r-lidr
* new package r-lidr w/ suggests to address masking issues
* fixed flake8 issues and added maintainers
* removed boost import statement for flake sake
* Score-P: Replace with-or-without, document options that are not currently explicitly mapped in package for mpi and shmem.
* trim long lines
---------
Co-authored-by: wrwilliams <wrwilliams@users.noreply.github.com>
Improve our typing by updating some todo locations in the code to use
`Literal` instead of a simple `str`.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Fix issue reported by some users regarding some build dependencies.
* Remove invalid configure-time flag that was recently introduced.
Signed-off-by: Samuel K. Gutierrez <samuel@lanl.gov>
There are still more fix ups required for the missing libs to work as
expected in CI. Dropping the error requirement in favor of moving to a
log scraping method until we can verify all package issues have been
addressed correctly.
* Set the "build_jobs" on concretization/generate for CI
build_jobs also controls the concretization pool size. Set this
in the config section for CI generate.
This config is overwritten by build_job CI using the SPACK_BUILD_JOBS
environment variable. This implicitly will drop the default build
CPU request on all "default" grouped build jobs from (max) 16 to 8.
* Add default allocations for build jobs
* Add common jobs and concretize args to ci generate and rebuild
* CI: Specify parallel concretize and build jobs via argument
* Increase power and cray concretization limits
Lowering limits for these stacks creates timeout
* Increase default pool size to 8
intermittent timeouts with 4 CPU
* Add reduced requests for windows for now
This turns some variant-specific methods for dealing with when-keyed dictionaries into
more generic versions, in preparation for conditional version definitions.
`_by_name`, `_names`, etc. are replaced with generic methods for transforming
when-keyed dictionaries:
* `_by_subkey()`
* `_subkeys()`
* `_num_definitions()`
* `_definitions()`
* `_remove_overridden_defs()`
And the variant accessors are refactored to use these methods underneath.
To do this, types like `WhenDict` had to be generified, and some `TypeVars`
were added for sortable keys and values.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
We are using more and more typing features in Spack, and without features like
protocols, typing core is becoming harder and harder.
I think it's worth vendoring `typing_extensions` for this. It will get us a number of
useful capabilities:
* `Literal`
* `TypedDict`
* `Protocol`
among others.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This commit adds a config option `config:shared_linking:missing_library_policy:error/warn/ignore` which will cause installation errors or warnings when ELF executables or libraries need shared libraries which cannot be resolved from RPATH search paths. The default is to ignore.
This is a safeguard against accidentally linking to system libraries instead of Spack libraries. It makes it more likely that build cache installs work on different machines. It works only at the level of libraries, not at the level of symbols. Some system dependencies are allowed (e.g. kernel and libc).
Packages can (but are discouraged to) set `unresolved_libraries` to a list of patterns of sonames/library names that are know to be unresolvable in RPATHs. In the future this could be made more fine-grained in a non-breaking way by allowing a dictionary of patterns `lib => [deps]`.
Extracted #45189
Common test setup has been extracted in fixtures. Some matrix
dimensions moved from being "compiler" to be "targets".
Use --fake install for packages in test.
The `_normal` attribute on specs is no longer used and has no meaning.
It's left over from part of the original concretizer.
The `concrete` constructor argument is also not used by any part of core.
- [x] remove `_normal` attribute from `Spec`
- [x] remove `concrete` argument from `Spec.__init__`
- [x] remove unused `check_diamond_normalized_dag` function in tests
- [x] simplify `Spec` constructor and docstrings
I tried to add typing to `Spec` here, but it creates a huge number of type issues
because *most* things on `Spec` are optional. We probably need separate `Spec` and
`ConcreteSpec` classes before attempting that.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* nim: fix deps, deprecate and patch old versions
* Fix runtime dependencies for produced binaries:
- Add -rpaths pointing at dependencies to
std wrapper modules
- Set version constraints for OpenSSL
- Added SQLite3 variant for <2.0
* Parallelize build with make
* Deprecate 1.0.10 due to CVEs
* Deprecate 1.9.3 as it's an old development version
* Backport patch for CVE-2021-21372 to <1.2.10/<1.4.4,
CVE-2021-21374 to 1.4.2 and CVE-2021-46872 to 1.4.*
* Avoid empty low ranges that include devel
* Add previously missing CVE comment
* Keep "link" type for dynamic libraries for MSVC
* Omit "run" type for library dependencies
* Disable SQLite variant by default
* Fix version ranges
Had assumed they were exclusive, but they're inclusive
* Correct version range for sqlite variant
Difference doesn't matter outside of development versions
* Move patches to use GitHub URLs instead of files
* Retry CI
* append ?full_index=1
Previously the pip setup would delete the visitmodule during the install
step. This was fixed by forcing the pip setup to only run once before
the dependents are created.
Add missing encoding=utf-8 to various open calls. This makes
files like spec.json, spack.yaml, spack.lock, config.yaml etc locale
independent w.r.t. text encoding. In practice this is not often an
issue since Python 3.7, where the C locale is promoted to
C.UTF-8. But it's better to enforce UTF-8 explicitly, since there is
no guarantee text files are written in the right encoding.
Also avoid opening in text mode if it can be avoided.
* `f.tell` on a `TextIOWrapper` does not return the offset in bytes, but
an opaque integer that can only be used for `f.seek` on the same
object. Spack assumes it's a byte offset.
* Do not open in a locale dependent way, but assume utf-8 (and allow
users to override that)
* Use tempfile to generate a backup/temporary file in a safe way
* Comparison between None and str is valid and on purpose.
Follow-up to #47956
* Rename `token.py` -> `tokenize.py`
* Rename `parser.py` -> `spec_parser.py`
* Move common code related to iterating over tokens into `tokenize.py`
* Add "unexpected character token" (i.e. `.`) to `SpecTokens` by default instead of having a separate tokenizer / regex.
* Python: deprecate 3.8
* Remove preference for EOL Python versions
* Explicitly deprecate things requiring EOL Python
* More deprecations
* deprecate old versions of slepc, py-petsc4py, py-slepc4py in sync with old versions of petsc
---------
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
The use of `^` in `depends_on` directives has never been allowed, since
the dawn of Spack.
Up to now, we used to have an audit to catch this kind of issue, mainly
because in that way we could easily collect all issues and report them
to packagers at once.
Due to implementation details, this audit doesn't work if a dependency
without a `^` is followed by the same dependency with a `^`.
This PR makes this pattern an error, which will be reported eagerly, and
removes the corresponding audit. It also fixes a package using the wrong
idiom.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
So far, the ESMF package recipe in spack assumes that the spack
compilers clang and apple-clang are using gfortran as the Fortran
compiler. But with the latest improvements to the LLVM compilers,
we need to also support clang with flang.
Reorganize the pipeline generation aspect of the ci module,
mostly to separate the representation, generation, and
pruning of pipeline graphs from platform-specific output
formatting.
Introduce a pipeline generation registry to support generating
pipelines for other platforms, though gitlab is still the only
supported format currently.
Fix a long-existing bug in pipeline pruning where only direct
dependencies were added to any nodes dependency list.
* Set the "build_jobs" on concretization/generate for CI
build_jobs also controls the concretization pool size. Set this
in the config section for CI generate.
This config is overwritten by build_job CI using the SPACK_BUILD_JOBS
environment variable. This implicitly will drop the default build
CPU request on all "default" grouped build jobs from (max) 16 to 8.
* Add default allocations for build jobs
* Add common jobs and concretize args to ci generate and rebuild
* CI: Specify parallel concretize and build jobs via argument
* Increase power and cray concretization limits
Lowering limits for these stacks creates timeout
* py-nbclassic: add v1.1
* py-nbclassic: reduce explicit dependencies for v1.1.0
Having all the 'excess' packages listed did not break anything, as
they were needed for `py-jupyter-server` (pulled in via `py-notebook-shim`)
anyway, but the change makes it more clear on why things are being pulled in.
* acts dependencies: new versions as of 2024/12/08
This commit includes a new version of ACTS, as well as new versions of
the ACTS algebra plugins, covfie, detray, and geomodel.
* Fixes
* covfie: depends_on cmake@3.21: when @0.11:
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Add py-param@2.1.1
* Add py-panel@1.5.2
* Add py-bokeh@3.5.2
* New package py-datashader
* New package py-geoviews
* New package py-holoviews
* WIP: new package py-uxarray
* New package py-antimeridian
* New package py-dask-expr
* New package py-spatialpandas
* New package py-hvplot
* Add dependency on py-dask-expr for 'py-dask@2024.3: +dataframe'
* Added all dependencies for py-uxarray; still having problems with py-dask +dataframe / py-dask-expr
* Fix style errors in many packages
* Clean up comments and fix style errors in var/spack/repos/builtin/packages/py-dask-expr/package.py
* In var/spack/repos/builtin/packages/py-dask/package.py: since 2023.8, the dataframe variant requires the array variant
* Fix style errors in py-uxarray package
- [x] Clean up arguments on the `resource` directive.
- [x] Add type annotations
- [x] Add `resource` to type annotations on `PackageBase`
- [x] Fix up `resource` docstrings
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Some of the class-level annotations were wrong, and some were missing. Annotate all the
functions here and fix the class properties to match what's actually happening.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
The first argument to each Spack directive is not a `PackageBase` instance but a
`PackageBase` class object, so fix the type annotations to reflect this.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`Optional` shouldn't be part of `PatchesType` -- it's clearer to specify `Optional` it
in the methods that need their arguments to be optional.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* update py-numl and py-nugraph recipes
this commit adds the develop branch as a valid option for each of these two packages. in order to enable this, package tarballs are now retrieved from the github source repository instead of pypi, and their checksums and the build system have been updated accordingly.
* rename versions "develop" -> "main" to be consistent with branch name
* nim: add latest versions
In addition:
- Create separate build and install phases.
- Remove koch nimble call as it's redundant with koch tools.
- Install all additional tools bundled with Nim instead of only Nimble.
* Fix 1.6 version
* nim: add devel
In addition:
- Fix build accessing user config/cache
* Bump Kokkos and Kokkos-kernels to 4.5.00
* petsc@:3.22 add a conflict with this new version of kokkos
* Update kokkos/kokkos-kernel dependency
---------
Co-authored-by: Satish Balay <balay@mcs.anl.gov>
the nis module was removed in python 3.13
we had it default to ~nis
no package requires +nis
required dependencies for +nis were missing
so better to remove the nis module entirely.
* acts dependencies: new versions as of 2024/11/25
This commit adds a new version of detray and two new versions of vecmem.
* acts dependencies: new versions as of 2024/12/02
This commit adds version 38 of ACTS and a new version of detray.
In preparation for adding `when=` to `version()`, I'm cleaning up the types in
`version_types` and making sure the methods here pass `mypy` checks. This started as an
attempt to use `ConcreteVersion` outside of `spack.version` and grew into a larger type
refactor.
The hierarchy now looks like this:
* `VersionType`
* `ConcreteVersion`
* `StandardVersion`
* `GitVersion`
* `ClosedOpenRange`
* `VersionList`
Note that the top-level thing can't easily be `Version` as that is a method and it
returns only `ConcreteVersion` right now. I *could* do something fancy with `__new__` to
make `Version` a synonym for the `ConcreteVersion` constructor, which would allow it to
be used as a type. I could also do something similar with `VersionRange` but not sure if
it's worth it just to make these into types.
There are still some places where I think `GitVersion` might not be handled properly,
but I have not attempted to fix those here.
- [x] Add a top-level `VersionType` class that all version types extend from
- [x] Define and document common methods and rich comparisons on `VersionType`
- [x] Replace complicated `Union` types with `VersionType` and `ConcreteVersion` as needed
- [x] Annotate most methods (skipping `__getitem__` and friends as the typing is a pain)
- [x] Fix up the `VersionList` constructor a bit
- [x] Add cases to methods that weren't handling all `VersionType`s
- [x] Rework some places to clarify typing for `mypy`
- [x] Simplify / optimize _next_version
- [x] Make StandardVersion.string a property to enable lazy comparison
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This visitor accepts the sub-dag of all nodes and unique edges that have
deptype X directly from given roots, or deptype Y transitively for any
of the roots.
LLVM can be a transitive link dependency of hip through gl's dependency mesa, which uses it for software rendering.
In this case make sure llvm-amdgpu is found with find_package(LLVM) and
find_package(Clang) by setting LLVM_ROOT and Clang_ROOT.
That makes the patch of find_package's HINTS redundant, so remove that.
It did not work anyways, because CMAKE_PREFIX_PATH has higher precedence
than HINTS.
Explcitly sets the CMake variables Faodel_INCLUDE_DIRS and Faodel_LIBRARY_DIRS when +faodel.
This seems to be needed for recent versions of seacas (seacas@2021-04-02:), but should be safe
to do for all versions.
For Faodel_INCLUDE_DIRS, it looks like Faodel has header files under $(Faodel_Prefix)/include/faodel,
but seacas is not including the "faodel" part in #includes. So add both $(Faodel_Prefix)/include
and $(Foadel_Prefix)/include/faodel
Co-authored-by: payerle <payerle@users.noreply.github.com>
* c/c++ flags should have been modified for all 2023.x.y versions, but
upper bound was too low
* Fortran flags should have been modified for all 2024.x.y versions, but
likewise the upper bound was too low
* gromacs: announce deprecation policy and start to implement
* Style it up
* [@spackbot] updating style on behalf of mabraham
* Bump versions used in CI
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Permit configuring GROMACS with support for mdrun to trace its timing
regions by calling the ITT API. This permits tools like VTune and
unitrace to augment their analysis with GROMACS-specific annotation.
external googletest breaks dependents because they end up with
ITK_LIBRARIES set to `GTest::GTest;GTest::Main`, which then end up
literally in a nonsensical link line `-lGTest::GtTest`.
the vendored googletest produces a cmake config file where
`ITKGoogleTest_LIBRARIES` is empty.
* gromacs: oneapi does not always require gcc
* Support intel_provided_gcc only with %intel classic compiler
Require gcc only when needed with %intel
* New approach depending on gcc-runtime directly
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
---------
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
* Visit: Add new versions 3.4.0 and 3.4.1
* Adios2: Restrict python, 3.11 doesn't not work for older Adios2
* VisIt: Set the VTK_VERSION for @3.4:
Older versions of VTK used the VTK_{MAJOR, MINOR}_VERSION variables for
VTK detection. VisIt >= 3.4 uses the full string VTK_VERSION.
* CI: Don't build llvm-amdgpu for non-HIP stack
* VisIt: v3.4.1 handles newer Adios2 correctly
* Visit: Add missing links in HDF5, set correct VTK version configuration parameter
* VisIt: Add py-pip requirement and patch visit with configuration changes
* HDF5 symlinks move when inside of callback
* VisIt ninja install fails with python module. Using make does not
* VisIt 3.4 has a high minimum cmake requirement
* HDF5: Early return when not mpi for mpi symlinks
* HDF5: Use platform agnostic method for creating legacy compatible MPI symlinks
* Fix VISIT_VTK_VERSION handling for 8.2.1a hack
* This extends PR #47285 to properly include some of the required version constrains of cgal 6 incl C++ standard. It also adds the new no-gmp backend as a variant.
* fix style
* disable cgal@6 +demo variant as the demos require qt6 which is not in spack
* disable the gmp variant until clarity on how its supposed to work is provided. bound shared and header_only variants to relevant versions
* Fix missing msvc compiler limit, fix variant left in
* Add more comments. Better describe the gmp variant. Remove testing code
* fix style
The default build of clang on darwin couldn't actually build anything
because of a lack of a sysroot built in. Also several compilation
errors finding the system libc++ cropped up, much like those in GCC, and
have been fixed.
* packages: Update 'yambo'
* add call to 'resource' method to download Ydriver and iotk during fetch instead of during build
* air-gapped installation could be performed since version 5.2.1
* add versions 5.2.3 and 5.2.4
* remove some inexistant configure options for versions "@5:"
* add a sanity_check on 'bin/yambo'
The gni libfabric provider works on some Cray systems, but not all. For
example, Slingshot-based machines use a different libfabric provider
(cxi). Therefore libfabric/gni should not be a dependency when using
Cray PMI.
* celeritas: remove deprecated versions through 0.3
* celeritas: deprecate old versions
* celeritas: add c++20 option
* Propagate vecgeom CUDA requirements
* Remove outdated conflicts and format it
Automatic splicing say `Spec` grow a `__len__` method but it's only used
in one place and it's not clear the semantics are useful elsewhere. It also
runs the risk of Specs one day being confused for other types of containers.
Rather than introduce a new function for one algorithm, let's use a more
specific method in the splice code.
- [x] Use topological ordering in `_resolve_automatic_splices` instead of
sorting by node count
- [x] delete `Spec.__len__()` and `Spec.__bool__()`
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
`spack spec` output has looked like this for a while:
```console
> spack spec /v5fn6xo /wd2p2v7
Input spec
--------------------------------
- /v5fn6xo
Concretized
--------------------------------
[+] openssl@3.3.1%apple-clang@16.0.0~docs+shared build_system=generic certs=mozilla arch=darwin-sequoia-m1
[+] ^ca-certificates-mozilla@2023-05-30%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
...
Input spec
--------------------------------
- /wd2p2v7
Concretized
--------------------------------
[+] py-six@1.16.0%apple-clang@16.0.0 build_system=python_pip arch=darwin-sequoia-m1
[+] ^py-pip@23.1.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
```
But the input spec is right there on the CLI, and it doesn't add anything to the output.
Also, since #44843, specs concretized in the CLI line can be unified, so it makes sense
to display them as we did in #44489 -- as one multi-root tree instead of as multiple
single-root trees.
With this PR, concretize output now looks like this:
```console
> spack spec /v5fn6xo /wd2p2v7
[+] openssl@3.3.1%apple-clang@16.0.0~docs+shared build_system=generic certs=mozilla arch=darwin-sequoia-m1
[+] ^ca-certificates-mozilla@2023-05-30%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] ^gmake@4.4.1%apple-clang@16.0.0~guile build_system=generic arch=darwin-sequoia-m1
[+] ^perl@5.40.0%apple-clang@16.0.0+cpanm+opcode+open+shared+threads build_system=generic arch=darwin-sequoia-m1
[+] ^berkeley-db@18.1.40%apple-clang@16.0.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=darwin-sequoia-m1
[+] ^bzip2@1.0.8%apple-clang@16.0.0~debug~pic+shared build_system=generic arch=darwin-sequoia-m1
[+] ^diffutils@3.10%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^libiconv@1.17%apple-clang@16.0.0 build_system=autotools libs=shared,static arch=darwin-sequoia-m1
[+] ^gdbm@1.23%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^readline@8.2%apple-clang@16.0.0 build_system=autotools patches=bbf97f1 arch=darwin-sequoia-m1
[+] ^ncurses@6.5%apple-clang@16.0.0~symlinks+termlib abi=none build_system=autotools patches=7a351bc arch=darwin-sequoia-m1
[+] ^pkgconf@2.2.0%apple-clang@16.0.0 build_system=autotools arch=darwin-sequoia-m1
[+] ^zlib-ng@2.2.1%apple-clang@16.0.0+compat+new_strategies+opt+pic+shared build_system=autotools arch=darwin-sequoia-m1
[+] ^gnuconfig@2022-09-17%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] py-six@1.16.0%apple-clang@16.0.0 build_system=python_pip arch=darwin-sequoia-m1
[+] ^py-pip@23.1.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[+] ^py-setuptools@69.2.0%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
[-] ^py-wheel@0.41.2%apple-clang@16.0.0 build_system=generic arch=darwin-sequoia-m1
...
```
With no input spec displayed -- just the concretization output shown as one consolidated
tree and multiple roots.
- [x] remove "Input Spec" section and "Concretized" header from `spack spec` output
- [x] print concretized specs as one BFS tree instead of multiple
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
This PR provides complementary 2 features:
1. An augmentation to the package language to express ABI compatibility relationships among packages.
2. An extension to the concretizer that can synthesize splices between ABI compatible packages.
1. The `can_splice` directive and ABI compatibility
We augment the package language with a single directive: `can_splice`. Here is an example of a package `Foo` exercising the `can_splice` directive:
class Foo(Package):
version("1.0")
version("1.1")
variant("compat", default=True)
variant("json", default=False)
variant("pic", default=False)
can_splice("foo@1.0", when="@1.1")
can_splice("bar@1.0", when="@1.0+compat")
can_splice("baz@1.0+compat", when="@1.0+compat", match_variants="*")
can_splice("quux@1.0", when=@1.1~compat", match_variants="json")
Explanations of the uses of each directive:
- `can_splice("foo@1.0", when="@1.1")`: If `foo@1.0` is the dependency of an already installed spec and `foo@1.1` could be a valid dependency for the parent spec, then `foo@1.1` can be spliced in for `foo@1.0` in the parent spec.
- `can_splice("bar@1.0", when="@1.0+compat")`: If `bar@1.0` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `bar@1.0+compat` in the parent spec
- `can_splice("baz@1.0", when="@1.0+compat", match_variants="*")`: If `baz@1.0+compat` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `baz@1.0+compat` in the parent spec, provided that they have the same value for all other variants (regardless of what those values are).
- `can_splice("quux@1.0", when=@1.1~compat", match_variants="json")`:If `quux@1.0` is the dependency of an already installed spec and `foo@1.1~compat` could be a valid dependency for the parent spec, then `foo@1.0~compat` can be spliced in for `quux@1.0` in the parent spec, provided that they have the same value for their `json` variant.
2. Augmenting the solver to synthesize splices
### Changes to the hash encoding in `asp.py`
Previously, when including concrete specs in the solve, they would have the following form:
installed_hash("foo", "xxxyyy")
imposed_constraint("xxxyyy", "foo", "attr1", ...)
imposed_constraint("xxxyyy", "foo", "attr2", ...)
% etc.
Concrete specs now have the following form:
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "foo", "attr1", ...)
hash_attr("xxxyyy", "foo", "attr2", ...)
This transformation allows us to control which constraints are imposed when we select a hash, to facilitate the splicing of dependencies.
2.1 Compiling `can_splice` directives in `asp.py`
Consider the concrete spec:
foo@2.72%gcc@11.4 arch=linux-ubuntu22.04-icelake build_system=autotools ^bar ...
It will emit the following facts for reuse (below is a subset)
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "hash", "foo", "xxxyyy")
hash_attr("xxxyyy", "version", "foo", "2.72")
hash_attr("xxxyyy", "node_os", "ubuntu22.04")
hash_attr("xxxyyy", "hash", "bar", "zzzqqq")
hash_attr("xxxyyy", "depends_on", "foo", "bar", "link")
Rules that derive abi_splice_conditions_hold will be generated from
use of the `can_splice` directive. They will have the following form:
can_splice("foo@1.0.0+a", when="@1.0.1+a", match_variants=["b"]) --->
abi_splice_conditions_hold(0, node(SID, "foo"), "foo", BaseHash) :-
installed_hash("foo", BaseHash),
attr("node", node(SID, SpliceName)),
attr("node_version_satisfies", node(SID, "foo"), "1.0.1"),
hash_attr("hash", "node_version_satisfies", "foo", "1.0.1"),
attr("variant_value", node(SID, "foo"), "a", "True"),
hash_attr("hash", "variant_value", "foo", "a", "True"),
attr("variant_value", node(SID, "foo"), "b", VariVar0),
hash_attr("hash", "variant_value", "foo", "b", VariVar0).
2.2 Synthesizing splices in `concretize.lp` and `splices.lp`
The ASP solver generates "splice_at_hash" attrs to indicate that a particular node has a splice in one of its immediate dependencies.
Splices can be introduced in the dependencies of concrete specs when `splices.lp` is conditionally loaded (based on the config option `concretizer:splice:True`.
2.3 Constructing spliced specs in `asp.py`
The method `SpecBuilder._resolve_splices` implements a top-down memoized implementation of hybrid splicing. This is an optimization over the more general `Spec.splice`, since the solver gives a global view of exactly which specs can be shared, to ensure the minimal number of splicing operations.
Misc changes to facilitate configuration and benchmarking
- Added the method `Solver.solve_with_stats` to expose timers from the public interface for easier benchmarking
- Added the boolean config option `concretizer:splice` to conditionally load splicing behavior
Co-authored-by: Greg Becker <becker33@llnl.gov>
We added unification semantics for parsing specs from the CLI, but there are a couple
of special cases in which we can avoid calls to the concretizer for speed when the
specs can all be resolved by lookups.
- [x] special case 1: solving a single spec
- [x] special case 2: all specs are either concrete (come from a file) or have an abstract
hash. In this case if concretizer:unify:true we need an additional check to confirm
the specs are compatible.
- [x] add a parameterized test for unifying on the CI
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* libunwind: Add 1.7.2 and 1.8.1
* libunwind: Remove deprecated 1.1 version
* libunwind: Add newer *-stable branches: Remove 1.5-stable branch as well as cleanup.
* libunwind: Use GitHub url for all versions
* libunwind: Add conflict for PPC and 1.8.*
* libunwind: Add conflict for aarch64 and 1.8:
Build fails with
aarch64/Gos-linux.c: In function '_ULaarch64_local_resume':
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
}
^
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
make[2]: *** [Makefile:4795: aarch64/Los-linux.lo] Error 1
* added updated versions
* added pyhmmer
* updated infernal
* fix blast-plus for apple-clang
* fix py-biopython build on apple-clang
* remove erroneous biopython dep: build issue is with python 3.8, not biopython
* deepsig python 3.9: expanding unnecessary python restrictions
* add pyrodigal
* fix unnecessarily strict diamond version
* builds and updates: blast-plus indexing broken, still need to test db download and bakta pipeline
* builds and runs
* revert blast-plus changes: remove my personal hacks to get blast-plus to build
* fix the build error during compilation of rocdecode.was dependent on libva-devel packag
* address review comment
* address review changes.commit the changes
* Add two_level_namespace variant (default is disabled) for MacOS to enable building
executables and libraries with two level namespace enabled.
* Addressed reviewer comments.
* Moved two_level_namespace variant ahead of the patch that uses that variant to
get concretize to work properly.
* Removed extra print statements
* soqt: Add SoQt package
The geomodel package needs this if visualization is turned on.
* make qt versions explicit
* use virtual dependency for qt
* pr feedback
Remove myself as maintainer
Remove v1.6.0
Remove unused qt variant
This addresses part [1] of #46345#44713 introduced a bug where all non-spec query parameters like date
ranges, -x, etc. were ignored when an env was active.
This fixes that issue and adds tests for it.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
`spack mirror add` and `set` now have flags `--oci-password-variable`, `--oci-password-variable`, `--s3-access-key-id-variable`, `--s3-access-key-secret-variable`, `--s3-access-token-variable`, which allows users to specify an environment variable in which a username or password is stored.
Storing plain text passwords in config files is considered deprecated.
The schema for mirrors.yaml has changed, notably the `access_pair` list is generally replaced with a dictionary of `{id: ..., secret_variable: ...}` or `{id_variable: ..., secret_variable: ...}`.
the py-oracledb package only has a single outdated version available in its recipe. this PR adds a much broader range of versions and their corresponding checksums.
* add more versions of py-oracledb
* update py-oracledb recipe
* add py-cython version dependencies
* tweak py-cython version dependencies
* remove older versions of py-oracledb
This filters any selected executable ending with `-ocl` from the list of executables being probed as candidate for external `llvm` installations.
I couldn't reproduce the entire issue, but with a simple script:
```
#!/bin/bash
touch foo.o
echo "clang version 10.0.0-4ubuntu1 "
echo "Target: x86_64-pc-linux-gnu"
echo "Thread model: posix"
echo "InstalledDir: /usr/bin"
exit 0
```
I noticed the executable was still probed:
```
$ spack -d compiler find /tmp/ocl
[ ... ]
==> [2024-11-11-08:38:41.933618] '/tmp/ocl/bin/clang-ocl' '--version'
```
and `foo.o` was left in the working directory. With this change, instead the executable is filtered out of the list on which we run `--version`, so `clang-ocl --version` is not run by Spack.
- [x] Get rid of a call to `parser.quote_if_needed()` during solver setup, which
introduces a circular import and also isn't necessary.
- [x] Rename `spack.variant.Value` to `spack.variant.ConditionalValue`, as it is *only*
used for conditional values. This makes it much easier to understand some of the
logic for variant definitions.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`conditional()`, which defines conditional variant values, and the other ways to declare
variant values should probably be in a layer above `spack.variant`. This does the simple
thing and moves *just* `conditional()` to `spack.directives` to avoid a circular import.
We can revisit the public variant interface later, when we split packages from core.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2024-11-11 01:54:57 -08:00
9585 changed files with 46705 additions and 39588 deletions
# Link above would use kernel32.GetStdHandle(-11) however this would not handle
# a redirected stdout appropriately, so we always refer to the current CONSOLE out
# which is defined as conout$ on Windows.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
