* Normalize Spack Win entrypoints
Currently Spack has multiple entrypoints on Windows that in addition to
differing from *nix implementations, differ from shell to shell on
Windows. This is a bit confusing for new users and in general
unnecessary.
This PR adds a normal setup script for the batch shell while preserving
the previous "click from file explorer for spack shell" behavior.
Additionally adds a shell title to both powershell and cmd letting users
know this is a Spack shell
* remove doskeys
#44588 we added logic to suppress deprecation warnings for the
Intel classic compilers. This depended on matching against
* The compiler names (looking for icc, icpc, ifort)
* The compiler version
When using an Intel compiler with fortran wrappers, the first check
always fails. To support using the fortran wrappers (in combination
with the classic Intel compilers), we remove the first check and
suppress if just the version matches. This works because:
* The newer compilers like icx can handle (ignore) the flags that
suppress deprecation warnings
* The Cray wrappers pass the underlying compiler version (e.g. they
report what icc would report)
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* new openjdk variant to symlink system certificate
* Update var/spack/repos/builtin/packages/openjdk/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Connection objects are Python version, platform and multiprocessing
start method independent, so better to use those than a mix of plain
file descriptors and inadequate guesses in the child process whether it
was forked or not.
This also allows us to delete the now redundant MultiProcessFd class,
hopefully making things a bit easier to follow.
This allows the following
```python
cache.init_entry("my/cache")
with cache.read_transaction("my/cache") as f:
data = f.read() if f is not None else None
```
mirroring `write_transaction`, which returns a tuple `(old, new)` where
`old` is `None` if the cache file did not exist yet.
The alternative that requires less defensive programming on the call
site would be to create the "old" file upon first read, but I did not
want to think about how to safely atomically create the file, and it's
not unthinkable that an empty file is an invalid format (for instance
the call site may expect a json file, which requires at least {} bytes).
This PR is in response to a question in the `environments` slack channel (https://spackpm.slack.com/archives/CMHK7MF51/p1729200068557219) about inadequate CLI help/documentation for one specific subcommand.
This PR uses the approach I took for the descriptions and help for `spack test` subcommands. Namely, I use the first line of the relevant docstring as the description, which is shown per subcommand in `spack env -h`, and the entire docstring as the help. I then added, where it seemed appropriate, help. I also tweaked argument docstrings to tighten them up, make consistent with similar arguments elsewhere in the command, and elaborate when it seemed important. (The only subcommand I didn't touch is `loads`.)
For example, before:
```
$ spack env update -h
usage: spack env update [-hy] env
positional arguments:
env name or directory of the environment to activate
optional arguments:
-h, --help show this help message and exit
-y, --yes-to-all assume "yes" is the answer to every confirmation request
```
After the changes in this PR:
```
$ spack env update -h
usage: spack env update [-hy] env
update the environment manifest to the latest schema format
update the environment to the latest schema format, which may not be
readable by older versions of spack
a backup copy of the manifest is retained in case there is a need to revert
this operation
positional arguments:
env name or directory of the environment
optional arguments:
-h, --help show this help message and exit
-y, --yes-to-all assume "yes" is the answer to every confirmation request
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Commit aa0825d642 accidentally added a semicolon
to the ANSI escape sequence even if the color code was `None` or unknown, breaking the
bold, uncolored font-face. This PR restores the old behavior.
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* Add type hints to all query* methods
* Inline docstrings
* Change defaults from `any` to `None` so they can be type hinted in old Python
* Pre-filter on given hashes instead of iterating over all db specs
* Fix a bug where the `--origin` option of uninstall had no effect
* Fix a bug where query args were not applied when searching by concrete spec
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Fixes a change in behavior/bug in
70412612c7, where partial environment
installs would mark the selected spec as explicitly installed, even if
it was not a root of the environment.
The desired behavior is that roots by definition are the to be
explicitly installed specs. The specs on the `spack -e ... install x`
command line are just filters for partial installs, so leave them
implicitly installed if they aren't roots.
ci: Remove deprecated logic from the ci module
Remove the following from the ci module, schema, and tests:
- deprecated ci stack and handling of old ci config
- deprecated mirror handling logic
- support for artifacts buildcache
- support for temporary storage url
* ParaView: Explicitly set the ENABLE_MPI on/off
* Disallow MPI in the DAG when ~mpi
* @5.13 uses 'remove_children', use pugixml@1.11:, See #47098
* cloud_pipelines/stacks/data-vis-sdk: paraview +raytracing: add +adios2 +fides
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
When building ParaView with ADIOS2 and allowing VTK-m to be
built, also build Fides. This reads ADIOS2 files with a
particular JSON schema, but it requires VTK-m to read data.
* Libmng: Restore Autotools system
CMake, when building its Qt gui, depends on Qt, which in turn, depends on libmng, a CMake based build. To avoid this obvious cyclic dependency, we re-introduce libmng's autotools build into Spack and require when building Qt as a CMake dependency, libmng is built with autotools
* Ensure autotools constraint is limited to non-Windows
* refactor qt-libmng relation from CMake
This commit remove all the uses of spec.compiler that
can be easily substituted by a more idiomatic approach,
e.g. using spec.satisfies or directives
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* py-sphinxcontrib-spelling: new package
* Dependency enchant: Add missing dep on pkgconfig
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Both `multiprocessing.connection.Connection.__del__` and `io.IOBase.__del__` called `os.close` on the same file descriptor. As of Python 3.13, this is an explicit warning. Ensure we close once by usef `os.fdopen(..., closefd=False)`
* stacks: add a stack for devtools on darwin
After getting this whole mess building on darwin, let's keep it that
way, and maybe make it so we have some non-ML darwin binaries in spack
as well.
* reuse: false for devtools
* dtc: fix darwin dylib name and id
On mac the convention is `lib<name>.<num>.dylib`, while the makefile
creates a num suffixed one by default. The id in the file is also a
local name rather than rewritten to the full path, this fixes both
problems.
* node-js: make whereis more deterministic
* relocation(darwin): catch Mach-O load failure
The MachO library can throw an exception rather than return no headers,
this happened in an elf file in the test data of go-bootstrap. Trying
catching the exception and moving on for now. May also need to look
into why we're trying to rewrite an elf file.
* qemu: add darwin flags to clear out warnings
There's a build failure for qemu in CI, but it's invisible because of
the immense mass of warning output. Explicitly specify the target macos
version and remove the extraneous unknown-warning-option flag.
* dtc: libyaml is also a link dependency
libyaml is required at runtime to run the dtc binary, lack of it caused
the ci for qemu to fail when the library wasn't found.
fixes#47101
The bug was introduced in #33495, where `spack find was not updated,
and wasn't caught by unit tests.
Now a Database can accept a custom predicate to select the installation
records. A unit test is added to prevent regressions. The weird convention
of having `any` as a default value has been replaced by the more commonly
used `None`.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* root: fix variant detection for external
A few fixes (possibly non-exhaustive) to `spack external find root`
Several variants have had `when=` clauses added that need to be
propagated to determine_variants. The previously used
Version.satifies("") method also has been removed, it seems. It's
slightly cumbersome that there is no self.spec to use in
determine_variants, but comparisons using Version(version_str) work at least
* remove debug printout
* sccache: add new package
* sccache: add older versions and minimum rust versions
* sccache: add more minimum rust versions
* sccache: add sccache executable and tag as build-tools
* sccache: add dist-server
* sccache: add determine_version and determin_variants
* sccache: add sccache-dist executable
* sccache: fix style
* Update var/spack/repos/builtin/packages/sccache/package.py
* In case building very old sccache <= 5 is not needed with these older rust version is not needed, they can be omitted.
* sccache: drop older versions
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* sccache: add openssl dependency
* sccache: openssl is a linux only dependency?
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Older builds of Boost were failing on Windows because they were
adding --without-... flags for libraries that did not exist in those
versions. So:
* lib variants are updated with version range info (current range
info for libs is not comprehensive, but represents changes over the
last few minor versions up to 1.85)
* On Windows, --without-... options are omitted for libraries when they
don't exist for the version of boost being built. Non-Windows uses
a different approach, which was not affected because the new libraries
were not activated by default. It would benefit from similar attention
though to avoid potential future issues.
#44327 made sure to always run `set_package_py_globals` on all
packages before running `setup_dependent_package` for any package,
so that packages implementing the latter could depend on variables
like `spack_cc` being defined.
This ran into an undocumented dependency: `std_cmake_args` is set in
`set_package_py_globals` and makes use of `cmake_prefix_paths` (if it
is defined in the package); `py-torch`es implementation of
`cmake_prefix_paths` depends on a variable set by
`setup_dependent_package` (`python_platlib`).
This generally restores #44327, and corrects the resulting issue by
moving assignment of `std_cmake_args` to after both actions have been
run.
* py-clip-anytorch: new package
* py-clip-anytorch: ran black
py-langchain-core: ran black
py-pydantic: ran black
py-dalle2-pytorch: ran black
* [py-clip-anytorch] fixed license(checked_by)
* Apply suggestion from Wouter on fixing CI
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Alex C Leute <acl2809@rit.edu>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* py-pytorch-warmup: new package
* py-clip-anytorch: ran black
py-langchain-core: ran black
py-pydantic: ran black
py-dalle2-pytorch: ran black
---------
Co-authored-by: Alex C Leute <acl2809@rit.edu>
Some unused methods in VTK-m resulted in compile errors. These were
not discovered because many compilers ignore unused methods in templated
classes, but the SYCL compiler for Aurora gave an error.
Turns out `os=...` of the spec and `MACOSX_DEPLOYMENT_TARGET` are kept
in sync, and the env variable is used to initialize
`CMAKE_MACOSX_DEPLOYMENT_TARGET`.
In bootstrap code we set the env variable, so these bits are redundant.
---------
Co-authored-by: haampie <haampie@users.noreply.github.com>
* Remove the implicit CORE-AVX512 since the CPU specific flags are added by the
compiler wrappers.
* Add `-i_use-path` to help `ifx` find `lld` even if `-gcc-name` is set in
`ifx.cfg`. This file is written by `intel-oneapi-compilers` package to find the
correct `gcc`. Not being able to find `ldd` is a bug in `ifx`. @rschon2 found
this workaround.
* e4s external rocm ci: upgrade to v6.2.1
* use ghcr.io/spack/spack/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.2.1:2024.10.08
* magma +rocm: add entry for v6.2.1
Extrae normally separates the C and MPI fortran interception libs, but
for mixed C/Fortran applications a combined lib is needed.
Co-authored-by: fpanichi <fpanichi@amd.com>
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* [ftgl] Restrict GCC 14+ patch to apply only to GCC 14+
The patch added by #46927 should only be applied where it is needed:
with GCC 11 it causes a compilation failure where none previously
existed.
* Fix the contraint for applying unsiged char patch to ^freetype@2.13.3:
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
The purpose of this CI job is to ensure that we
can use a modern clingo to concretize specs, if
e.g. it was installed in a virtual environment
with pip.
Since there is no need to re-test unrelated parts
of Spack, reduce the number of tests we run to just
concretize.py
* lua: update luarocks resource to 3.11.1
We have kept an older 3.8 for some time, but that now uses an incorrect
value for the deployment target for macos, causing builds for bundles to
succeed but in such a way that they can't be linked to applications by
`ld` but only loaded by dlopen. This fixes that, and also generally
updates the tool.
* lua-luajit-openresty: add new version fix LUA_PATH
Adds a newer version of openresty's luajor, and adds the slightly odd
extra share path they use that contains the `jit.*` modules. Without
that, things that use bytecode-saving and other jit routines (like
neovim) fail.
* lua-lpeg: fix lpeg build to work for neovim on OSX
Normally luarocks builds all lua libraries as bundles on macos, this
makes sense, but means they can't then be linked by LD into executables
the way neovim expects to do. I'm not sure how this ever worked, if it
did. This patch adds the appropriate variables to have luarocks build
the library as a shared librar, and subsequently fix the id with
install_name_tool (the built-in functionality for this does not
trigger).
This also adds a symlink from `liblpeg.dylib` to `lpeg.so` because
neovim will not build on macos without it. See corresponding upstream
pull request at https://github.com/neovim/neovim/pull/30749
Remove the constraint for concrete specs and simply take the
max(version) if a version is not given. This should default to the
highest infinity version which is also the logical best guess for
doing development.
* Remove concrete verision constriant for develop, set docs
* Add unit-test
* Update lib/spack/docs/environments.rst
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
* Update lib/spack/spack/cmd/develop.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Consolidate env collection in cmd
* Style
---------
Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
Remove the `build-tools` tag of python, otherwise these types of
concretizations are possible:
```
py-root
^py-pip
^python@3.12
^python@3.13
```
So, a package would be configured with py-pip using python 3.12, but
installed for 3.13, which does not work.
* Add new version for master branch
Added new version for master branch. Also added additional functions to ensure tempo will actually run. Tempo assumes the stage directory sticks around and references numerous files and directory there. That has been corrected here only if using the master version. The LWA-10-2020 version will also have this problem but they may have additional setup in their compute/Spack environment to address this issue already so I did not modify anything when that's the version. Example of what happens in the LWA-10-17-2020 version regarding missing files is given below
user@cs:~/spack/bin$ tempo
more: cannot open /tempo.hlp: No such file or directory
* Updated to fix format errors
Flake8 check found errors. Fixed those formatting issues
* Additional format change
Removed redundant setup_dependent_run_environment missed in previous update
* Update url to use https: https is the usual transport and is needed to support checkout behind some firewalls
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
The CMake builder in Spack actually adds incorrect rpaths. They are
unfiltered and incorrectly ordered compared to what the compiler wrapper
adds.
There is no need to specify paths to dependencies in `CMAKE_INSTALL_RPATH`
because of two reasons:
1. CMake preserves "toolchain" rpaths, which includes the rpaths injected
by our compiler wrapper.
2. We use `CMAKE_INSTALL_RPATH_USE_LINK_PATH=ON`, so libraries we link
to are rpath'ed automatically.
However, CMake does not create install rpaths to directories in the package's
own install prefix, so we set `CMAKE_INSTALL_RPATH` to the educated guess
`<prefix>/{lib,lib64}`, but omit dependencies.
Some assertion are not testing DAG invariants, and they are passing only
because of the simple structure of the builtin.mock repository on develop.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* update versions of Neo4j and Redis deps
* deprecating older versions due to security vulnerabilities
* [@spackbot] updating style on behalf of kchilleri
* Update var/spack/repos/builtin/packages/redis/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* adding previous urls to use archives on project websites
* [@spackbot] updating style on behalf of kchilleri
* adding new required maven version
* label when to use specific maven versions
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Krishna Chilleri <krishnachilleri@lanl.gov>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Add versions 1.14.4-3, 1.14.5, develop-1.16, and update develop-1.15 to
develop-1.17.
* Remove unused list_url, list_depth, and style disapproved spaced in url
= "...
* One more style fix.
This PR allows users to configure explicit splicing replacement of an abstract spec in the concretizer.
concretizer:
splice:
explicit:
- target: mpi
replacement: mpich/abcdef
transitive: true
This config block would mean "for any spec that concretizes to use mpi, splice in mpich/abcdef in place of the mpi it would naturally concretize to use. See #20262, #26873, #27919, and #46382 for PRs enabling splicing in the Spec object. This PR will be the first place the splice method is used in a user-facing manner. See https://spack.readthedocs.io/en/latest/spack.html#spack.spec.Spec.splice for more information on splicing.
This will allow users to reuse generic public binaries while splicing in the performant local mpi implementation on their system.
In the config file, the target may be any abstract spec. The replacement must be a spec that includes an abstract hash `/abcdef`. The transitive key is optional, defaulting to true if left out.
Two important items to note:
1. When writing explicit splice config, the user is in charge of ensuring that the replacement specs they use are binary compatible with whatever targets they replace. In practice, this will likely require either specific knowledge of what packages will be installed by the user's workflow, or somewhat more specific abstract "target" specs for splicing, to ensure binary compatibility.
2. Explicit splices can cause the output of the concretizer not to satisfy the input. For example, using the config above and consider a package in a binary cache `hdf5/xyzabc` that depends on mvapich2. Then the command `spack install hdf5/xyzabc` will instead install the result of splicing `mpich/abcdef` into `hdf5/xyzabc` in place of whatever mvapich2 spec it previously depended on. When this occurs, a warning message is printed `Warning: explicit splice configuration has caused the the concretized spec {concrete_spec} not to satisfy the input spec {input_spec}".
Highlighted technical details of implementation:
1. This PR required modifying the installer to have two separate types of Tasks, `RewireTask` and `BuildTask`. Spliced specs are queued as `RewireTask` and standard specs are queued as `BuildTask`. Each spliced spec retains a pointer to its build_spec for provenance. If a RewireTask is dequeued and the associated `build_spec` is neither available in the install_tree nor from a binary cache, the RewireTask is requeued with a new dependency on a BuildTask for the build_spec, and BuildTasks are queued for the build spec and its dependencies.
2. Relocation is modified so that a spack binary can be simultaneously installed and rewired. This ensures that installing the build_spec is not necessary when splicing from a binary cache.
3. The splicing model is modified to more accurately represent build dependencies -- that is, spliced specs do not have build dependencies, as spliced specs are never built. Their build_specs retain the build dependencies, as they may be built as part of installing the spliced spec.
4. There were vestiges of the compiler bootstrapping logic that were not removed in #46237 because I asked alalazo to leave them in to avoid making the rebase for this PR harder than it needed to be. Those last remains are removed in this PR.
Co-authored-by: Nathan Hanford <hanford1@llnl.gov>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* cuda: Add 12.6.2
* Update cuda build system
- Remove gcc@6 conflict that was only a deprecation (probably has be added again with cuda@13)
- Update cuda_arch support by CUDA version
- Kepler support has ended with cuda@12
- Recently added 90a Hopper "experimental" features architecture was
missing the dependency on cuda@12:
adds the lima-vm project, in order to make that useful adds a newer
version of qemu so qemu VMs can work, and builds qemu with flags that
allow it to do things like give the VMs networking and virtfs
filesystems.
Also adds vde as a dependency of qemu.
* CMake: Improve incremental build speed.
CMake automatically embeds an updated configure step into make/ninja that will be called during the build phase. By default if a `CMakeCache.txt` file exists in the build directory CMake will use it and this + `spec.is_develop` is sufficient evidence of an incremental build.
This PR removes duplicate work/expense from CMake packages when using `spack develop`.
* Update cmake.py
* [@spackbot] updating style on behalf of psakievich
* Update cmake.py
meant self not spec...
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
* fix typo in variable name in hepmc3 variant
* set cxx standard to 14 when using protobuf
* add myself to hepmc3 maintainer list
* hepmc3: Applied suggestion of @acecbs for spec.satisfies("+protobuf") (agreed!)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* hepmc3: cxx_standard for protobuf
only set cxx standard to meet protobuf minimum (14) if not also rootio variant as that sets the cxx standard to match the root public API standard requirements
* Environment.clear: ensure clearing is passed through to manifest
* test/cmd/env: make test_remove_commmand round-trip to disk
* cleanup vestigial variables
Some Windows Python installations may store the Python exe in Scripts/
rather than the base directory. Update `.command` to search in both
locations on Windows. On all systems, the search is now done
recursively from the search root: on Windows, that is the base install
directory, and on other systems it is bin/.
* Add libALL support
* cabana: also require ALL
* cabana: Bugfix: Fix spec for cmake>=3.26 to be @3.26: and HDF5 support requires MPI
* cabana: MPI requires C: Add depends_on("c", type="build", when="+mpi")
* cabana: +mpi requires C, but at least for some CMake versions, Cabana's enable of C is too late. Patch it.
* cabana: simplify disabling of find_package's for disabled options and improve comment
* cabana: +grid of 0.6.0 does not compile with gcc-13: It misses iostream includes
* cabana: +test requires googletest at build time: gtest is a linked library(not a plugin or tool)
* cabana: 0.6.0+cuda requires kokkos@3.7:, see https://github.com/ECP-copa/Cabana/releases
* cabana: As 0.6.0+grid does not support gcc-13 and newer, I think it's good to add 0.6.1 and 0.7.0?
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
* added py-evodiff and dependencies
* deleted the FIXME
* fixed style issues
* added versions for biotite dependencies; added hash to py-hatch-vcs
* added python version for py-hatch-cython
* updated biotraj dependencies
* - added versions for the packages and dependencies
- added more dependencies for py-hatch
- added rust versions
- added py-uv as a new package
* updated pacakges and their dependencies according to the PR review by @meyersbs
* typo fix for hatchling version; fix the minimum required setuptools version for evodiff
* added 1.9.0 and 1.7.0 userpath versions; required as a dependency
* added mlflow as a dependency
* changed biopython to an optional dependency according to review from @meyersbs; variant esmfold
* Updated Specs
- Pinned biotraj to 1:1 for py-biotite
- Added numpy and other dependencies for py-biotraj; they are dependent
on the versions
- Excluded py-mlflow as a dependency for package py-evodiff; missing
usage in
package.
- Removed versioned dependencies from py-fair-esm
- Added a version to py-packaging
- Added py-setuptools as a dependency in py-userpath
- Added sha256 as hashes for py-uv
* style changes
* Remove "modify_object_macholib"
According to documentation, this function is used when installing
Mach-O binaries on linux. The implementation seems questionable at
least, and the code seems to be never hit (Spack currently doesn't
support installing Mach-O binaries on linux).
* Fix relocation on macOS, when store projection changes
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Enable tests for symlink-based views (this works with almost no
modifications to the view logic). View logic is not yet robust
for hardlink/junction-based views, so those are disabled for now
(both in the tests and as subcommands to `spack view`).
* add new darshan-runtime variants
- `lustre` variant enables instrumentation of Lustre files
* requires Lustre headers, so Lustre is a proper dependency
for this variant
- `log_path` variant allows setting of a centralized log directory
for Darshan logs (commonly used for facility deployments)
* when this variant is used, the `DARSHAN_LOG_DIR_PATH` env var
is no longer used to set the log file path
- `group_readable_logs` variant sets Darshan log permissions to
allow reads from users in the same group
* add mmap_logs variant to enable usage of mmap logs
Also adds support for Paraview and CMake to build with Qt support on
Windows.
The remaining edits are to enable building of Qt itself on Windows:
* Several packages needed to update `.libs` to properly locate
libraries on Windows
* Qt needed a patch to allow it to build using a Python with a space
in the path
* Some Qt dependencies had not been ported to Windows yet
(e.g. `harfbuzz` and `lcms`)
This PR does not provide a sufficient GL for Qt to use Qt Quick2, as
such Qt Quick2 is disabled on the Windows platform by this PR.
---------
Co-authored-by: Dan Lipsa <dan.lipsa@kitware.com>
* Bugfix/Installer: properly track task queueing
* Move ordinal() to llnl.string; change time to attempt
* Convert BuildTask to use kwargs (after pkg); convert STATUS_ to BuildStats enum
* BuildTask: instantiate with keyword only args after the request
* Installer: build request is required for initializing task
* Installer: only the initial BuildTask cannnot have status REMOVED
* Change queueing check
* ordinal(): simplify suffix determination [tgamblin]
* BuildStatus: ADDED -> QUEUED [becker33]
* BuildTask: clarify TypeError for 'installed' argument
* madgraph5amc: add v3.5.6, add a preferred version and remove urls
* Fix format
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
We run tests for more python versions on `develop` than we do for PRs, so codecov
project status is nearly always failing. There is about a 1% difference in max coverage
between `develop` tests and PR tests, so we should increase the project threshold to 2%
to allow for this difference.
The purpose of the project test on PRs is just to make sure that nothing done on the PR
massively affects coverage of code not covered by the PR. This is valuable, but rare. It
only really affects PRs that deal with test or coverage configuration.
- [x] change project coverage threshold from .2% to 2%
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Add v3.0.0 version for libfirefly package
* [@spackbot] updating style on behalf of tbhaxor
---------
Co-authored-by: tbhaxor <tbhaxor@users.noreply.github.com>
`spack gc` has so far been a global or environment-specific thing.
This adds the ability to restrict garbage collection to specific specs,
e.g. if you *just* want to get rid of all your unused python installations,
you could write:
```console
spack gc python
```
- [x] add `constraint` arg to `spack gc`
- [x] add a simple test
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Avoid that hdf5 searches all search paths for ZLIB.cmake config files (inluding /usr/lib), before it looks for zlib without cmake config files, which is how Spack installs it
* WarpX: 24.10
This updates WarpX and dependencies for the 24.10 release.
New features:
- EB runtime control: we can now compile with EB on by default,
because it is not an incompatible binary option anymore
- Catalyst2 support: AMReX/WarpX 24.09+ support Catalyst2 through
the existing Conduit bindings
* Fix Typo in Variant
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve Python Dep Version Ranges
* Add Missing `-DWarpX_CATALYST`
* AMReX: Missing CMake Options for Vis
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
This allows us to keep the workflow file tidier, and avoid
using indirections to perform platform specific operations.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* py-mpi4py: add v4.0.0
* sensei: update mpi4py dependency
build with py-mpi4py@4.0.0 due to fatal no such file or directory error
* petsc4py: update license, and remove C++/Fortran dependency
There was a bit of mystery surrounding the arguments for `_setup_pkg_and_run`. It passes
two file descriptors for handling the `gmake`'s job server in child processes, but they are
unsed in the method.
It turns out that there are good reasons to do this -- depending on the multiprocessing
backend, these file descriptors may be closed in the child if they're not passed
directly to it.
- [x] Document all args to `_setup_pkg_and_run`.
- [x] Document all arguments to `_setup_pkg_and_run`.
- [x] Add type hints for `_setup_pkg_and_run`.
- [x] Refactor exception handling in `_setup_pkg_and_run` so it's easier to add type
hints. `exc_info()` was problematic because it *can* return `None` (just not
in the context where it's used). `mypy` was too dumb to notice this.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* new builtin package: ambertools
* fixes for the style test
* yet more changes for the style test
* hope this is the last fix for the style test
* netlib-xblas is a dependency, it needs a depends_on("m4", type="build")
* ambertools: Add new setuptool dependency, limit python to <= 3.10 (does not build with 3.11+)
---------
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
We mostly use `spack style` and `spack style --fix`, but it's nice to also be able to
run plain old `black .` in the repo.
- [x] Fix includes and excludes `pyproject.toml` so that we *only* cover files we expect
to be blackened.
Note that `spack style` is still likely the better way to go, because it looks at `git
status` and tells black to only check files that changed from `develop`. `black` with
`pyproject.toml` won't do that. Of course, you can always manually specify which files
you want blackened.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* postgresql: Add icu4c dependency for versions 16+
* postgresql: make ICU an option
* postgresql: ICU variant only needed for v16+
* postgresql: Check for negated option
Check for negated option instead of negating the test
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* [py-flash-attn] Add version 2.6.3
* Update dependencies according to the latest version
* Add max_jobs environmental variable to avoid oom error
---------
Co-authored-by: aurianer <8trash-can8@protonmail.ch>
* Revert "`cc`: ensure that RPATHs passed to linker are unique"
This reverts commit 2613a14c43.
* Revert "`cc`: simplify ordered list handling"
This reverts commit a76a48c42e.
Updated the terminology for the two types of environments to be
consistent with that used in the tutorial for the last three years.
Additionally:
* changed 'anonymous' to 'independent in environment command+test for consistency.
* Update package.py
Update to pull Totalview tar files from AWS instead of requiring the user to download ahead of time. Use new license type, RLM license. Only allow for installs of versions using the new license type. 2024.1 and 2024.2. User selects the platform with the version as it is down from the TotalView downloads website.
* Update package.py
Update to pass style test
* Update package.py
fixing syle
* Updating to pass style check
removing more spaces to pass style check
* final style fixes
fixing the last 2 style errors
* Typo
Typo correction to pass style check
* REmove new line
removing new line character
* Ran black to reformat
Ran black to clear errors
* Changing to use sha256
Updating to use sha256 checksums for all TotalView files.
* acts dependencies: new versions as of 2024/09/30
This commit adds new versions of acts, actsvg, and detray.
* Add vecmem version, patch detray version
#45205 already removed previous use of single letter packages
from unit tests, in view of reserving `c` as a language (see #45191).
Some use of them has been re-introduced accidentally in #46382, and
is making unit-tests fail in the feature branch #45189 since there
`c` is a virtual package.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Kokkos: adding some sanity checks
We can pretty much guarentee that if bin, include or lib directory
is missing, something is wrong. Additionally KokkosCore_config.h
and Kokkos_Core.hpp. I guess technically we could look for all
public headers at least but that seems a bit overkill as well?
* Kokkos Kernels: adding sanity checks
* Remove check for lib directory since it might end up being lib64
* Also remove lib from kokkos-kernels sanity check
* Add latest releases of Camp, RAJA, Umpire, CHAI and CARE
* Address review comments + blt requirement in Umpire
* CARE @develop & @main: Submodules -> False
* Changes in Umpire
* Changes in RAJA
* Changes in CHAI
* Changes in RAJA: prefer 'spec.satisfies' to 'in spec'
This is due to a non-equivalence in Spack with providers like mpi.
See e.g. https://github.com/spack/spack/pull/46126
* Changes in Umpire: prefer 'spec.satisfies' to 'in spec'
This is due to a non-equivalence in Spack with providers like mpi.
See e.g. https://github.com/spack/spack/pull/46126
* Changes in CARE:
Still need to update to CachedCMakePackage based on RADIUSS Spack Configs version
* Missing change in RAJA + changes in fmt
* Fix synta
* Changes in Camp
* Fix style
* CHAI: when ~raja, turn off RAJA in build system
* Fix: Ascent@0.9.3 does not support RAJA@2024.07.0
* Enforce same version constraint on Umpire as for RAJA
* Enforce preferred version of vtk-m in ascent 0.9.3
* Migrate CARE package to CachedCMakePackage
* Fix style in CARE package
* CARE: Apply changes for uniform implementation accross RADIUSS projects
* Caliper: move to CachedCMakePackage, from RADIUSS Spack Configs
* Adapt RAJA Perf to spack CI
* Activate CHAI, CARE and RAJAPerf in Spack CI
* Fixes and diffs with RADIUSS Spack Configs
* Caliper: fix
* Caliper : fix + RAJAPerf : style
* RAJAPerf: fixes
* Update maintainers
* raja-perf: fix license header
* raja-perf: Fix variant naming openmp_target -> omptarget
* raja-perf: style and blt dependency versions
* CARE: benchmark and examples off by default (like tests)
* CARE: fix missing variable
* Update var/spack/repos/builtin/packages/raja-perf/package.py
* CARE: fix branch name
* Revert changes in MFEM to pass CI
* Fix CXX17 condition in RAJA + add sycl option in RAJAPerf
---------
Co-authored-by: Rich Hornung <hornung1@llnl.gov>
* cbindgen: new package
* Attempting to add rust dependencies for cbindgen
* adding rust-toml min rust version
* Removing dependencies that don't install with cargo
* cleanup broken packages
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
On sysroot systems like gentoo prefix, as well as nix/guix, our "is
system path" logic is broken cause it's static.
Talking about "the system paths" is not helpful, we have to talk
about default search paths of the dynamic linker instead.
If glibc is recent enough, we can query the dynamic loader's default
search paths, which is a much more robust way to avoid registering
rpaths to system dirs, which can shadow Spack dirs.
This PR adds an **additional** filter on rpaths the compiler wrapper
adds, dropping rpaths that are default search paths. The PR **does
not** remove any of the original `is_system_path` code yet.
This fixes issues where build systems run just-built executables
linked against their *not-yet-installed libraries*, typically:
```
LD_LIBRARY_PATH=. ./exe
```
which happens in `perl`, `python`, and other non-cmake packages.
If a default path is rpath'ed, it takes precedence over
`LD_LIBRARY_PATH`, and a system library gets loaded instead
of the just-built library in the stage dir, breaking the build. If
default paths are not rpath'ed, then LD_LIBRARY_PATH takes
precedence, as is desired.
This PR additionally fixes an inconsistency in rpaths between
cmake and non-cmake packages. The cmake build system
computed rpaths by itself, but used a different order than
computed for the compiler wrapper. In fact it's not necessary
to compute rpaths at all, since we let cmake do that thanks to
`CMAKE_INSTALL_RPATH_USE_LINK_PATH`. This covers rpaths
for all dependencies. The only install rpaths we need to set are
`<install prefix>/{lib,lib64}`, which cmake unfortunately omits,
although it could also know these. Also, cmake does *not*
delete rpaths added by the toolchain (i.e. Spack's compiler
wrapper), so I don't think it should be controversial to simplify
things.
https://docs.sylabs.io/guides/main/admin-guide/configfiles.html#loop-devices
shared loop devices: This allows containers running the same image
to share a single loop device. This minimizes loop device usage and
helps optimize kernel cache usage.
Enabling this feature can be particularly useful for large MPI jobs.
The current `Spec.splice` model is very limited by the inability to splice specs that
contain multiple nodes with the same name. This is an artifact of the original
algorithm design predating the separate concretization of build dependencies,
which was the first feature to allow multiple specs in a DAG to share a name.
This PR provides a complete reimplementation of `Spec.splice` to avoid that
limitation. At the same time, the new algorithm ensures that build dependencies
for spliced specs are not changed, since the splice by definition cannot change
the build-time information of the spec. This is handled by splitting the dependency
edges and link/run edges into separate dependencies as needed.
Signed-off-by: Gregory Becker <becker33@llnl.gov>
* CI: Add documentation for adding new stacks and runners
* More docs for runner registration
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
Co-authored-by: Bernhard Kaindl <contact@bernhard.kaindl.dev>
This PR shorten the string representation for concrete specs,
in order to make it more legible.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
macOS Sequoia's linker will complain if RPATHs on the CLI are specified more than once.
To avoid errors due to this, make `cc` only append unique RPATHs to the final args list.
This required a few improvements to the logic in `cc`:
1. List functions in `cc` didn't have any way to append unique elements to a list. Add a
`contains()` shell function that works like our other list functions. Use it to implement
an optional `"unique"` argument to `append()` and an `extend_unique()`. Use that to add
RPATHs to the `args_list`.
2. In the pure `ld` case, we weren't actually parsing `RPATH` arguments separately as we
do for `ccld`. Fix this by adding *another* nested case statement for raw `RPATH`
parsing. There are now 3 places where we deal with `-rpath` and friends, but I don't
see a great way to unify them, as `-Wl,`, `-Xlinker`, and raw `-rpath` arguments are
all ever so slightly different.
3. Fix ordering of assertions to make `pytest` diffs more intelligible. The meaning of
`+` and `-` in diffs changed in `pytest` 6.0 and the "preferred" order for assertions
became `assert actual == expected` instead of the other way around.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`cc` divides most paths up into system paths, spack managed paths, and other paths.
This gets really repetitive and makes the code hard to read. Simplify the script
by adding some functions to do most of the redundant work for us.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* axom/stand-alone tests: build and run in test stage directory
* Removed unused glob
* axom/stand-alone tests: add example_stage_dir variable for clarity
SimpleFilesystemView was producing an error due to looking for a
<prefix>/lib/.spack folder. Also, view_destination had no effect and
wasn't called. Changed this by instead patching in the correct
installation prefix for dictionaries.
Since aspell is using the resolved path of the executable prefix, the
runtime environment variable ASPELL_CONF is set to correct the prefix
when in a view. With this change aspell can now find installed
dictionaries. Verified with:
aspell dump config
aspell dump dicts
* shorten version number validations per reviewer feedback
* rename set_lib_path per reviewer feedback
* Add E4S tag
* Set CHPL_CUDA_PATH to ensure Chapel installer finds the right package
* Update ROCm dependency for Chapel 2.2
* Fix llvm-amdgpu and CHPL_TARGET_* for llvm=bundled
* Ensure CHPL_TARGET_COMPILER is set to "llvm" when required (llvm=spack
or +cuda or +rocm).
* Ensure CHPL_TARGET_{CC,CXX} are only set when using llvm=spack or llvm=none
* Use hip.prefix to set CHPL_ROCM_PATH
Since we might not directly depend on llvm-amdgpu, thus it might
not appear in our spec
* limit m4 dependency to +gmp
* limit names of env vars created from variants
* Ensure that +cuda and +rocm variants are Sticky
The concretizer should never be permitted to select GPU support, because
it's only meaningful and functional when the appropriate hardware is actually
available, and the concretizer cannot reliably determine that.
Also: Chapel's GPU support includes alot of complicated dependencies
and constraints, so leaving that choice free to the concretizer leads to alot
of extraneous and confusing messages when failing to concretize a
non-GPU-enabled spec.
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
Add pre-built sbcl for x86 and arm for various glibc versions, making
way for an actual sblc built from source.
Also switch to use set_env in a context manager over setting the
environment variable for the build environment. I hit an issue with the
build system due to this in the sbcl package, pre-empting the same issue
here.
* dla-future: Add DLAF_ prefix to LAPACK_LIBRARY CMake variable in newer versions
* dla-future: Use spec.satisfies to check version constraint for LAPACK_LIBRARY variable prefix
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
---------
Co-authored-by: Alberto Invernizzi <9337627+albestro@users.noreply.github.com>
* py-sphinx-tabs: new version 3.4.5
* py-sphinx-design: new versions 0.5.0, 0.6.0, and 0.6.1
* py-requests: new version 2.32.3
* py-dnspython: new version 2.6.1
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* py-hatch-vcs: new version 0.4.0
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Python 3.12 removed the `distutils` module, which is being required
by the build process of LLVM <= 14: Conflict with it for +python.
Fix build to not pick host tools like an incompatible Python from host
Co-authored-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
gcc on ubuntu has fix-cortex-a53-843419 set by default - this causes linking
issues (symbol relocation errors) for tf, even when compiling for different
cpus.
If `add_padding()` is allowed to return a path with a trailing path
separator, it will get collapsed elsewhere in Spack. This can lead to
buildcache entries that have RPATHS that are too short to be replaced by
other users whose install root happens to be padded to the correct
length. Detect this and replace the trailing path separator with a
concrete path character.
Signed-off-by: Samuel E. Browne <sebrown@sandia.gov>
Also: set the build and install directories to the source directory
because the build system unfortunately expects the `src_ext` directory
to be under the current working directory when building the bundled
third-party libraries, even when the configure script is run from
another directory.
@scemama pointed out that 'make' just calls 'dune' which is already
parallel, so make itself should not have more than one job.
opam@:2.1 need 'make lib-ext' for cmdliner, above it's obsolete.
Co-authored-by: Bernhard Kaindl <bernhard.kaindl@cloud.com>
The detection logic for the prefix used in py-bind11 if broken for spack
resulting in an empty prefix. However, the package provides an escape
hatch in the form of `prefix_for_pc_file`. Use this escape hatch to
provide the correct path; spack will always know better than pybind11's
CMake.
Co-authored-by: Robert Underwood <runderwood@anl.gov>
We've seen `getfqdn()` cause slowdowns on macOS in CI when added elsewhere. It's also
called by database.py every time we write the DB file.
- [x] replace the call with a memoized version so that it is only called once per process.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
This PR introduces a new heuristic for the solver, which behaves better when
compilers are treated as nodes. Apparently, it performs better also on `develop`,
where compilers are still node attributes.
The new heuristic:
- Sets an initial priority for guessing a few attributes. The order is "nodes" (300),
"dependencies" (150), "virtual dependencies" (60), "version" and "variants" (30), and
"targets" and "compilers" (1). This initial priority decays over time during the solve, and
falls back to the defaults.
- By default, it considers most guessed facts as "false". For instance, by default a node
doesn't exist in the optimal answer set, or a version is not picked as a node version etc.
- There are certain conditions that override the default heuristic using the _priority_ of
a rule, which previously we didn't use. For instance, by default we guess that a
`attr("variant", Node, Variant, Value)` is false, but if we know that the node is already
in the answer set, and the value is the default one, then we guess it is true.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This includes a test_linux699 variant which "activates" a version
that pulls from a repository other than the official repository.
This version is required to work with Linux kernel version
6.9.9 or later. Future official `msr-safe` versions are expected
to support later Linux kernel versions.
* opendatadetector: Add an env variable pointing to the share directory
* Rename the new variable to OPENDATADETECTOR_DATA and use join_path
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
The `spack.target.Target` class is a weird entity, that is just needed to:
1. Sort microarchitectures in lists deterministically
2. Being able to use microarchitectures in hashed containers
This PR removes it, and uses `archspec.cpu.Microarchitecture` directly. To sort lists, we use a proper `key=` when needed. Being able to use `Microarchitecture` objects in sets is achieved by updating the external `archspec`.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Introduce the bufr_query library from NOAA-EMC (#461)
This PR adds in a new package.py script for the new bufr_query library from NOAA-EMC. This is being used by JEDI and other applications.
* Add explicit build dependency spec to the pybind11 depends_on spec
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Convert patch file to the URL form which pulls the changes from github.
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Added new version (0.0.3) and removed obsolete site-packages.patch file
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
While the existing getting started guide does in fact reference the
powershell support, it's a footnote and easily missed. This PR adds
explicit, upfront mentions of the powershell support. Additionally
this PR adds notes about some of the issues with certain components
of the spec syntax when using CMD.
If the spec is external, it has extra attributes. If not, we know
which names are used. In both cases we don't need to search again
for executables.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Removes `spack.package_base.PackageBase.do_{install,deprecate}` in favor of
`spack.installer.PackageInstaller.install` and `spack.installer.deprecate` resp.
That drops a dependency of `spack.package_base` on `spack.installer`, which is
necessary to get rid of circular dependencies in Spack.
Also change the signature of `PackageInstaller.__init__` from taking a dict as
positional argument, to an explicit list of keyword arguments.
Continuing the work started in #40326, his changes the structure
of Variant metadata on Packages from a single variant definition
per name with a list of `when` specs:
```
name: (Variant, [when_spec, ...])
```
to a Variant definition per `when_spec` per name:
```
when_spec: { name: Variant }
```
With this change, everything on a package *except* versions is
keyed by `when` spec. This:
1. makes things consistent, in that conditional things are (nearly)
all modeled in the same way; and
2. fixes an issue where we would lose information about multiple
variant definitions in a package (see #38302). We can now have,
e.g., different defaults for the same variant in different
versions of a package.
Some notes:
1. This required some pretty deep changes to the solver. Previously,
the solver's job was to select value(s) for a single variant definition
per name per package. Now, the solver needs to:
a. Determine which variant definition should be used for a given node,
which can depend on the node's version, compiler, target, other variants, etc.
b. Select valid value(s) for variants for each node based on the selected
variant definition.
When multiple variant definitions are enabled via their `when=` clause, we will
always prefer the *last* matching definition, by declaration order in packages. This
is implemented by adding a `precedence` to each variant at definition time, and we
ensure they are added to the solver in order of precedence.
This has the effect that variant definitions from derived classes are preferred over
definitions from superclasses, and the last definition within the same class sticks.
This matches python semantics. Some examples:
```python
class ROCmPackage(PackageBase):
variant("amdgpu_target", ..., when="+rocm")
class Hipblas(ROCmPackage):
variant("amdgpu_target", ...)
```
The global variant in `hipblas` will always supersede the `when="+rocm"` variant in
`ROCmPackage`. If `hipblas`'s variant was also conditional on `+rocm` (as it probably
should be), we would again filter out the definition from `ROCmPackage` because it
could never be activated. If you instead have:
```python
class ROCmPackage(PackageBase):
variant("amdgpu_target", ..., when="+rocm")
class Hipblas(ROCmPackage):
variant("amdgpu_target", ..., when="+rocm+foo")
```
The variant on `hipblas` will win for `+rocm+foo` but the one on `ROCmPackage` will
win with `rocm~foo`.
So, *if* we can statically determine if a variant is overridden, we filter it out.
This isn't strictly necessary, as the solver can handle many definitions fine, but
this reduces the complexity of the problem instance presented to `clingo`, and
simplifies output in `spack info` for derived packages. e.g., `spack info hipblas`
now shows only one definition of `amdgpu_target` where before it showed two, one of
which would never be used.
2. Nearly all access to the `variants` dictionary on packages has been refactored to
use the following class methods on `PackageBase`:
* `variant_names(cls) -> List[str]`: get all variant names for a package
* `has_variant(cls, name) -> bool`: whether a package has a variant with a given name
* `variant_definitions(cls, name: str) -> List[Tuple[Spec, Variant]]`: all definitions
of variant `name` that are possible, along with their `when` specs.
* `variant_items() -> `: iterate over `pkg.variants.items()`, with impossible variants
filtered out.
Consolidating to these methods seems to simplify the code a lot.
3. The solver does a lot more validation on variant values at setup time now. In
particular, it checks whether a variant value on a spec is valid given the other
constraints on that spec. This allowed us to remove the crufty logic in
`update_variant_validate`, which was needed because we previously didn't *know* after
a solve which variant definition had been used. Now, variant values from solves are
constructed strictly based on which variant definition was selected -- no more
heuristics.
4. The same prevalidation can now be done in package audits, and you can run:
```
spack audit packages --strict-variants
```
This turns up around 18 different places where a variant specification isn't valid
given the conditions on variant definitions in packages. I haven't fixed those here
but will open a separate PR to iterate on them. I plan to make strict checking the
defaults once all existing package issues are resolved. It's not clear to me that
strict checking should be the default for the prevalidation done at solve time.
There are a few other changes here that might be of interest:
1. The `generator` variant in `CMakePackage` is now only defined when `build_system=cmake`.
2. `spack info` has been updated to support the new metadata layout.
3. split out variant propagation into its own `.lp` file in the `solver` code.
4. Add better typing and clean up code for variant types in `variant.py`.
5. Add tests for new variant behavior.
Historically, every PR, push, etc. to Spack generates a bunch of jobs, each of which
uploads its coverage report to codecov independently. This means that we get annoying
partial coverage numbers when only a few of the jobs have finished, and frequently
codecov is bad at understanding when to merge reports for a given PR. The numbers of the
site can be weird as a result.
This restructures our coverage handling so that we do all the merging ourselves and
upload exactly one report per GitHub actions workflow. In practice, that means that
every push to every PR will get exactly one coverage report and exactly one coverage
number reported. I think this will at least partially restore peoples' faith in what
codecov is telling them, and it might even make codecov handle Spack a bit better, since
this reduces the report burden by ~7x.
- [x] test and audit jobs now upload artifacts for coverage
- [x] add a new job that downloads artifacts and merges coverage reports together
- [x] set `paths` section of `pyproject.toml` so that cross-platform clone locations are merged
- [x] upload to codecov once, at the end of the workflow
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* kokkos, kokkos-kernels, kokkos-nvcc-wrapper: add v4.4.01
* trilinos: update @[master,develop] dependency on kokkos
==> Error: InstallError: For Trilinos@[master,develop], ^kokkos version in spec must match version in Trilinos source code. Specify ^kokkos@4.4.01 for trilinos@[master,develop] instead of ^kokkos@4.4.00.
* petsc: configure requires rocm-core/rocm_version.h to detect ROCM_VERSION_MAJOR.ROCM_VERSION_MINOR.ROCM_VERSION_PATCH
* mfem: add dependency on rocprim (as needed via petsc dependency)
In file included from linalg/petsc.cpp:19:
In file included from linalg/linalg.hpp:65:
In file included from linalg/petsc.hpp:48:
In file included from /scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/petsc-3.22.0-7dsxwizo24ycnqvwnsscupuh4i7yusrh/include/petscsystypes.h:530:
In file included from /scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/rocthrust-6.1.2-ux5nmi4utw27oaqmz3sfjmhb6hyt5zed/include/thrust/complex.h:30:
/scratch/svcpetsc/spack.x/opt/spack/linux-ubuntu22.04-x86_64/gcc-11.4.0/rocthrust-6.1.2-ux5nmi4utw27oaqmz3sfjmhb6hyt5zed/include/thrust/detail/type_traits.h:29:10: fatal error: 'rocprim/detail/match_result_type.hpp' file not found
29 | #include <rocprim/detail/match_result_type.hpp>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Update seacas package.py
Adding libcatalyst variant to seacas package
When seacas is installed with "seacas +libcatalyst"
then a dependency on the spack package "libcatalyst"
(which is catalyst api 2 from kitware) is added, and
the appropriage cmake variable for the catalyst TPL
is set. The mpi variant option in catalyst (i.e. build
with mpi or build without mpi) is passed on to
libcatalyst. The default of the libcatalyst variant
is false/off, so if seacas is installed without the
"+libcatalyst" in the spec it will behave exactly as
it did before the introduction of this variant.
* shortened line 202 to comply with < 100 characters per line style requirement
* py-httpx: New version
* [py-httpx] fix when for dependencies
* [py-httpx] organized dependencies
* [py-httpx] added version 0.27.2
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
* Automated deployment to update package flux-sched 2024-09-05
* flux-sched: add back check for run environment
* flux-sched: add conflict for gcc and clang above 0.37.0
---------
Co-authored-by: github-actions <github-actions@users.noreply.github.com>
autotools packages with a configure script should generate the libtool
executable, there's no point in `depends_on("libtool", type="build")`.
the libtool executable in `<libtool prefix>/bin/libtool` is configured
for the wrong toolchain (libtools %compiler instead of the package's
%compiler).
Some package link to `libltdl.so`, which is fine, but had a wrong
dependency type.
See https://github.com/spack/spack/pull/46314#discussion_r1752940332.
This further simplifies `cxxstd` variant handling in `acts` by removing superfluous
version constraints from dependencies for `geant4` and `root`.
The version constraints in the loop are redundant with the conditional variant
values here:
```python
_cxxstd_values = (
conditional("14", when="@:0.8.1"),
conditional("17", when="@:35"),
conditional("20", when="@24:"),
)
_cxxstd_common = {
"values": _cxxstd_values,
"multi": False,
"description": "Use the specified C++ standard when building.",
}
variant("cxxstd", default="17", when="@:35", **_cxxstd_common)
variant("cxxstd", default="20", when="@36:", **_cxxstd_common)
```
So we can simplify the dependencies in the loop to:
```python
for _cxxstd in _cxxstd_values:
for _v in _cxxstd:
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +geant4")
depends_on(f"geant4 cxxstd={_v.value}", when=f"cxxstd={_v.value} +fatras_geant4")
depends_on(f"root cxxstd={_v.value}", when=f"cxxstd={_v.value} +tgeo")
```
And avoid the potential for impossible variant expressions.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Openmpi provider statements were changed in #46102. The package change
was fine in and of itself, but apparently one of our tests depends on
the precise constraints used in those statements. I updated the test
to remove the checks for constraints that were removed.
In #44425, we add stricter variant audits that catch expressions that can never match.
This fixes 13 packages that had this type of issue.
Most packages had lingering spec expressions from before conditional variants with
`when=` statements were added. For example:
* Given `conflicts("+a", when="~b")`, if the package has since added
`variant("a", when="+b")`, the conflict is no longer needed, because
`a` and `b` will never exist together.
* Similarly, two packages that depended on `py-torch` depended on
`py-torch~cuda~cudnn`, which can't match because the `cudnn` variant
doesn't exist when `cuda` is disabled. Note that neither `+foo` or `~foo`
match (intentionally) if the `foo` variant doesn't exist.
* Some packages referred to impossible version/variant combinations, e.g.,
`ceed@1.0.0+mfem~petsc` when the `petsc` variant only exist at version `2`
or higher.
Some of these correct real issues (e.g. the `py-torch` dependencies would have never
worked). Others simply declutter old code in packages by making all constraints
consistent with version and variant updates.
The only one of these that I think is not all that useful is the one for `acts`,
where looping over `cxxstd` versions and package versions ends up adding some
constraints that are impossible. The additional dependencies could never have
happened and the code is more complicated with the needed extra constriant.
I think *probably* the best thing to do in `acts` is to just not to use a loop
and to write out the constraints explicitly, but maybe the code is easier to
maintain as written.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* Update var/spack/repos/builtin/packages/fms/package.py: apply patch for fms@2023.03 to fix compiler options bug in cmake config, add variant shared and corresponding patch for fms@2024.02
* Fix fms package audit: use c9bba516ba.patch?full_index=1 instead of c9bba516ba.patch?full_index=1
* Update checksum of patch for fms@2023.03
* CUDA: support Grace Hopper 9.0a compute capability
* Fix other packages
* Add type annotations
* Support ancient Python versions
* isort
* spec -> self.spec
Co-authored-by: Andrew W Elble <aweits@rit.edu>
* [@spackbot] updating style on behalf of adamjstewart
---------
Co-authored-by: Andrew W Elble <aweits@rit.edu>
Co-authored-by: adamjstewart <adamjstewart@users.noreply.github.com>
fixes#46295
A proper solution would be a tag directive that accumulates tags
with the ones defined in base classes.
For the time being, rewrite them explicitly.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Boost:Adjust bootstrapping/b2 options as needed for Windows (the
bootstrapping phase sufficiently differs between Windows/Unix
that it is handled entirely within its own branch).
* Boost: Paths in user-config.jam should be POSIX, including on Windows
* Python: `.libs` for the Python package should return link libraries
on Windows. The libraries are also stored in a different directory.
The option config:install_missing_compilers is currently buggy,
and has been for a while. Remove it, since it won't be needed
when compilers are treated as dependencies.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fast-float: new package
* fast-float: add test dependency
* fast-float: fix doctest dependency type
* fast-float: convert deps to tuple
* fast-float: add v6.1.5 and v6.1.6
* fast-float: patch older versions to use find_package(doctest)
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
* Changed software pull location
The original package pulled a tagged release version from GitHub. That tagged version was created in 2022 and has not been updated since. It no longer runs because newer versions of numpy have removed deprecation warnings for several of their calls. The main branch for this repository has addressed these numpy issues as well as some other important fixes but no new release has been generated. Because of this and the apparent minimal development that now appears to be going on, it is probably best to always pull from the main branch
* [@spackbot] updating style on behalf of aweaver1fandm
* py-your: Changed software pull location
1. Restored original URL and version (0.6.7) as requested
2. Updated py-numpy dependency versions to be constrained based on the version of your. Because of numpy deprecations related to your version 0.6.7 need to ensure that the numpy version used is not 1.24 or greater because the depracations were removed starting with that version
* gptune: new test API
* gptune: cleanup; finish API changes; separate unrelated test parts
* gptune: standalone test cleanup with timeout constraints
* gptune: ensure stand-alone test bash failures terminate; enable in CI
* gptune: add directory to terminate_bash_failures
* gptune/stand-alone tests: use satisifes for checking variants
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add numactl 2.0.16-2.0.18
* Create link-with-latomic-if-needed-v2.0.16.patch
Add a link to libatomic, if needed, for numactl v2.0.16.
* Add some missing patches to v2.0.16
* Create numactl-2.0.18-syscall-NR-ppc64.patch
In short, we need numactl to set __NR_set_mempolicy_home_node on ppc64, if it's not already defined.
* Apply a necessary patch for v2.0.18 on PPC64
* Add libatomic patch for v2.0.16
`spack reindex` relies on projections from configuration to locate
installed specs and prefixes. This is problematic because config can
change over time, and we have reasons to do so when turning compilers
into depedencies (removing `{compiler.name}-{compiler.version}` from
projections)
This commit makes reindex recursively search for .spack/ metadirs.
* py-httpcore: Added new version
* [py-httpcore]
- added version 0.18.0
- restructured dependencies as everything has a when and
type/when ordering was all over the place
* [py-httpcore] ordered dependencies in the order listed in v1.0.5 pyproject.toml
---------
Co-authored-by: Alex C Leute <aclrc@rit.edu>
When a package is running `setup_dependent_package` on a parent, ensure
that module variables like `spack_cc` are available. This was often
true prior to this commit, but externals were an exception.
---------
Co-authored-by: John Parent <john.parent@kitware.com>
* New package: py-monai
* Fixed linked issues with py-monai
* [py-monai] removed extra line
* [py-monai]
- New version 1.3.2
- ran black
- update copyright
* [py-monai] added license
* [py-monai]
- moved build only dependencies
- specified newer python requirements for newer versions
---------
Co-authored-by: vehrc <vehrc@rit.edu>
* Replace if ... in spec with spec.satisfies in f* and g* packages
* gromacs: ^amdfftw -> ^[virtuals=fftw-api] amdfftw
* flamemaster: add virtuals lapack for the amdlibflame dependency
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Allow flags from different sources (compilers, `require:`, command-line
specs, and `depends_on`) to be merged together, and enforce a consistent
order among them.
The order is based on the sources, e.g. flags on specs from the command
line always come last. Some flag order consistency issues are fixed:
1. Flags from `compilers.yaml` and the command line were always intra- and
inter-source order consistent.
2. Flags from dependents and packages.yaml (introduced via `require:`)
were not: for `-a -b` from one source and `-c` from another, the final
result might rearrange `-a -b`, and would also be inconsistent in terms
of whether `-c` came before or after.
(1) is/was handled by going back to the original source, i.e., flags are
retrieved directly from the command line spec rather than the solver.
(2) is addressed by:
* Keeping track of grouped flags in the solver
* Keeping track of flag sources in the solver on a per-flag basis
The latter info is used in this PR to enforce DAG ordering on flags
applied from multiple dependents to the same package, e.g., for this
graph:
```
a
/|\
b | c
\|/
d
```
If `a`, `b`, and `c` impose flags on `d`, the combined flags on `d` will
contain the flags of `a`, `b`, and `c` -- in that order.
Conflicting flags are allowed (e.g. -O2 and -O3). `Spec.satisifes()` has
been updated such that X satisfies Y as long as X has *at least* all of
the flags that Y has. This is also true in the solver constraints.
`.satisfies` does not account for how order can change behavior (so
`-O2 -O3` can satisfy `-O3 -O2`); it is expected that this can be
addressed later (e.g. by prohibiting flag conflicts).
`Spec.constrain` and `.intersects` have been updated to be consistent
with this new definition of `.satisfies`.
Spack can now bootstrap two new dependencies on Windows: GnuPG, and file.
These dependencies are modeled as a separate package, and they install a cross-compiled binary.
Details on how they binaries are built are in https://github.com/spack/windows-bootstrap-resources
This PR adds py-pybind11 versions 2.13.0, 2.13.1, 2.13.2, 2.13.3, and
2.13.4. It also adds a new conflict between gcc 14 and pybind versions
up to and including 2.13.1.
* Allow deprecating more than one property in config
This internal change allows the customization of errors
and warnings to be printed when deprecating a property.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* fix
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Use a list comprehension for "issues"
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
fixes#40791
Currently stacks behave differently if used in unify:false
environments, which leads to inconsistencies during concretization.
For instance, we might have two abstract user specs that do not
intersect with each other map to the same concrete spec in the
environment. This is clearly wrong.
This PR removes the best effort expansion, so that user specs
are always applied strictly.
* gaudi: Specify boost components and add +fiber for v39
* gaudi: Limit fmt version to allow building master branch
* Make boost dependencies a bit more readable
* Remove patches for no longer existing versions
* Replace if ... in spec with spec.satisfies in d* and e* packages
* Use virtuals for different mpi implementations in esmf
* esmf: ^[virtuals=mpi] mpt
* extrae: ^[virtuals=mpi] intel-oneapi-mpi
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
This change aligns the build condition for parmetis with the
depends_on condition.
The current build condition of parmetis looks for "+parmetis" in
the spec which is not added by the depends_on as that adds
"^parmetis" instead.
* Adding addtional check for omptarget library for amdgpu in nvidia environment
* Avoiding registration of duplicate when built on cuda
* Adding hsa library path in LD_LIBRARY_PATH
* Correction in hsa prefix library path in LD_LIBRARY_PATH
==> Error: InstallError: For Trilinos@[master,develop], ^kokkos version in spec must match version in Trilinos source code. Specify ^kokkos@4.4.00 for trilinos@[master,develop] instead of ^kokkos@4.3.01.
* root: Add dependency on libglx
We have been trying to build the Acts package on MacOS, and in this
process we have been running into problems with the ROOT spec on that
operating system; the primary issue we are encountering is that the
compiler is unable to find the `GL/glx.h` header, which is part of glx.
It seems, therefore, that ROOT depends on libglx, but this is not
currently encoded in the spec. This commit ensures that ROOT depends on
the virtual libglx package when both the OpenCL and X11 variants are
enabled.
* Enable builtin glew on MacOS
* Allow `root+opengl+aqua~x` on macOS
dd4hep versions up to and including 1.27 had a conflict with root
versions starting from 6.31.1, as shown in
https://github.com/AIDASoft/DD4hep/issues/1210. This PR explicitly adds
that conflict to the spec.
* whizard: add a patch when using hepmc3 3.3.0 or newer
* whizard: comment with patch origin
---------
Co-authored-by: jmcarcell <jmcarcell@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* podio: apply patch for gcc 14 builds
Podio versions after 0.17.0 but before 1.0.0 are broken when using gcc
14 because of a missing include, which is addressed in the podio pull
request at https://github.com/AIDASoft/podio/pull/600. This commit
patches pre-1.0.0 versions of Podio so they can be compiled with gcc 14,
which is important for building Acts.
* Style
* Style 2
* Fixes
* Add comment:
* Add sha256
This should help not selecting, by default, some niche implementation that are supposed to be externals.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This PR simplifies the code doing external spec detection by removing
the `DetectedPackage` class. Now, functions accepting or returning lists
of `DetectedPackage`, will accept or return list of specs.
Performance doesn't seem to change if we use `Spec.__reduce__` instead
of `DetectionPackage.__reduce__`.
- Honor compiler extra_rpaths (this build bypasses spack wrappers,
so the RPATHs are added explicitly as CMake options)
- Use root_cmakelists_dir instead of adding it to the command line
- Add BLT as a dependency, allowing different versions outside of what
is in the tarball
- Put a copy of host-config in the stage directory: this allows
examination of the host-config when a build fails (before, the
host-config was just stored in the install directory, which is
deleted by default on a failed build)
* Add conflict between NVSHMEM and cuFFTMp for GROMACS package
These don't work in the same build configuration.
* [@spackbot] updating style on behalf of mabraham
* Update package.py
Also constrain NVSHMEM appropriately
* Update var/spack/repos/builtin/packages/gromacs/package.py
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
* Update package.py
* [@spackbot] updating style on behalf of mabraham
* Update package.py
* Update package.py
* Update package.py
* [@spackbot] updating style on behalf of mabraham
---------
Co-authored-by: mabraham <mabraham@users.noreply.github.com>
Co-authored-by: Christoph Junghans <christoph.junghans@gmail.com>
Co-authored-by: Andrey Alekseenko <al42and@gmail.com>
Change the signature of Spec.from_detection to set the
external prefix, and the external modules, if they are
present.
Delete "spack.package_prefs.spec_externals" since it
is unused.
This context manager was used to prevent detecting a platform
as "cray". Since now Cray machines are detected as linux, we can
remove the context manager.
This commit adds version 0.73.0 of the detray package. As this version
drops support for pre-C++20 standards, I had to update the `cxxstd`
variant logic.
Right now if you run `spack ci generate` you get a warning about CDash credentials even
if there's no CDash configuration specified. We should only warn if there was actually a
CDash config.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Source mirrors store entries by digest and add a human readable alias of the
form 'name-version'. If no digest is available, the alias is used as the primary
storage location.
Spack erroneously fetches by alias when the digest path does not exist. This is
problematic if `version(..., sha256=...)` changes in package.py, and the mirror
is populated with the old shasum. That would result in an error when a digest
is available, but in case of git versions with a modified commit sha, the wrong
sources would be fetched without error. With this PR, only the digest path is
used, not the alias, in case a digest is available. This is also a small performance
optimization, as the number of request is halved for mirrors that don't contain
the sources.
Further, for git sources the tag was used as a digest, but this is a moving
target. Only commit sha is used now.
Also whenever the alias already existed, Spack used to keep it in place when
updating the mirror cache, which means that aliases would always point to
outdated mirror entries whenever digests are modified. With this PR the alias
is moved in place.
Lastly, fix a recent regression where `Stage.disable_mirrors` disabled mirrors
but not the local download cache, which was the intention.
* VTK-m: Point to github mirror for source tarball
The gitlab.kitware.com source location seems to have intermittent
network issues. Switching the to mirror hosted at Github may alleviate
some of the timeouts.
* Update sha256 for GitHub tarballs
---------
Co-authored-by: Zack Galbreath <zack.galbreath@kitware.com>
* davix: add versions 0.8.2-0.8.7 and dependencies
This commit adds new versions 0.8.2-0.8.7 of the davix package, and it
also improves the handling of embedded packages. Davix will try to build
libcurl from its own embedded version of that code, which doesn't mesh
well with Spack's design philosophy, so I've changed the CMake
configuration to disallow the builtin libcurl and use a Spack dependency
instead. Up to version 0.8.7, RapidJSON was also builtin, but version
0.8.7 allows users to specify that they want to use a pre-installed
version of RapidJSON, so this commit also adds that as a dependency for
versions 0.8.7:.
* Fix old versions
* acts: add v36.0.0, v36.1.0 and fixes
This commit makes several changes to the Acts repository, namely:
1. It adds versions 36.0.0 and 36.1.0.
2. It adds the traccc plugin and related dependencies.
3. It updates the version requirements of some dependencies.
4. It adds the Geant4 module of GeoModel.
5. It updates the C++ standard requirement.
6. It adds a new variant determining the scalar type to use.
This commit supercedes #45851. Thanks @jmcarcell for the version
updates; I have added you as co-author.
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Updates
* alphabetic cmake args
---------
Co-authored-by: Juan Miguel Carceller <22276694+jmcarcell@users.noreply.github.com>
* Add missing MultiMethodMeta metaclass in builders
and remove the Python 2 fallback option in favor of hard errors to catch
similar issues going forward.
The fallback option can cause about 10K stat calls due to use of
`realpath` in the inspect module, depending on how deep Spack itself is
nested in the file system, which is ... undesirable.
* code shuffling to avoid circular import
* more reshuffling
* move reserved variant names into variants module
Currently, the llvm-amdgpu package doesn't compile on MacOS, but it is
also not marked as a conflict. This causes problems because it seems
that Spack is very happy to pull in llvm-amdgpu as the default package
to satisfy any virtual libllvm dependency, which can cause dependent
specs to fail to install on MacOS. This commit marks a conflict between
this llvm package and the Darwin platform.
* root: set +webgui~http conflict from @6.28.12:
Currently, the ROOT spec correctly identifies a conflict between +webgui
and ~http, but this conflict is marked as affecting @6.29.00: only. As a
matter of fact, ROOT 6.28.12 is also affected. This commit, therefore,
updates the when clause on the conflict to @6.28.12:.
* Remove when clause entirely
* oops
* Adds an MSBuild system + Builder to the icu4c package
* Adds custom install method as MSBuild system does not vendor an
install target
* The cxxstd variant is not supported on Windows (there are no config
options you use to tell the build system what cxx standard to
build against), so the variant definition was updated to occur
everywhere except Windows
Also, this commit defines the c/cxx..._flag properties of the MSVC
compiler (although they are not used by `icu4c` and not strictly
necessary to bundle with this PR).
This commit adds a dependency on GeoModel 4.6.0 when the GeoModel plugin
is enabled. Note that the dependency is upgraded to 6.3.0 in Acts
36.1.0, but that will need to be covered in #45851.
* geomodel: add versions 5.1.0-6.4.0
This PR adds new versions 5.1.0 through 6.4.0 of the geomodel package.
It also updates the CMake configuration to use the `define_from_variant`
mechanism and it adds me as a maintainer.
* Undo cmake change
The avx2 version can be downloaded from the ORCA [forum](https://orcaforum.kofo.mpg.de/app.php/dlext/?view=detail&df_id=214#).
The version is named `avx2-6.0.0` (as opposed to the more
natural-looking `6.0.0-avx2`) to avoid the avx2 version shadowing the
non-avx2 one. Definitely open for better suggestion.
* hsakmt-roct: new test API
* hsakmt-roct: minor change to check_install script variable name
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parallel-netcdf: new test API
* parallel-netcdf: fix test args and tweak docstring and variables
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* WarpX: Python on pyAMReX
Long overdue update for WarpX: in 2024, we updated our Python
bindings to rely on the new pyAMReX package. This deprecates the old
`py-warpx` package and adds a new dependency and variant to WarpX.
Also deprecates old versions that we will not continue to support.
* Update Cloud/E4S Pipelines for WarpX
`py-warpx` is replaced by `warpx +python`
oneAPI does not support IPO/LTO: diable for `py-amrex` even though
pybind11 strongly encourages it.
* Add `+python` variant
* `esmf` package installs Python bindings when `+python` is set
Note: this does not inherit `PythonPackage`, which force an either/or
choice between the Makefile and Pip builder: it instantiates a
`PythonPipBuilder` as needed (when `+python` is set).
Paths over 260 characters in length are not handled by `shutil.rmtree`
unless they use the extended-length path syntax (using a prefix of
"\\?\").
This fixes an issue where stage cleaning fails when paths in a stage
exceed the normal 260-character limit.
This indicates that other parts of the codebase should be examined/
refactored to handle long paths.
* harfbuzz: enable freetype in MesonBuilder to facilitate depends_on("freetype")
* spack style fix
* freetype is defined as a depends_on(), so set as enabled in MesonBuilder rather than an option/flag/variant
* add back depends_on($lang) lines for new test api
* keep original order
Previously `spack buildcache push --only=package` errored in the OCI
case, but it's been requested that OCI can be used as pure storage w/o
the need for runnable container images.
This commit makes it so that
1. manifests refer only to runtime dependencies that were selected to be
pushed
2. failure to upload a blob among the selected specs does not prevent a
manifest/tag to be created for dependents: they just don't refer to
the missing blob as a layer/dependency
This fixes the following issues:
1. dependents of non-redistributable specs can now be pushed to oci
build caches without error
2. failure to upload one tarball does not cause cascading failures for
dependents whose tarballs do upload succesfully -- so it's better
best-effort behavior
3. for some people uploading with deps caused a massive amount of
fetches of their manifests (which certain registries count as a
download of an image, even though their layers are not fetched) --
being able to specify --only=package reduces the number of fetches.
Stage objects create mirrors ad-hoc from current config.
- There is no way to prevent mirrors from being used
- There is no way to restrict mirrors to source/binary, which is of
course context dependent.
- Stage is also used in build caches, where iterating over mirrors is
already implemented differently, and wouldn't work anyways cause it's
source only, and in particular it makes no sense for OCI build caches.
This commit:
1. Injects the sensible mirrors into the stage object from contexts
where it is relevant
2. Separates mirrors from cache, so that w/o mirrors download cache can
still be used
The test_foreground_background unit test has been marked
xfail for a while, meaning:
- Nobody looks at the results of the test
- It still runs every time
That test happens to hang frequently on some Apple M1 I have access to,
so here I mark it as skip.
Also went through other xfailing and skipped tests, and applied minor changes.
* Use absolute paths instead of https:// fake mirrors (this speed-up tests by avoiding requests)
* Add a fixture to gather in a single place code that is copy/pasted in a lot of tests
* General clean-up of tests and repeated code
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* hypre: get rid of depreciated run_test method
* hypre: use mkdirp directly
* hypre: use install() for ij for addition of permissions fix
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* parsec: old to new test API
* parsec: restore stand-alone test subparts; preliminary test build fixes
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* n2p2: convert from old to new test API
* n2p2: Enhance stand-alone testing checks to reduce unnecessary processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* stand-alone test API update: self.cache_extra_test_sources(...) -> cache_extra_test_sources(self, ...)
* superlu: switch to new cache_extra_test_sources API
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* Add options for sparse checkout in GitFetcher
Newer versions of git have a beta feature called sparse checkout
that allow users to check out a portion of a large repo.
This feature will be ideal for monolithic repo projects that want to
model their infrastructure via spack. This PR implements an addition
to the GitFetcher that allows users to add a `git_sparse_paths`
attribute to package classes or versions which will then use sparse
checkout on those directories/files for the package.
* Style
* Split git clone into multiple functions
* Add sparse-checkout impl
* Internalize src clone functions
* Docs
* Adding sparse clone test
* Add test for partial clone
* [@spackbot] updating style on behalf of psakievich
* Small fixes
* Restore default branch status
* Fix attributes for package
* Update lib/spack/docs/packaging_guide.rst
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Extend unit test to multiple git versions
* style
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Matthew Mosby <44072882+mdmosby@users.noreply.github.com>
* Feature update py-fortranformat
Add more recent versions of py-fortranformat. The currently included release (0.2.5) is from 2014. I've added the latest point release of each of the major versions from the last 4 years.
* update homepage
* magma: old to new test API
* magma: simplify stand-alone test method/part docstrings/purposes
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* initial commit to enable asan in remaining rocm packages
* remove os import
* add rocm-opencl rocm-dbgapi rocm-debug-agent
* add libclang path to LD_LIBRARY_PATH
* enable asan for rocfft
* add f-string and add +asan dependencies for hip and rocm-opencl
* add conflicts for centos7/8 and rhel 9
When sending Repo and RepoPath over to a child process,
we go through a marshaling procedure with pickle. The
default behavior for these classes is highly inefficient,
as it serializes a lot of specs that can just be
reconstructed on the other end of the pipe.
Here we write optimized procedures to __reduce__ both
classes.
* raja: new test API
* raja: tweak test method names and docstrings
* raja: restore running tests under proper directory
* raja: cleanup skiptest message and example call
* raja: Tweak expected outputs to match current
* raja: test_views -> test_stencil_offset_layout
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* mptensor: Changed skiptest, test name, and added docstring
* mptensor: make stand-alone test method name and docstring more specific
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add version 2.0.-2
* Newer version at the top
* [@spackbot] updating style on behalf of rahmans1
* py-htgettoken: reorder comments
* Use sha256sum instead of commit id for version 2.0-2 and above
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Extracted from #44419
This adds / modifies some external detection tests for compilers,
to reproduce cases that are currently tested in unit tests.
The unit tests will later be removed.
Extracted from #45638
When adding the "detectable" tag to a package class that has the
"tag" attribute inherited from a base class, we need to copy it to
avoid modifying the base class.
* kickoff attempt
* resource similar to fortran
* delete unused install_component_codeplay
* Adding conflict for versions <= 2022.2.1, moving install to package.py, adding sha256 for version 2024.2.1
* [@spackbot] updating style on behalf of kaanolgu
---------
Co-authored-by: Kaan Olgu <kaan.olgu@bristol.ac.uk>
"spack buildcache push" for partially installed environments pushes all it
can by default, and only dumps errors towards the end.
If --fail-fast is provided, error out before pushing anything if any
of the packages is uninstalled
oci build caches using parallel push now use futures to ensure pushing
goes in best-effort style.
* migraphx: Old to new test API
* migraphx: tweak name and docstring to be more descriptive
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* c: new test API
* gcc: provides('c')
* c: bugfix and simplification of the new stand-alone test method
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* fortran: new test API
* fortran: add provides to gcc package
* fortran: simplify stand-alone test processing
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* hypre-cmake: old to new test API
* hypre-cmake: update Makefile to use installed files
* hypre-cmake: make stand-alone test method name more specific
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
The old concretizer is still used to bootstrap clingo from source. If we switch to a DAG model
where compilers are treated as nodes, we need to either:
1. fix the old concretizer to support this (which is a lot of work and possibly research), or
2. bootstrap `clingo` without the old concretizer.
This PR takes the second approach and gets rid of the old concretizer code. To bootstrap
`clingo`, we store some concrete spec prototypes as JSON, select one according to the
coarse-grained system architecture, and tweak them according to the current host.
The old concretizer and related dead code are removed. In particular, this removes
`Spec.normalize()` and related methods, which were used in many unit-tests to set
up the test context. The tests have been updated not to use `normalize()`.
- [x] Bootstrap clingo concretization based on a JSON file
- [x] Bootstrap clingo *before* patchelf
- [x] Remove any use of the old concretizer, including:
* Remove only_clingo and only_original fixtures
* Remove _old_concretize and _new_concretize
* Remove _concretize_together_old
* Remove _concretize_together_new
* Remove any use of `SPACK_TEST_SOLVER`
* Simplify CI jobs
- [x] ensure bootstrapping `clingo` works on on Darwin and Windows
- [x] Raise an intelligible error when a compiler is missing
- [x] Ensure bootstrapping works on FreeBSD
- [x] remove normalize and related methods
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
`setup-env.sh` is meant to be sourced, not executed directly.
By revoking execution permissions, users who accidentally execute
the script will receive an error instead of seeing no effect.
* Remove execution permission from `setup-env.sh` and friends
* Don't make output file executable in `spack commands --update-completion`
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* On Windows it is built with CMake, however CMake is built against
libuv, so libuv must depend on cmake+ownlibs to short circuit the
circular dependency
* libuv currently fetches the -dist source distribution of libuv for
certain versions because those versions contain a pre-generated
./configure script. However those distributions have all CMake
files removed, so they cannot be used to build on Windows.
Because the source distributions are different, this means the
checksums are different, and necessitates an additional version
declaration for each version we want to support with CMake.
* Update available versions
* fix style
* Reduce added versions to just a single new version as per review
* fix style
* Set dependency versions in line with pyproject and setup.py
* add new interp variant
* add wheel as required
* Add variant description
* change +interp default, actually add packaging this time
Fixes#43494
Add a set of environment variables SPACK_ALWAYS_CFLAGS (etc.) that
are always applied by the compiler wrapper.
Unlike SPACK_CFLAGS, for example, these will also be applied to
version checks (both SPACK_CFLAGS and SPACK_ALWAYS_CFLAGS will be
applied to the other invocation modes like ccld etc.).
Using this new functionality, the classic Intel and oneAPI compilers
are updated to pass compiler flags that disable warning messages
when newer versions are invoked via their older binary names
(these warnings were also generated for version checks, hence the
need for a new wrapper variable).
---------
Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
* namd:add compile options for ROCm
* Combine --rocm-prefix and its value in a single opts.extend call to ensure they remain ordered correctly and improve code robustness.
* Standardize the code and add ROCm conflicts
* add single_node_gpu conflict
---------
Co-authored-by: U-PALLAS\boudaoud <boudaoud@pc44.pallas.cines.fr>
When clang is installed I get `clang: Permission denied`.
Setting CLANG to the full path of clang fixes this.
Signed-off-by: Bernhard Kaindl <bernhardkaindl7@gmail.com>
Fixes error:
```txt
3 errors found in build log:
61 ==> xmlto: Executing phase: 'build'
62 ==> [2023-10-26-09:48:35.425903] 'make' '-j16' 'V=1'
63 make all-am
64 make[1]: Entering directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
65 /bin/sh ./ylwrap xmlif/xmlif.l .c xmlif/xmlif.c -- /bin/sh /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing flex
66 /tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src/missing: line 81: flex: command not found
>> 67 WARNING: 'flex' is missing on your system.
68 You should only need it if you modified a '.l' file.
69 You may want to install the Fast Lexical Analyzer package:
70 <http://flex.sourceforge.net/>
>> 71 make[1]: *** [Makefile:757: xmlif/xmlif.c] Error 127
72 make[1]: Leaving directory '/tmp/root/spack-stage/spack-stage-xmlto-0.0.28-huuygrp4qasytrezg774yavnnaxzgp2e/spack-src'
>> 73 make: *** [Makefile:584: all] Error 2
```
For example: spack install py-pyqt5 or py-pyqt6 would install pylupdate5 pyrcc5 and
pyuic5 into in python-venv's install prefix.
Fix https://github.com/spack/spack/issues/45359
* [py-configspace] fix dependency on cython
* py-cython not needed starting from 1.0.0
* added py-configspace 1.0.0 and 1.0.1
* py-configspace: fix style
* added py-configspace version 1.1.0
* added py-configspace version 1.1.1
* py-configspace: two more versions and new maintainer
* py-configspace: fixed typo
* py-configspace: added version 1.1.4
* Allow spec queries by `namespace`
Spack specs have "namespaces" that indicate what package repository they come from, but
there has not been a way to use the spec syntax to match one.
You can say things like this:
```console
spack find builtin.zlib
spack find myrepo.zlib
```
But, because namespaces are written as a dot-separated prefix on the name, you can't say
"find me all specs in namespace myrepo". The syntax doesn't allow it.
This PR allows you to specify anonymous specs with namespaces on the CLI. Specifically
you can do queries like this:
```console
spack find namespace=builtin
spack find namespace=myrepo
```
You can use this anywhere else you use spec syntax, e.g. in a config file to separate
installations based on what repo they came from:
```yaml
spack:
config:
install_tree:
root: $spack/opt/spack
projections:
namespace=myrepo: "myrepo_special_path/{name}-{hash}"
namespace=builtin: "builtin/{name}-{hash}"
```
This PR adds a special `namespace_if_anonymous` attribute to specs, which returns the
`namespace` if the spec has no name, otherwise it returns `None`. This allows us to
print the namespace for anonymous specs but to continue hiding it for most views, as
we've done so far.
This is implemented as a special case, but it's one that already exists, along with
`platform`, `os`, `target`, etc. This also reserves existing special case names for
variants so that users cannot define them in their package files. This change is
potentially breaking, but I do not think it will be common. There are no builtin
packages with a variant called `namespace`, and defining `os`, `target`, or `platform`
as a variant would've likely caused other problems if they were already being used.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* format: allow spaces in format specifiers
Key-value pair format specifiers can now contain spaces in the key. This allows us to
add spaces to format strings that are *only* present when the attribute formatted is not
``None``. Instead of writing:
```
{arch=architecture}
```
and special casing `arch=` like a sigil in `Spec.format()`, we can now write:
```
{ arch=architecture}
```
And the space is *only* printed when `architecture` is not `None`. This allows us to
remove the special case in `Spec.format()` for `arch=`.
Previously the only `key=` prefix allowed in format specifiers was `arch=`, but this PR
removes that requirement, and the `key=` part of a key-value specifier can be any name.
It does *not* have to correspond to the formatted attribute.
- [x] modify `SPEC_FORMAT_RE` to allow arbitrary keys in key-value pairs.
- [x] remove special case for `arch=` from `Spec.format()`.
- [x] modify format strings using `{arch=architecture}` to use `{ arch=architecture}`
- [x] add more tests for formatting
This PR saves other more complex attributes like compiler flags and their spacing for later.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
VC toolset versions 144 and 143 are both associated with the platform
toolset 143; this deviates from prior version choices made by the
MSVC devs; add a special case to report platform toolset version
as 143 when VC toolset version is >= 143 (this will need revisiting
for later releases).
The setSet tool (removed in version 10) has an optional dependency on readline.
The build script will use the system readline if not provided by Spack.
* dd4hep: restrict to podio@0 to prevent failures with podio@1
* acts: restrict to podio@0 to prevent failures with podio@1
* dd4hep: close when range for podio
* acts: close when range for podio
* acts: fix when range for podio
* acts: close when range for podio
* acts,dd4hep: ensure main/master still depends on podio
* Fix +rocm variant, to ensure correct dependencies on ROCm packages
and use of AMD LLVM
* Add a +pshm variant for comm=gasnet to enable fast shared-memory
comms between co-locales
* Add logic to ensure we get the native CXI libfabric network provider
on Cray EX
* Expand dependency type for package modules to encompass runtime
dependencies
* Factor logic for setting (LD_)LIBRARY_PATH and PKG_CONFIG_PATH of
runtime dependencies
* Workaround issue #44746 that causes a transitive dependency on lua
to break SLURM
* Disable nonfunctional checkChplDoc test
* Annotate some variants as conditional, to improve spack info output
and reduce confusion
---------
Co-authored-by: Dan Bonachea <dobonachea@lbl.gov>
`apple-libuuid` includes types that aren't available in other `uuid`
providers; this cause issues in consuming packages (e.g., py-matplotlib)
that use SDKs like CarbonCore.framework when they attempt to use
`util-linux-uuid` as a `uuid` provider on Mac OS.
Tweak `util-linux-uuid` to indicate that it does not provide `uuid`
on Mac OS.
This was missed in #45394 because we don't run unit tests for package PRs, and
`test_all_virtual_packages_have_default_providers`, which would've caught it, is a unit
test, not an audit.
- [x] add a default provider for `wasi-sdk` in `etc/spack/defaults/packages.yaml` (which
we require for all virtuals)
- [x] rework `test_all_virtual_packages_have_default_providers` as an audit called
`_ensure_all_virtual_packages_have_default_providers`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* pinentry: New API
* move code around
* added back version check
* Complete check_version refactor
* Honor original handling of guis (i.e., don't try if not there)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* upgrade apptainer: security fix
apptainer 1.3.1 has a high severity security issue: CVE-2024-3727
Upgrade to 1.3.2 or preferably 1.3.3
* added comment for 1.3.1
---------
Co-authored-by: Walter de Jong <walter.dejong@surf.nl>
* intel-oneapi-compilers: update description with current compilers
* Update var/spack/repos/builtin/packages/intel-oneapi-compilers/package.py
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* intel-oneapi-compilers: break docstring line
---------
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* kokkos: new test API
* kokkos: added import llnl.util.lang as lang because earlier versions couldn't be installed without it
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* povray: new test API
* capture output and test name change
* povray: add v3.7.0.10, deprecate 3.7.0.8
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* umpire: old to new test format
* umpire: old to new test method and refactor
* indentation
* black reformat
* last minute syntax
* docstring and checks
* black format
* change test name
* method call correction
* Resolve problem with stand-alone tests (or examples) not loading library (Fixes#44960)
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* sz: new test API
* fix typo; check installed executable; conform to subpart naming convention
* skip tests early if not installed; remove unnecessary "_sz" from test part names
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Library names on Windows are not typically prefixed with lib; the default
`.libs` implementation accounts for this, but `netlib-lapack` has a
custom implementation of `.libs` that did not account for this.
The `Executable` class emits a warning when you pass quoted arguments to it:
```
> spack pkg grep '"namespace"'
==> Warning: Quotes in command arguments can confuse scripts like configure.
The following arguments may cause problems when executed:
"namespace"
Quotes aren't needed because spack doesn't use a shell. Consider removing them.
If multiple levels of quotation are required, use `ignore_quotes=True`.
```
This is to warn new package authors who aren't used to calling build commands in python.
It's not useful for `spack pkg grep`, where we really are passing args on the command
line, and if we pass a quoted string, we probably meant to.
- [x] make `ignore_quotes` an instance variable, not just an argument to ``__call__`
- [x] set `ignore_quotes` to `True` in `spack pkg grep`
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* package/linux-pam: dependencies
* Adding variants to linux-pam
* Updating linux-pam variants
* Fixing variants for linux-pam after testing
* clean up flag handling
* clean up terrible tabs
* cant use default_args for compiler dependencies
* Change selinux to off by default
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Adding new versions and compilation conflict for nodejs
* Update failed version for gcc14
* Updating conflicts notes for correctness/clarity/format
* Applying spack-ized versions of fix in https://github.com/nodejs/node/issues/52223 to adddress CI failures
* Update fix-old-glibc-random-headers.patch
* Update package.py
* Update fix-old-glibc-random-headers.patch
* Update fix-old-glibc-random-headers.patch
* Adding conflict for older glibc
* Fixing patch for older systems, need to undef
* Removing overly strict conflicts
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Several tests can be enabled on Windows with no changes to logic
(either the test logic or logic being tested)
* Test for `spack location` requires modification of the test logic,
but with a minor change can be enabled on Windows
using self.compiler.cc_pic_flag here results in these errors:
==> sqlite: Executing phase: 'install'
==> Error: AttributeError: 'AutotoolsBuilder' object has no attribute 'compiler'
change it to self.pkg.compiler.cc_pic_flag instead.
* imoroved-rdock: new test API
* Make test subpart names and or descriptions more accurate
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* e4s ci: enable some disabled specs
* comment out cp2k +cuda due to unsupported cuda_arch
* comment out dealii+cuda due to concretize error
* work through concretize errors
* e4s: comment out failing builds
* e4s stack: disabled non-building specs
* comment out failing specs
* comment out failing specs
* cleanup comments
* rust: rework external find to require both rustc and cargo
* rust: handle unable to parse version
* [@spackbot] updating style on behalf of wdconinc
* rust: not x or not y -> not (x and y)
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: pick first rustc found
Co-authored-by: Alec Scott <hi@alecbcs.com>
* rust: list comprehensions
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
Co-authored-by: Alec Scott <hi@alecbcs.com>
* libmesh: add missing v1.7.1 release
* libmesh: avoid pulling in petsc if only +mpi or +metis is given
* libmesh: add shared variant
Co-authored-by: rbberger <rbberger@users.noreply.github.com>
The "use_store" context manager is used to swap the value
of a global variable (spack.store.STORE), while keeping
another global variable consistent (spack.config.CONFIG).
When doing that it tries to evaluate the previous value
of the store, if that was not done already. This is wrong,
since the configuration might be in an "intermediate" state
that was never meant to trigger side effects.
Remove that operation, and add a unit test to
prevent regressions.
Add language dependencies `c`, `cxx`, and `fortran`.
These `depends_on` statements are auto-generated based on file extensions found
in source tarballs/zipfiles.
The `# generated` comment can be removed by package maintainers after
validating correctness.
* detray: new package
This commit adds the detray package, a detector description library for
HEP experiments that is designed to be GPU-friendly.
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* Update var/spack/repos/builtin/packages/detray/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* vasp: add new version 6.4.3 and rework package
* vasp: remove redundant cuda dep
* vasp: bump fftlib variant version restriction
* vasp: honor the still existing scalapack variant
---------
Signed-off-by: Shane Nehring <snehring@iastate.edu>
Since the the MetaPathFinder now owns a lazily constructed RepoPath object, we need to deserialize environments before the package that needs to be restored. Before we were relying on globals to be inconsistent in a way that let the entire process go.
The patch which enables icx/ifx compilers is only added for `wrf@4.4:`. This PR prints a useful message at concretization time instead of failing the installation later on.
Co-authored-by: stephenmsachs <stephenmsachs@users.noreply.github.com>
* slate: changed from old to new format
* make code tighter
* replace assert method
* SkipTest plus other cleanup
---------
Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
* vecmem: add infrastructure for working with SYCL
The vecmem package uses an unorthodox build system where, instead of
expecting a SYCL-capable compiler in the `CXX` environment variable, it
expects one in `SYCLCXX`. It also needs the correct SYCL flags to be
set. This commit adds a custom build environment for the vecmem package
which allows it to be built in this way. I've also added an extra CMake
flag to ensure that the build system doesn't download any unwanted
dependencies.
* Update var/spack/repos/builtin/packages/vecmem/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
- Add explicit conflict on ppc64le for 12.5 and newer.
- Update/fix intel compiler conflict to reflect that intel@2021 is compatible
only since 11.4.1 and not since 11.1.1.
- Add intel compiler conflicts to reflect strict support matrix since
12.2.0.
As discussed in https://github.com/spack/spack/pull/44881#issuecomment-2218411735 a `spack install py-globus-cli` fails to concretize on an Ubuntu 22.04 under Windows WSL2 because of too strict of explicit dependencies.
Let's try to remove them here (since these are "just in case" and in all honesty should be handled by `py-globus-sdk` anyways).
* Now builds with latest fmt release (11.0.1)
* Missing array include in nem_spread
* Fix timestep conssitency check in file-per-rank case if one or more dbs have no timesteps.
If an external happens to be a provider of anything, the solver would
set its weight to 0, meaning that it is most preferred, even if
packages.yaml config disagrees.
That was done so that `spack external find mpich` would be sufficent to
pick it up as mpi provider.
That may have made sense for mpi specifically, but doesn't make sense
for other virtuals. For example `glibc` provides iconv, and is an
external by design, but it's better to use libiconv as a separate
package as a provider.
Therefore, drop this rule, and instead let users add config:
```
mpi:
require: [mpich]
```
or
```
mpi:
buildable: false
```
which is well-documented.
This change makes `spack external find --not-buildable` mark virtuals
provided by detected packages as non-buildable, so that it's sufficient
for users to let spack detect say mpich and have the concretizer pick it
up as mpi provider even when openmpi is "more preferred".
Consistently with ArmPL@24:, the include directory for acfl@24:
has changed to include. The change wants to update to this change
and distinguish the include path for releases previous to 24.04
and the future ones
* rocm-cmake: changed test format from old to new
* Rename cmake variable
* post-conflict resolution: remove remaining version check
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Added Release 0.0.4 and 0.0.5
* Changed requirement for gotcha
use gotcha 1.0.5 for 0.0.2 and 0.0.3
* Combine gotcha 1.0.7 for master and develop
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
in case of inheritance the static tags prop may be updated multiple
times, and it turns out builder classes magically inherit from
traditional package classes
While trying to build packages with the OneAPI compiler version 2024.1 I
ran into the following error, indicating that the compiler is unable to
find some necessary libraries:
```
/storage/Software/oneapi/2024.1/compiler/2024.1/bin/sycl-post-link: error
while loading shared libraries: libonnxruntime.1.12.22.721.so: cannot open
shared object file: No such file or directory
icpx: error: unable to execute command: No such file or directory
icpx: error: sycl-post-link command failed due to signal (use -v to see
invocation)
```
Indeed, `libonnxruntime.1.12.22.721.so` does come bundled with the
OneAPI compiler, but it is not available in the build environment by
default. In this commit, I update the custom environment created by
OneAPI to include the `lib/` directory in which these libraries reside
in the `LD_LIBRARY_PATH` environment variable.
In my enthusiasm to add dfelibs to Spack, I didn't realise that the
unit tests of dfelibs use Boost and, as such, Boost is required as a
testing dependency.
Originally if you had `x -> y -> z`, and an env with `x` in its speclist that is concretized but not installed, then `spack find -c y` would not show anything. This was intended: `spack find` has up-until-now only ever listed out installed specs (and `-c` was for adding a preamble section about roots).
This changes `spack find` so:
* `-c` makes it search through all concretized specs in the env (in a sense it is anticipated that a concretized environment would serve as a "speculative" DB and users may want to query it like they query the DB outside of envs)
* Adds a `-i/--install-status` option, equivalent to `-I` from `spack spec`
* Shows install status for either `-c` or `-i`
* As a side effect to prior point, `spack find -i` can now distinguish different installation states (upstream/external)
Examples:
```
$ spack find -r
==> In environment findtest
==> 1 root specs
- raja
==> 6 installed packages (not shown)
==> 12 concretized packages to be installed (not shown)
```
```
$ spack find
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
berkeley-db@18.1.40 bzip2@1.0.8 diffutils@3.10 gmake@4.4.1 gnuconfig@2022-09-17 libiconv@1.17
==> 6 installed packages
==> 12 concretized packages to be installed (show with `spack find -c`)
```
```
$ spack find -c
==> In environment findtest
==> 1 root specs
- raja
-- darwin-ventura-m1 / apple-clang@14.0.3 -----------------------
[+] berkeley-db@18.1.40 [+] bzip2@1.0.8 - cmake@3.29.4 [+] diffutils@3.10 [+] gmake@4.4.1 [+] libiconv@1.17 - nghttp2@1.62.0 - pkgconf@2.2.0 - readline@8.2
- blt@0.6.2 - camp@2024.02.1 - curl@8.7.1 - gdbm@1.23 [+] gnuconfig@2022-09-17 - ncurses@6.5 - perl@5.38.2 - raja@2024.02.2 - zlib-ng@2.1.6
==> 6 installed packages
==> 12 concretized packages to be installed
```
$ spack -E find
...
==> 82 installed packages
```
* Add py-datasets version 2.20.0
* Add dependency requirements for version 2.20 + refactor
* Add missing tqdm and requests versions and to install latest py-datasets
* Add missing python requirements for 2.8.0 and 2.20.0
Fix a bug where Spec.tree with cover=nodes reduces deptypes from all
in-edges, including from nodes not reachable from the root, which almost
always happens for concrete specs
`Spec.__getitem__` queries dependent edges, which almost always point to
nodes outside the sub-dag considered. It should only ever look at edges
being traversed.
When using a newer Clang for Kokkos than supported by a given CUDA version, the
CUDA compiler detection in Legion's CMake still needs to be passed
CMAKE_CUDA_FLAGS to pass the compiler check.
This modifies heuristic to decay to clingo default
over time. The hope is that this helps with specs
that have an optimal solution with a high penalty.
Let target and compiler heuristic decay too, do not
guess compiler
* dfelibs: new package
This commits adds the `dfelibs` package, which is an Acts project fork
of dfelibs.
* dfelibs: docstring typo
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Add MatrixSwitch package
Add libOMM package
Use libOMM as external in ELSI
Add include paths for all libraries
Fortran modules need one directory up (can't use libs.directories since they are just the include/ directories)
This addresses a few issues in the kentutils package:
- Issue #44372
- Cleaning up formattting and styles
- Removing old versions that are not avaialble anymore
- Removing newer versions that are also not available anymore
- The installer does not install the static libs
that are expected by packages that depend on kentutils
- There are conflicts and less-than-desirable dependencies
that can be addressed via variants
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
* Bumping up to ROCm 6.1.2
* Bump up rocmlir & amdsmi to 6.1.2
* Removing llvm dependency from amdsmi
* Removed redundant for loops and brought rocm-cmake in for loop
* Removing version check of deprecated versions
This adds spack.util.spack_yaml.anchorify, which takes a non-cyclic
dict/list structure, and replaces identical values with (back)
references to the first instance, so that yaml serialization will use
anchors.
`repr` is used to identify sub-dags, which in principle is quadratic
complexity in depth of the graph, but in practice the depth is O(1) so
this should not matter.
Then this is used in CI to reduce the size of generated YAML files to
30% of their original size.
* lammps: add new versions
* lammps: add directories for plugins
* lammps-example-plugin: new package
Example illustrating how to install a LAMMPS plugin, based on the
example part of the LAMMPS distribution.
If netlib-lapack is built with ~external-blas, it internally links
liblapack.so with libblas.so, meaning that whenever netlib-lapack is
used as a lapack provider, the package must also be a blas provider.
Conversely using netli-lapack as a blas provider does not imply that it
also must provide lapack, but nothing is lost disallowing that...
In practice people don't care about having their build dependencies reinstalled with say cflags=-O3 if that is what is set at the input spec, so restrict propagation to link/run deps only.
Also simplify the encoding in asp.
* Changed test method from old method to new method
* Corrected test name and added docstring
* Split test method into stand alone-tests
* Added SkipTest
* code refactor
* removed repeated code
* Remove exe from test part purpose
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Improve proj version selection to avoid too new of a proj with older api usage in vtk9
* Proj constraint range around 8.2.1
---------
Co-authored-by: John Parent <john.parent@kitware.com>
Fix the following on Windows:
* `spack load --list` (this printed 0 packages even if packages were
loaded)
* `spack unload <package>` (this said that the package is not loaded
even if it was)
Update unit tests for `spack load` to also run on Windows (specifically
for ".bat"). This involved refactoring a few tests to parameterize
based on whether the unit tests are being run on a Windows system
(and to account for batch syntax).
* Adding additional versions to plink2 and switching to tarballs to allow for better version detection in the future
---------
Signed-off-by: Teague Sterling <teaguesterling@gmail.com>
The pcluster pipeline runs spack external find and autodetects the most ancient of zlib versions on the system: 1.2.7, release date May 3, 2012.
Turns out some packages require parts of the zlib api added within the last decade.
* update e4s stacks
* adios2 +rocm: disable kokkos due to spack issue #44832
* comment out mgard+cuda due to spack issue #44833
* comment out cabana, legion, arborx due to kokkos spack issue #44832
* comment out slepc, petsc due to petsc spack issue #44600
* comment out adios2+rocm due to kokkos rocm spack issue #44832
* comment out kokkos due to spack issue #44832
When we changed how to deal with errors in November,
we didn't realize that for an unconstrained choice
rule it is more important in the heuristic to guess
what is NOT in the answer set, since it will be the
majority of options.
Previously this was following automatically from what
was in the answer set, via `1 { ... } 1` cardinality
constraints.
Here we improve the heuristic and the solve time for specs.
* Patched CMakeLists.txt to find drm include at libdrm - archlinux
* Restricted usage patch to versions 34 onwards
* Update var/spack/repos/builtin/packages/rdma-core/libdrm.patch
---------
Co-authored-by: Mark Abraham <Mark.J.Abraham@gmail.com>
* Spglib: add version 2.4.0
* DLA-Future: fix +test option
* elsi
* elsi improvements
* [@spackbot] updating style on behalf of RMeli
* remove test variants
* fix ntpoly and and use externals as default
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* py-multiqc: updating to @1.21, adding new dependency py-pyyaml-env
* Correcting dependencies
* Drop matplotlib depends_on() for versions of multiqc not included in package.py
---------
Co-authored-by: LMS Bioinformatics <bioinformatics@lms.mrc.ac.uk>
* Allow zlib to find external installations.
* Apply suggestions from code review
* Fixed the determine_version function to loop over all of the potential
libraries that could be installed by zlib.
---------
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
* Update hohqmesh and feq-parse versions
Update feq-parse homepage to new documentation page
Update feq-parse license to be consistent with 2.2.2 release
---------
Co-authored-by: fluidnumerics-joe <fluidnumerics-joe@users.noreply.github.com>
This commit adds new versions of actsvg. I have also taken the liberty
of adding myself a maintainer as I work actively on the ACTS project and
its dependencies, and would like to start keeping the Spack specs more
up to date.
* e4s external rocm ci: bump rocm stack to v6.1.1
* comment out exago+rocm due to issue with raja@0.14.0 see spack issue #44593
* comment out adios2+rocm due to spack issue #44594
* comment out petsc+rocm due to spack issue #44600
* comment out sundials+rocm due to spack issue #44601
* comment out slepc+rocm due to petsc spack issue #44600
* comment out tau+rocm due to spack issue #44659
* comment out ecp-data-vis-sdk due to spack issue #44745
* packages: register rocm-core as external
* re-enable tau due to issue #44659 having been resolved
* use latest ci image: ecpe4s/ubuntu22.04-runner-amd64-gcc-11.4-rocm6.1.1:2024.06.17
* comment out paraview due to spack issue #44745
* comment out ecp-data-vis-sdk +vtkm due to issue https://gitlab.spack.io/spack/spack/-/jobs/11632511
* Update for 2.0.1
* cloudpickle dependency is only 'run'
* Follow new formatting guidelines
* black wants trailing commas
* Simplified version ranges, as recommended by @tldahlgren
* vtk-m: Changed test method names and skipping non-applicable tests from old to new approach
-----
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* add cray detection taken from upcxx
* add CUDA/ROCm support
* add numerous pass-through options to Chapel build,
like gpu_mem_strategy, comm_substrate, etc.; all variants are
translated to analogous CHPL_* environment variables. As a side
effect, this defines a number of environment variables that are
not actually used by Chapel.
* Define LD_LIBRARY_PATH, LIBRARY_PATH, and PKG_CONFIG_PATH to
help programs built with Chapel properly locate needed runtime
dependencies
---------
Co-authored-by: bonachea <dobonachea@lbl.gov>
* Projsync needs curl
This fixes a proj compilation error from paraview
* tiff does not set TIFF_INCLUDE_DIR tested by proj
* TIFF patch exists for proj 9.2
* Add comments with location of proj patch
* Use TIFF patches for different cmake versions
* Use modules starting cmake 3.28 and variables before that
* Fix black style
* Simpler patches, add newer proj
* Remove duplicated flag
* Deprecate PROJ 4.8 and older
* Remove PRIVATE
* Deprecate PROJ 7.0.0
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Not sure why I had this here, as `/bin/sh` will find the first spack in `PATH` just like
`env`.
- [x] remove `/usr/bin/env` and avoid an extra process launch.
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
* omega-h: reformatted from old test method to new test method
* omega-h: added proper output checking
--------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The ``+cluster` variant requires there be an MPI family known in the
spec. When using externals it is easy to miss-configure this requirement
leading to a runtime exception which is not desirable. This converts the
exception to a package rule.
* xorg libs: new versions
* xorg pkgs: update homepage from cgit to gitlab
* xorgs pkgs: fix homepage since, yeah, I did edit those 122 files by hand...
* libxmu: don't need the .0 patch version here
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Fixed some issues with installation breaking due to mysql_client
* Remove debugging details
* Adding client_only to deps
* Adding client_only to variant and deps
* Undoingw client_only to variant and deps
* py-pyopengl: new simple package
* py-pyopengl: Fix typo in comment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pyopengl: add variants and dependencies
* py-pyopengl: build from source and improve variants
* py-pyopengl: use corrected freeglut libs
* py-pyopengl: update copyright
* py-pyopengl: remove duplicate conflict clause
* py-pyopengl: change dependency to link
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Core change: logic for extracting RPATHs from modules may return
`None`: filter this out of the set of RPATHs that is auto-generated
* Core change: `CachedCMakePackage` no longer adds ldflags to
`CMAKE_STATIC_LINKER_FLAGS`: generally these flags are not appropriate
for static linking (e.g. invocation of `ar`)
* [mfem] Add version 4.7
* [mfem] Add variant for precision (single/double). Enforce consistency
for precision amongst mfem and hypre/petsc/mumps dependencies
* [mfem] Add cxxstd (and related constraints preventing use of
old cxxstd values for newer versions of some dependencies)
* [hypre] In line with prior point, added support for specifying
precision
* [petsc] Add config option to avoid error when building against
`superlu-dist+rocm`
* [hiop] add proper `raja`/`umpire`/`camp` version constraints for
`hiop` versions 0.3.99-0.4.x; require `+raja` for `+rocm`, and
add dependency on `hiprand` for `+rocm`
* [butterflypack, mfem, strumpack, suite-sparse] Require
`CRAYLIBS_{target-family}` env var to be defined
* [suite-sparse] versions `@7.4:` changed install location of headers:
add symlink from old location to new location
* [zlib-ng] Fix error where shared libs were not successfully built for
`%cce@17` (the build did not fail, but the finished `zlib-ng%cce@17`
install did not have shared libs)
* suite-sparse library path can be lib or lib64
* fixing style check error
* Adding the missing import os
* Style check error fix
* Updating patch for 6.1.0 with correct lib path
* Preserve higher weight for CLI git ref versions
Currently the concretizer fails if you reuse a git ref version
that has already been installed but modify the spec at all.
See #38484 for futher diagnosis
The issue here is that since there is no established provenance for
these versions the highest weight they are currently assigned is
that of prior install. Re-use checks then fail because the weight of
the version is identical to the solver.
Ironically, these versions are given the highest weights possible when
specified on the CLI for the first time. They should only appear in a
DAG if they are an exact match or if the user specifies them at the CLI.
Therefore it makes sense to preserve their higher ordering.
Getting this right is critical to moving all branch based versions to a pinned
git-ref in the future.
* [@spackbot] updating style on behalf of psakievich
* Update lib/spack/spack/solver/asp.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Add provenance specific to git ref installs
* Sensitvity to name that I could not track down
* Add regression test
* Adjust test
* Add prefer standard unit-test
* Style
* Add required mock
* Format and mark
* Make unit-test case reproduce CLI investigation
* Remove unnecessary mock package
* [@spackbot] updating style on behalf of psakievich
* Use already developed fixture
* Add zlib-ng to mocks again
* Remove accidental adds
* Remove maintainer
* [@spackbot] updating style on behalf of psakievich
* Rename test file
* [@spackbot] updating style on behalf of psakievich
* Remove unused imports
* Update tests
* [@spackbot] updating style on behalf of psakievich
* Style
* Update lib/spack/spack/test/concretize.py
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Update solver rule
* Duplicate installed rules for installed_git_version
* Revert "Duplicate installed rules for installed_git_version"
This reverts commit 17223fc8d1.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
* libdistributed: renamed test to check_test
* libpressio-rmetric: renamed test method
* libpressio-tools: renamed test method
* Renamed build-time test methods for: libpressio-opt, libpressio-tthresh, opened
---------
Co-authored-by: Jackson Peter Lawrence <lawrence31@borax5.llnl.gov>
* Provide instructions for building a developmental version of lcov.
* style fix
* style fix
* Promote single version git URL to the package level.
* Formatting fix.
* verilator: Some environment varibles are no longer required
* Revert depends_on for flex back to the deafult case
* Use decorator, when, for setup_run_environment insdtead of if inside function
* libtiff: add v4.6.0 and default disable opengl (#44545)
* libtiff: Fix typo in CMake key
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: Broader description of OpenGL variant
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* libtiff: reformat using spack style black recommendation
* libtiff: couple opengl flag with autotools
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There hasn't been a release in almost a decade, and build failures with
GCC 14 were reported. I don't think it makes sense to patch it, since
the project moved over to `util-linux`, and Spack's default provider is
that package. Better to just get rid of it in the next Spack release.
* py-your: new package
Spack package recipe for YOUR, Your Unified Reader. YOUR processes pulsar data in different formats.
Output below from spack install py-your
spack install py-your
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/pcre-8.45-33dfhul
[+] /opt/apps/spack/libpciaccess-0.17-2qdxdjo
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/qhull-2020.2-klc7ewb
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/alsa-lib-1.2.3.2-a7icjdy
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/yasm-1.3.0-v4etmmm
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/unzip-6.0-mqftjtp
[+] /opt/apps/spack/libjpeg-turbo-3.0.0-vjvivme
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/libpng-1.6.39-kiuku4y
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/swig-4.0.2-fortran-z3sbnze
[+] /opt/apps/spack/binutils-2.42-vkjcvfr
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/hdf5-1.14.3-sbuiw6q
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/ffmpeg-6.1.1-2vhrbda
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/freetype-2.13.2-in4taxi
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/hwloc-2.9.1-fhoz6al
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/lua-5.3.6-47356ia
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/elfutils-0.190-uswzaiw
[+] /opt/apps/spack/py-pytz-2023.3-ojdlhrd
[+] /opt/apps/spack/py-cycler-0.11.0-b7mjvv7
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/llvm-14.0.6-3nosumn
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-six-1.16.0-iv6iv3q
[+] /opt/apps/spack/py-pybind11-2.12.0-5rvupjv
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-pyparsing-3.1.2-fq6imlt
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/py-llvmlite-0.41.1-qom3l5h
[+] /opt/apps/spack/py-astropy-4.0.1.post1-xbojixg
[+] /opt/apps/spack/py-python-dateutil-2.8.2-kzdfskc
[+] /opt/apps/spack/py-numexpr-2.8.4-fc5xc5s
[+] /opt/apps/spack/py-h5py-3.11.0-y6drk3j
[+] /opt/apps/spack/py-tifffile-2023.8.30-oof4try
[+] /opt/apps/spack/py-pygments-2.16.1-stgrccl
[+] /opt/apps/spack/py-mdurl-0.1.2-nqk43ry
[+] /opt/apps/spack/py-scipy-1.13.0-vxjgfov
[+] /opt/apps/spack/py-contourpy-1.0.7-jg5lhss
[+] /opt/apps/spack/py-pillow-10.3.0-ijh2cju
[+] /opt/apps/spack/py-kiwisolver-1.4.5-vdh5sq5
[+] /opt/apps/spack/py-fonttools-4.39.4-x5ydbox
[+] /opt/apps/spack/py-bottleneck-1.3.7-ztsm4lg
[+] /opt/apps/spack/py-lazy-loader-0.4-k7hgvka
[+] /opt/apps/spack/py-numba-0.58.1-hzvrjwj
[+] /opt/apps/spack/py-markdown-it-py-3.0.0-l4p4qv5
[+] /opt/apps/spack/py-imageio-2.34.0-z5hu4yc
[+] /opt/apps/spack/py-matplotlib-3.8.4-azq2fzm
[+] /opt/apps/spack/py-pandas-1.5.3-p3gnh6t
[+] /opt/apps/spack/py-rich-13.4.2-okhgwwz
[+] /opt/apps/spack/py-networkx-3.1-b54br7r
[+] /opt/apps/spack/py-scikit-image-0.23.2-cvyzh3t
==> Installing py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x [83/83]
==> No binary for py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x found: installing from source
==> Fetching https://github.com/thepetabyteproject/your/archive/refs/tags/0.6.7.tar.gz
==> No patches needed for py-your
==> py-your: Executing phase: 'install'
==> py-your: Successfully installed py-your-0.6.7-djfzsn2lutp24ik6wrk6tjx5f7hil76x
Stage: 1.43s. Install: 0.99s. Post-install: 0.12s. Total: 3.12s
* Removed setup_run_environment
After some testing, both spack load and module load for the package will include the bin directory generated by py-your as well as the path to the version of python the package was built with, without the need for the setup_run_environment function.
I removed that function (Although, like Tamara I thought it would be necessary based on other package setups I used as a basis for this package).
Note: I also updated the required version of py-astropy from py-astropy@4.0: to @py-astropy@6.1.0: In my test builds, the install was picking up version py-astropy@4.0.1.post1 and numpy1.26. However when I tried to run some of the code I was getting errors about py-astropy making numpy calls that are now removed. The newer version of py-astropy corrects these. Ideally this would be handled in the py-astropy package to make sure numpy isn't too new
#40773 introduced python-venv, which improved build isolation and avoids issues with,
e.g., `ubuntu`'s system python modifying `sysconfig` to include a (very unwanted)
`local` directory within the default install layout.
This addresses a few cases where #40773 removed functionality, without harming the
default cases where we use `python-venv`.
Traditionally, *every* view with `python` in it was essentially a virtual environment,
because we would copy the `python` interpreter and `os.py` into every view when linking.
We now rely on `python-venv` to do that, but only when it's used (i.e. new builds) and
only for packages that have an `extends("python")` directive.
This again makes every view with `python` in it a virtual environment, but only
if we're not already using a package like `python-venv`. This uses a different
mechanism from before -- instead of using the `virtualenv` trick of copying `python`
into the prefix, we instead create a `pyvenv.cfg` like `venv` (the more modern way
to do it).
This fixes two things:
1. If you already had an environment before Spack `v0.22` that worked, it would
stop working without a reconcretize and rebuild in `v0.22`, because we no longer
copy the python interpreter on link. Adding `pyvenv.cfg` fixes this in a more
modern way, so old views will keep working.
2. If you have an env that only includes python packages that use `depends_on("python")`
instead of `extends("python")`, those packages will now be importable as before,
though they won't have the same level of build isolation you'd get with `extends`
and `python-venv`.
* views: avoid making client code deal with link functions
Users of views and ViewDescriptors shouldn't have to deal with link functions -- they
should just say what type of linking they want.
- [x] views take a link_type, not a link function
- [x] views work out the link function from the link type
- [x] view descriptors and commands now just tell the view what they want.
* python: simplify logic for avoiding pyvenv.cfg in copy views
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
In #30882, we made Spack ignore `-Werror` calls so that it could more easily build
projects that inject `-Werror` into their builds. We did this by translating them to
`-Wno-error` in the compiler wrapper. However, some compilers (like `nvhpc`) do not
support `-Wno-error`. We need to exclude them from this feature until they do.
- [x] make a property on `PackageBase` for `keep_werror` that knows not to use it for
`nvhpc`.
- [x] update property so that it keeps only the specific `-Werror=...` args for newer nvhpc's,
which support `-Wno-error` but not `-Wno-error=...`
---------
Co-authored-by: William Mou <william.mou1024@gmail.com>
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
When Spack concretizes environments, it prints every (newly concretized) root spec
individually with all of its dependencies. For most reasonably sized environments, this
is too much output. This is true for three commands:
* `spack concretize` when concretizing an environment with newly added specs
* `spack install` when installing an environment with newly added specs
* `spack spec` with no arguments in an environment
The output dates back to before we had unified environments or nicer spec traversal
routines, and we can improve it.
This PR makes environment concretization output analogous to what we do for regular
specs. Just like `spack spec` for a single spec, we show all root specs with no
indentation, so you can easily see the specs you explicitly requested. Dependencies are
shown:
1. With indentation according to their depth in a breadth-first traversal starting at
the roots;
2. Only once if they appear on paths from multiple roots
So, the default is now consistent with `spack spec` for one spec--it's `--cover=nodes`.
i.e., if there are 100 specs in your environment, you'll get 100 lines of output.
If you want to see more details, you can do that with `spack spec` using the arguments
you're already familiar with. For example, if you wanted to see dependency types and
*all* dependencies, you could use `spack spec -l --cover=edges`. Or you could add
deptypes and namespaces with, e.g. `spack spec -ltN`.
With no arguments in an environment, `spack spec` concretizes (if necessary) and shows
the concretized environment. If you run `spack concretize` *first*, inspecting the
environment repeatedly with `spack spec` will be fast, as everything is already in the
`spack.lock` file.
- [x] factor most logic of `Spec.tree()` out of `Spec` class into `spack.spec.tree()`,
which can take multiple specs as roots.
- [x] make `Spec.tree()` call `spack.spec.tree()`
- [x] `spack.environment.display_specs()` now uses `spack.spec.tree()`
- [x] Update `spack concretize`
- [x] Update `spack install`
- [x] Update `spack spec` to call `spack.spec.tree()` for environments.
- [x] Continue to output specs individually for `spack spec` when using
`--yaml` or `--json`
* Apply patch from upstream to allow vtk compilation with %gcc 12 and 14.
* Fixes#44331
* fix soec usage
* fix compiler version range
* Finalize version range, switch to .diff file
---------
Co-authored-by: Chrismarsh <Chrismarsh@users.noreply.github.com>
* Now need to explicitly depend on libfabric for Cray EX
* Ensure build uses the selected CUDA and ROCm versions
* Correct spelling on `configure --with-ldflags`
* Patch a defect regarding `configure --with-ldflags`
* Default to Spack's copy of GASNet-EX, which is newer than embedded
e2fsprogs: adding version 1.47.0 and fuse2fs variant
fuse-overlayfs: adding version 1.13
squashfuse: adding version 0.5.1 and 0.5.2
e2fsprogs: fixing audit errors
apptainer: expanding deps to cover more versions, fix binary_path logic.
apptainer: add go version deps, fix e2fsprogs patch sums, rework dep paths
fuse-overlayfs: tightening up the libfuse dep
Signed-off-by: Shane Nehring <snehring@iastate.edu>
* util-macros: ensure url_for_version only used for older versions
* util-macros: use url.substitute_version after xz -> bz2
* util-macros: mv url_for_version down the file
`BuildcacheBootstrapper` uses `Spec.intersects` to match specs needed
for bootstrapping against the binary cache. The specs were not
sufficiently-detailed to prevent matching e.g. cached binaries for
Mac OS on Windows; this commit adds the platform to each requested
bootstrap spec to prevent that.
* initial commit for RoCm-6.1.1 release
* fix style issues
* update the version for rocmlir and rocm-cmake
* restrict the patch for 6.0.0 release
* fix build failure for hipsolver-6.1.1 release
* update the hipblaslt for rocm-6.1.1 release
* add the patch for hipsparselt for 6.1.1 rel
Remove support for `cray` as a separate platform.
Any platform previously detected as `cray` is now detected as `linux`.
Users who still need platform=cray have to stick to Spack 0.22
Fixes a bug in the concretizer where specs depending on a host
incompatible libc would be used. This bug triggers when nothing is
built.
In the case where everything is reused, there is no libc provider from
the perspective of the solver, there is only compatible_libc. This
commit ensures that we require a host compatible libc on any reused
spec, additionally to requiring compat with the chosen libc provider.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Patch to add hsa prefix/include path:wq
* Style check error fix
* The patch need to be applied to 2024.01.stable. ROCm 6.0 requires hpctoolkit 2024.01.1 or later
* Fixing style chec error
* Adding a py4j variant that requires Java via spack to avoid situations where a system doesn't have Java and py4j expects it
* Adding new versions of py-pyspark
* Adding a new variant to require java (via py4j) and clean up dependency handling
* Adding myself as a maintainer for py-pyspark and py-py4j
* Fix overlooked version bump in py4j
* Version bump to meet py-spark expectations
* Version bump to add latest compatibile version with pyspark
* Matching py-grpcio bump
* Adding variants and dependents for pyspark
* Adding runtime deps
* Changing default java requirement. I'm not sure this is the right call
* Changing py4j with java dependency handling
* Fix style
* Update package.py fix unnecessary f-strinh
* Make +java the default for both
* Fix nested deps
* Revert styles after default change
* Added new versions and GCC 14 compatbility conflicts
* Added new versions and compatibility conflicts for gcc 14
* Added new versions paired to arrow (for gcc14 compat)
* Update py-protobuf compiler conflict
* Update depends to match
See https://github.com/grpc/grpc/blob/master/src/python/grpcio_status/setup.py
* Updating dependencies and conflicts for py-googleapis-common-protos. Added new version to avoid future issues
* Remove upper bound version on py-protobuf and add default_args
* Adding new versions and updating dependencies back to versions 1.35.0
* Updating oldest numpy deps
* Fixing merge
* bit more cleaniness for var/spack/repos/builtin/packages/py-googleapis-common-protos/package.py
* Adding latest matching version of py-grpcio and py-grpcio-status
* Update package.py
https://github.com/spack/spack/pull/44263#discussion_r1612317943
* Update dependencies
* Adding additional versions for dependent packages. Deprecated two versions: 1.16 is old, built for python ~3.6, and does not build for 3.8. 1.52.0 was removed from pypi
* Revert py-grpcio-tools changes. Will include in separate PR
* Adding patches and constraints to get 1.48 to build as it's a dependency that is called out for some other packages
* Updating to account for yanked packages for dependencies
* Fix style
* Update sha256 for py-grpcio v0.16.0 to reflect change
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* py-astropy: Add version 6.1
Added build info for version 6.1 in py-astropy. Requires a new additional package, astropy-iers-data which has been included as py-astropy-iers-data to match with spack's general naming conventions.
Below is the output of the spack install showing successful build fro version 6.1 and the new py-astropy-iers-data package
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-pyerfa-2.0.0.1-kyfazhs
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
[+] /opt/apps/spack/git-2.45.1-tuc5jnb
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
==> Installing py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj [52/55]
==> No binary for py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj found: installing from source
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-main-ukchsfzhfcyz6e6fxar6mtykqiavporj
Stage: 1.74s. Install: 0.93s. Post-install: 0.52s. Total: 3.36s
[+] /opt/apps/spack/py-astropy-iers-data-main-ukchsfz
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-qdhxchg
==> Installing py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo [55/55]
==> No binary for py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> Warning: Module file /opt/modulefiles/spack/Core/py-astropy/6.1.0.lua exists and will not be overwritten
==> py-astropy: Successfully installed py-astropy-6.1.0-f4pffru3kmyion2kq6muomgrfs5y4gdo
Stage: 1.29s. Install: 1m 5.77s. Post-install: 0.60s. Total: 1m 7.94s
* Removed extra-whitespace
A blank line had white space, removed the white space
* Additional formatting changes for black
* Additional package updates
Based on previous recommendations updated py-astropy and py-astropy-iers-data packages. Also added a new version to py-pyerfa package to match the 6.1.0 dependencies better
Of importance in these updates, I did add pypi and version info to py-astropy-iers-data. Originally I had argued that this package updates quite frequently (on a weekly basis) and so it did not make sense to include pypi/versions and we should instead use the non-version git-repo structure based on the master branch. However, when I tried to build the package py-setuptools-scm errored out when trying to build the package:
/opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro/lib/python3.10/site-packages/setuptools_scm/git.py:163: UserWarning: "/tmp/root/spack-stage/spack-stage-py-astropy-iers-data-main-iw2mdzlukb37mkcbcozjjefjoefw2eyp/spack-src" is shallow and may cause errors
I believe this is due to the download file/stage directory not containing the version and instead including the branch. I changed the package to use versions instead and it worked just fine as shown below. In addition, when I had done some preliminary testing, the package installed fine using the non-version master branch. When I checked, that installation it used py-setuptools-scm@7.1 while in this installation run it used a much more recent 8.0.4 so it is possible that somewhere between scm7.1 and 8.0.04 something changed that caused this error to show up. Since the setuptools-scm package has something to do extracting package versions, I imagine it's some kind of mismatch issue
Output from build:
spack install py-astropy
[+] /usr (external glibc-2.28-oj2wjfl2ao5inhfz4qehw6hlck2hizvp)
[+] /opt/apps/spack/gcc-runtime-8.5.0-5k6kvi5
[+] /opt/apps/spack/wcslib-7.3-zvcqq7o
[+] /opt/apps/spack/xz-5.4.6-axoznvt
[+] /opt/apps/spack/libffi-3.4.6-ibucrfe
[+] /opt/apps/spack/erfa-2.0.0-4qkta2n
[+] /opt/apps/spack/libmd-1.0.4-zbdiprt
[+] /opt/apps/spack/util-linux-uuid-2.38.1-w3kgjq3
[+] /opt/apps/spack/libiconv-1.17-jskazis
[+] /opt/apps/spack/berkeley-db-18.1.40-jftva2u
[+] /opt/apps/spack/zstd-1.5.6-nyk6gt6
[+] /opt/apps/spack/ncurses-6.4-xbvwv2w
[+] /opt/apps/spack/bzip2-1.0.8-t65bq3t
[+] /opt/apps/spack/libunistring-1.2-whrov3e
[+] /opt/apps/spack/pcre2-10.42-fu62kky
[+] /opt/apps/spack/pkgconf-1.9.5-ckjdqjm
[+] /opt/apps/spack/zlib-ng-2.1.6-ccn5qny
[+] /opt/apps/spack/openblas-0.3.26-pfyk2vi
[+] /opt/apps/spack/libxcrypt-4.4.35-zigqpjo
[+] /opt/apps/spack/libyaml-0.2.5-fxathvq
[+] /opt/apps/spack/libbsd-0.12.1-njt5grs
[+] /opt/apps/spack/readline-8.2-2ys6ede
[+] /opt/apps/spack/libidn2-2.3.7-vnie4rz
[+] /opt/apps/spack/nghttp2-1.57.0-u72gxms
[+] /opt/apps/spack/libedit-3.1-20230828-676jwbd
[+] /opt/apps/spack/libxml2-2.10.3-37klvxv
[+] /opt/apps/spack/openssl-3.2.1-4lqdgni
[+] /opt/apps/spack/pigz-2.8-rx263bp
[+] /opt/apps/spack/expat-2.6.2-7kfe3hb
[+] /opt/apps/spack/sqlite-3.43.2-axuxulg
[+] /opt/apps/spack/gdbm-1.23-cylmqwx
[+] /opt/apps/spack/curl-8.6.0-gpzsr3p
[+] /opt/apps/spack/tar-1.34-wjzs4wj
[+] /opt/apps/spack/perl-5.38.0-gzljgek
[+] /opt/apps/spack/cfitsio-3.49-mmy3dbr
[+] /opt/apps/spack/gettext-0.22.4-zjsp346
[+] /opt/apps/spack/krb5-1.20.1-tqiapsx
[+] /opt/apps/spack/python-3.10.13-fz7fymx
[+] /opt/apps/spack/openssh-9.7p1-jxrkzso
[+] /opt/apps/spack/py-tomli-2.0.1-eanxpu2
[+] /opt/apps/spack/py-setuptools-69.2.0-3do76jw
[+] /opt/apps/spack/py-numpy-1.26.4-t5acjcv
[+] /opt/apps/spack/python-venv-1.0-2cz5c3s
[+] /opt/apps/spack/py-packaging-23.1-wkeyuk6
[+] /opt/apps/spack/py-pip-23.0-lxkcvby
[+] /opt/apps/spack/git-2.45.1-zu6qkoc
[+] /opt/apps/spack/py-markupsafe-2.1.3-isgtki6
[+] /opt/apps/spack/py-wheel-0.41.2-brm3k3h
==> Installing py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a [49/57]
==> No binary for py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a found: installing from source
==> Fetching ac8a6fe91c.tar.gz
==> No patches needed for py-extension-helpers
==> py-extension-helpers: Executing phase: 'install'
==> py-extension-helpers: Successfully installed py-extension-helpers-0.1-a5hmr6jtrvpcq3ibwkwhvwlydthjif5a
Stage: 0.37s. Install: 0.88s. Post-install: 0.52s. Total: 1.89s
[+] /opt/apps/spack/py-extension-helpers-0.1-a5hmr6j
==> Installing py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m [50/57]
==> No binary for py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m found: installing from source
==> Fetching 350b18f967.tar.gz
==> No patches needed for py-cython
==> py-cython: Executing phase: 'install'
==> py-cython: Successfully installed py-cython-3.0.0-zx62ssdy4p6ddwuqbixel2vcsihjcs6m
Stage: 1.07s. Install: 2m 49.80s. Post-install: 0.40s. Total: 2m 51.37s
[+] /opt/apps/spack/py-cython-3.0.0-zx62ssd
[+] /opt/apps/spack/py-jinja2-3.1.2-wacpq7j
[+] /opt/apps/spack/py-typing-extensions-4.8.0-ujwbb6g
[+] /opt/apps/spack/py-pyyaml-6.0-rju7jls
[+] /opt/apps/spack/py-setuptools-scm-8.0.4-ax2zqro
==> Installing py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 [55/57]
==> No binary for py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6 found: installing from source
==> Using cached archive: /opt/spack/var/spack/cache/_source-cache/archive/7f/7fff3d3404ae8560533ac0e685db7acc02c4d8984faa4ac3d607096879fba2d1.tar.gz
==> No patches needed for py-astropy-iers-data
==> py-astropy-iers-data: Executing phase: 'install'
==> py-astropy-iers-data: Successfully installed py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6icqn3aqhyjidfir3byyjq5aq6
Stage: 0.07s. Install: 1.21s. Post-install: 0.39s. Total: 1.88s
[+] /opt/apps/spack/py-astropy-iers-data-0.2024.5.20.0.29.40-wckxjf6
==> Installing py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp [56/57]
==> No binary for py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/p/pyerfa/pyerfa-2.0.1.1.tar.gz
==> No patches needed for py-pyerfa
==> py-pyerfa: Executing phase: 'install'
==> py-pyerfa: Successfully installed py-pyerfa-2.0.1.1-pkokp6usk7m2bjxcba3nwgqgrjufumcp
Stage: 0.93s. Install: 17.72s. Post-install: 0.33s. Total: 19.15s
[+] /opt/apps/spack/py-pyerfa-2.0.1.1-pkokp6u
==> Installing py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs [57/57]
==> No binary for py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs found: installing from source
==> Fetching https://files.pythonhosted.org/packages/source/a/astropy/astropy-6.1.0.tar.gz
==> Ran patch() for py-astropy
==> py-astropy: Executing phase: 'install'
==> py-astropy: Successfully installed py-astropy-6.1.0-5brbkjnjzfg3lc6h34qku24ep5pwsxzs
Stage: 1.16s. Install: 1m 3.34s. Post-install: 1.05s. Total: 1m 5.77s
[+] /opt/apps/spack/py-astropy-6.1.0-5brbkjn
* Updated to match with black formatting
Change the installer to take `([pkg], args)` in the constructor instead
of `[(pkg, args)]`. The reason is that certain arguments are global
settings, and the new API ensures that those arguments cannot be
different across different "build requests".
The `explicit` install arg is now a list of hashes, and the installer is
no longer responsible for determining what package is installed
explicitly. This way environment installs can simply pass the list of
environment roots, without them necessarily being explicit build
requests. For example an env with two roots [a, b], where b depends on
a, would not always cause spack install to mark b as explicit.
Notice that `overwrite` already took a list of hashes, this makes
`explicit` consistent.
`package.do_install(explicit=True)` continues to take a boolean.
* legion: add pip dependency for build
pip is needed for the build when the Legion Python binding is enabled
* legion: add variants for gc logging and system OpenMP use
This also removes the `cmake_cxx_flags` variable from `cmake_args()`
because that variable had no effect.
* [@spackbot] updating style on behalf of mpokorny
* legion: use spec.satisfies() in cmake_args()
e.g, replace use of '"+foo" in spec' with 'spec.satisfies("+foo")'
* legion: avoid configuring with multiple "-DLegion_REDOP_COMPLEX=ON" arguments
In the previous version, when both '+redop_complex' and '+bindings'
was set, two instances of "-DLegion_REDOP_COMPLEX=ON" arguments were
generated for cmake configuration.
* legion: fix value of "Legion_OUTPUT_LEVEL" for configuration
the previous version had no effect on setting the configuration value
---------
Co-authored-by: mpokorny <mpokorny@users.noreply.github.com>
The windows wrappers for basic functions like `os.symlink`,
`os.readlink` and `os.path.islink` in the `llnl.util.symlink` module
have bugs, and trigger more file system operations on non-windows than
they should.
This commit just binds `llnl.util.symlink.symlink = os.symlink` etc so
built-in functions are used on non-windows
`spack clean <spec>` will now resolve specs based on the active environment if one is active.
If an env is active but no matching spec is found, this will fall back on fully concretizing.
Before this PR, if Spack could see a possibility to reuse a spec that
doesn't match a strong preference, it would do so. After the PR, a
strong preference would take precedence.
`glibc` and `musl` provide a basic implementation of `iconv` (`iconv`,
`iconv_open`, `iconv_close`), but in practice the installation may be
missing the character encoding methods to make them usable. On Fedora
for example, users need to
```yum install glibc-gconv-extra```
to get the character encodings that `gettext` requires during configure,
namely EUC-JP. Users may not have permissions to install the missing
parts of glibc.
Since Spack can install `libiconv`, it is simpler to use that by
default.
* Spglib: add version 2.4.0
* dla-future-fortran: new package version 0.1.0
* [@spackbot] updating style on behalf of RMeli
* apply suggestion and add maintainer
---------
Co-authored-by: RMeli <RMeli@users.noreply.github.com>
* [emacs] When installing on Mac OS, build and install the native
Emacs.app along with the standard executables.
* [emacs] Make the GUI build on Mac optional by adding "gui" variant
* Apply reviewer suggestion
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Add emacs version 29.3
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
avoid calling `spec.target` when None.
When an external compiler package has an `os` set but no `target` set, Spack
currently falls into a codepath that calls `spec.target` (which itself calls
`spec.architecture.target.Microarchitecture`) when `spec.architecture.target`
is None, throwing an error.
e.g.
```
packages:
gcc:
externals:
- spec: gcc@12.3.1 os=rhel7
prefix: /usr
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* glew: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-demos: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* mesa-glu: rework dependency on gl
This simplifies the package and ensures a single gl implementation is
pulled in. Before we were adding direct dependencies, and those are
not unified through the virtual.
* paraview: fix dependency on glew
* mesa: group dependency on when("+glx")
* Add missing dependency on libxml2
* paraview: remove the "osmesa" and "egl" variant
Instead, enforce consistency using the "gl" virtual that allows
only one provider.
* visit: remove osmesa variant
* Disable paraview in the aws-isc stacks
* data-vis-sdk: rework constrains to enforce front-ends
* e4s-power: remove redundant paraview
* Pipelines: update osmesa variants
* trilinos-catalyst-ioss-adapter: make gl a run dependency
Starting from 24.04, armpl-gcc will only have three packages files: dev, rpm, and macOS.
Only one version for gcc is provided, so the changes in the code reflect that the tar provided
for gcc is only one rather than many.
* Remove mesa18 and libosmesa
mesa18 was introduced in #19528 as a way to maintain the old
autotools build of mesa separate from the new meson build.
We could add a second build system to mesa, but since mesa18 has
been deprecated for a long time, we'll just remove it.
libosmesa was used to multiplex the gl provider between mesa18
and mesa, and is thus unecessary. Remove it to reduce complexity
in the graphical stack.
* Remove references to mesa18 and libosmesa
* vtk: rework dependency on gl and osmesa
* memsurfer: rework dependency on vtk
* visit: minimal fix to avoid having both osmesa and glx
* Use hdf5::hdf5 on Windows from Paraview CMake
This patch is already applied on VTK 9 or greater.
* Add comments stating that vtk and paraview patches are the same and should be modified in concert.
This fixes a bug occurring when two root specs need to select
old versions, and these versions have the same penalty in the
optimization. This sometimes caused an older version to be
preferred to a more recent one.
The issue was the omission of `PackageNode` in the optimization
tuple.
* rayleigh: new package
* Update var/spack/repos/builtin/packages/rayleigh/package.py
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
* split edit into three methods
* add comments to clarify use of configure
* rayleigh: copyright year
---------
Co-authored-by: Wouter Deconinck <wdconinc@gmail.com>
Use correct path separator in get_all_package_diffs for all platforms.
Ensures correct package change computation on Windows when pruning unchanged specs in Gitlab CI
* Update, fix serenity & serenity-libint
Version 1.6.1 of `serenity` is added, some dependencies are made more
explicit, some options are improved or fixed.
The url of `serenity-libint` is fixed. The old url is not reachable
anymore.
* Use upstream patch, modify cmake patch
* Update var/spack/repos/builtin/packages/serenity/package.py
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* heimdall: Astronomy software package. Requires dedisp and psrdada (included as part of this commit)
* Updated packages to align with Spack's style
Minor updates based on wdconinc's comments regarding Spack's style guide
* [@spackbot] updating style on behalf of aweaver1fandm
* Minor edits to fix copyright year and dedisp install
Fixed copyright year to be 2024 instead of 2023
Removed the overridden version of install and created a preinstall function to create the missing lib and inc directories, therefore allowing the default install to run
Here is output from the spack install of heimdall showing successful build with cuda. If necessary I do a spack clean and freshly install it
./spack install heimdall +cuda cuda_arch=86
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libiconv-1.17-enpmbhsi3kztebwmpclpub2afhlbr3gy
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/xz-5.4.1-pte76kujkezxb3laqse3o4sctlbygsaw
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zlib-1.2.13-utlfo5ltxz5v5bckirn5v3amtbxjdvwh
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/bzip2-1.0.8-x6navz7ucgfnb5xq7aelqmgd4zxsz5bs
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libmd-1.0.4-ncomhrodpdul4dm64o6b7426fhmc2u64
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/ncurses-6.4-c77h34rooycbzapxjvc27sg5td5jiwyb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/zstd-1.5.5-eporpybumydxveg5rwtfzysrsu4eqzcv
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libffi-3.4.4-vskntokaojclfqxjfzkbyirkeogddbpx
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxcrypt-4.4.33-gtpn32p6mxztul3c3dxzqj7gvcyh555j
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/util-linux-uuid-2.38.1-g322a5peqjaad6gl5q64cdu4qo7kvw6o
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libxml2-2.10.3-pza3kz2mtbncbbeim6rejfqkgftnf4rz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/openssl-1.1.1t-adbquvgg4qpc3vq6jynf44qzq3gfwrv5
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/pigz-2.7-e7mxj4ya2u4a6zb4hu64g7docujmkxeb
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/libbsd-0.11.7-wh4xleivbe7wndiqt5nsehzlfrccnjcg
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/readline-8.2-hhb647bwmbcj7iwpmtetbylninfm5rxf
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/cuda-11.7.1-osue2sx5rv7dgzhsmaemydpwhyribxng
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/tar-1.34-35f5gki2ycxmy5zd7zs5tsvp3xoszxum
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/expat-2.5.0-4gizyhhqklciuyrbyinq2tdggt73gds4
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gdbm-1.23-w3uzihtubj2iwv6es55fis6nt2q5zwlr
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/sqlite-3.40.1-m4ntzvuupsnbtkdfbz7oqpbjdlaffp2a
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/dedisp-1.0-pq6r3jnyxq6jzoygz3fp2e6jc2ojpvap
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/gettext-0.21.1-hglzdeadmgkzjb76bmemt6dnulfkrpha
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/python-3.10.10-khu36qq4p2te7jf475ewr2h7egidekfl
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/psrdada-master-by4w6mrpcfnoihlhos2jcfo2roiyagaz
[+] /home/htony/spack/opt/spack/linux-ubuntu22.04-zen3/gcc-11.3.0/heimdall-1.0-ohtdnltuhhejysshcert25h6nmuvluqp
* [@spackbot] updating style on behalf of aweaver1fandm
Restrict the hpcrun-fmt.txt patch to :2022. It's fixed in the code
after that, and in recent develop, some code paths have moved causing
this patch to fail.
This fixes an issue where ghcr, gitlab and possibly other container registries paginate tags by default, which violates the OCI spec v1.0, but is common practice (the spec was broken itself). After this commit, you can create build cache indices of > 100 specs on ghcr.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
* pythia8: prefer 8.310
* [@spackbot] updating style on behalf of wdconinc
* pythia8: filter_file to remove sed n
* Revert "[@spackbot] updating style on behalf of wdconinc"
This reverts commit e2a3decaffbd3f464d1bd992025e1812df49f088.
* Revert "pythia8: prefer 8.310"
This reverts commit 568cb056b87129085e245d9dbef1732ee1c6c0aa.
* [@spackbot] updating style on behalf of wdconinc
* pythia8: comment for fix
* pythia8: fix style
* pythia8: filter_file with raw string because of escaped pipe
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
The release tarballs are not available anymore which means autoconf, automake and libtool are always needed.
The NVHPC specific patches don't make sense anymore being that the patched files are not distributed in the new tar files.
Add support for Gitlab CI on Windows
This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows
infrastructure).
* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Symlinks on Windows can use longpath prefixes (\\?\); these are fine
in the context of win32 API interactions but break numerous facets of
Spack behavior that rely on string parsing/matching (archiving,
binary distributions, tarball extraction, view regen, etc).
Spack's internal readlink method (llnl.util.symlink.readlink)
gracefully handles this by removing the prefix and otherwise behaving
exactly as os.readlink does, so we should prefer that in all cases.
* py-matplotlib: qualify when to do a post install
Older versions of py-matplotlib don't seem to have some of the
files that the post install step is trying to install.
Looks like the files first appeared in 3.6.0 and later.
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
* Change install paths for older matplotlib
---------
Signed-off-by: Howard Pritchard <hppritcha@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add maintainer and fix linting
* allow for fewer deps in archive
* use meson for archive packages
* Fix version spec and f-string
* fix blas dependency links
* Add new release to spack
* Fix checksums for latest release
* Update version of iperf3 from 3.6 to 3.16
Spack currently only explicitly has version 3.6 of the iPerf3 package
(out of ESnet / LBNL). This makes the default the latest version of 3.16,
and adds some other versions (found in some Linux distros, for possible
compatibility purposes).
* iperf3: update to 3.17; update 3.6 hash for new url
* protobuf: update hash for patch needed when="@3.4:3.21"
* Revert "protobuf: update hash for patch needed when="@3.4:3.21""
This reverts commit 4d168d0b27.
* initial commit to enable tensorflow-2.14 for spack built ROCm
* fix style errors
* modify hipcc patch
* updates for rocm 6.1
* updates for tf-rocm-enhanced 2.16
* fix styling
* fix styling
* add patch for 2.16
* add patch for 2.16
* Update var/spack/repos/builtin/packages/py-tensorflow/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* update rocm enhanced version names
* changes for rocm-enhanced version name change
* fix styling
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix gromacs installation with SVE. Issue 44062
* [@spackbot] updating style on behalf of paolotricerri
* Remove `neoverse_n2` target
We have removed the neoverse_n2 target as its detection is more involved
compared to neoverse_v*.
Set MPI_C_COMPILER etc. env vars when building with fujitsu-mpi, which
override CMake logic. Without using these variables to explicitly
request the Fujitsu MPI wrappers, the builtin CMake logic is unwilling
to use these wrappers unless the Fujitsu compiler is used, but they
should be used for %clang as well.
This avoids patching CMake module files like in #16864, primarily to
avoid the possibility of altering behavior for specs that do not use
%fj or ^fujitsu-mpi.
* Add openfoam to aws-pcluster-neoverse_v1 stack
* Add more apps to aws-pcluster-x86_64_v4 stack
* Remove WRF while hdf5 cannot build in buildcache at the moment
* Update comment
* Add more apps for aws-pcluster-neoverse_v1 stack
* Remove apps that currently do not build
* Disable those packages that won't build
* Modify syntax such that correct cflags are used
* Changing syntax again to what works with other packages
* Fix overriding packages.yaml entry for gettext
* Use new `prefer` and `require:when` clauses to clarify intent
* Use newer spack version to install intel compiler
This removes the need for patches and makes sure the `prefer` directives in
`package.yaml` are understood.
* `prefer` not strong enough, need to set compilers
* Revert "Use newer spack version to install intel compiler"
This reverts commit ecb25a192c.
Cannot update the spack version to install intel compiler as this changes the
compiler hash but not the version. This leads to incompatible compiler paths. If
we update this spack version in the future make sure the compiler version also updates.
Tested-by: Stephen Sachs <stesachs@amazon.com>
* Remove `prefer` clause as it is not strong enough for our needs
This way we can safely go back to installing the intel compiler with an older
spack version.
* Prefer gcc or oneapi to build gettext
* Pin gettext version compatible with system glibc-headers
* relax gettext version requirement to enable later versions
* oneapi cannot build older gettext version
* octave: use pcre2 for @8:
* Add 'pcre2' variant to octave to control pcre vs. pcre2
* Update var/spack/repos/builtin/packages/octave/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alex Richert <alexander.richert@noaa.gov>
Co-authored-by: Alec Scott <hi@alecbcs.com>
Apparently urllib can throw a range of different exceptions:
1. HTTPError
2. URLError with e.reason set to the actual exception
3. TimeoutError from getresponse, which is not wrapped
* New package: qpdf
* Update var/spack/repos/builtin/packages/qpdf/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
* Fix format of cmake_args
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
* archive: relative links only
Ensure all links written into tarfiles generated from Spack prefixes do not contain symlinks pointing outside the prefix
* binary_distribution: limit extraction to prefix
Ensure files extracted from spackballs are not links pointing outside of the prefix
* Ensure rpaths are properly set on Windows
* hard error on extraction of absolute links
* refactor for non link-modifying approach
* Restore tarball extraction to original impl
* use custom readlink
* cleanup symlink module
* make lstrip
* flux-sched: set the version if the ver file is missing
problem: flux-sched needs a version, it normally gets this from a
release tarball or from git tags, but if using a source archive or a git
clone without tags the version is missing
solution: set the version through cmake based on the version spack sees
when the version file is missing
* Update var/spack/repos/builtin/packages/flux-sched/package.py
Co-authored-by: Alec Scott <hi@alecbcs.com>
---------
Co-authored-by: Alec Scott <hi@alecbcs.com>
Problem: the libsodium download endpoints are not reliable,
they fail multiple times a day for several of our automated
pipelines.
Solution: use the GitHub releases and branches.
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
GCC C++ headers like cstdlib use `#include_next <stdlib.h>` to wrap libc
headers. We're using `-isystem` for libc, which puts those headers too
early in the search path. `-idirafter` fixes this so `include_next`
works.
* add darshan-3.4.5 packages, update URLs
* py-setuptools version switches for py-darshan
* more py-darshan test dependencies
* try a conditional importlib_resources dependency
Add the ability to include any number of (potentially nested) concrete environments, e.g.:
```yaml
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
or, from the CLI:
```console
$ spack env create myenv
$ spack -e myenv add python
$ spack -e myenv concretize
$ spack env create --include-concrete myenv included_env
```
The contents of included concrete environments' spack.lock files are
included in the environment's lock file at creation time. Any changes
to included concrete environments are only reflected after the environment
is re-concretized from the re-concretized included environments.
- [x] Concretize included envs
- [x] Save concrete specs in memory by hash
- [x] Add included envs to combined env's lock file
- [x] Add test
- [x] Update documentation
Co-authored-by: Kayla Butler <<butler59@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co
m>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Currently SPACK_COLOR=always is not respected in the build process on
macOS, because the global `_force_color` is re-evaluated in global scope
during module setup, where it is always `None`.
So, move global init bits from main.py to the module itself.
Some specs which were excluded from reuse,
are currently added back to the solve when
we traverse dependencies of other reusable
specs.
This fixes the issue by keeping track of what
we can explicitly reuse.
This commit adds a layer of indirection to improve build isolation with
and without external Python, as well as usability of environment views.
It adds `python-venv` as a dependency to all packages that `extends("python")`,
which has the following advantages:
1. Build isolation: only `PYTHONPATH` is considered in builds, not
user / system packages
2. Stable install layout: fixes the problem on Debian, RHEL and Fedora where
external / system python produces `bin/local` subdirs in Spack install prefixes.
3. Environment views are Python virtual environments (and if you add
`py-pip` things like `pip list` work)
Views work whether they're symlink, hardlink or copy type.
This commit additionally makes `spec["python"].command` return
`spec["python-venv"].command`. The rationale is that packages in repos we do
not own do not pass the underlying python to the build system, which could still
result in incorrectly computed install layouts.
Other attributes like `libs`, `headers` should be on `python` anyways and need no change.
Currently bootstrapping from source fails because clingo requires gnupg
requires clingo.
This commit stops eager bootstrapping. We don't need `patchelf` nor `gnupg`
generally. They're bootstrapped when needed.
This creates shared infrastructure for compiler packages to implement the
detailed search capabilities from the `spack compiler find` command for the
`spack external find` command.
After this commit, `spack compiler find` can be replaced with
`spack external find --tag compiler`, with the exception of mixed toolchains.
A named env cannot contain `.` and `/`.
So when a user runs `spack env create ./here` do not error but treat it
as `spack env create -d ./here`.
Also fix help string of `spack env create`, which seems to have been
copied from `activate` incorrectly.
Since reuse is the default now, `--reuse-deps` can be confusing, as it
technically does not imply roots are fresh.
So add `--fresh-roots`, which is also easier to discover when running
`spack concretize --fre<tab>`
* WIP NWChem with ARMCI-MPI and TCE optional
* rename armci-mpi to armcimpi
* add version for git
* add ARMCI-MPI support to NWChem package
EXTERNAL_ARMCI_PATH needs to be set to the ARMCI-MPI package install prefix.
I could not get it from spec["armcimpi"] but it worked to use the dependent build environment method to export a generic environmental variable to use instead.
* i suppose i can maintain NWChem as well
* check rejects option that dozens of packages use
i do not have time to fight with this nonsense
Run . share/spack/setup-env.sh
==> Error: armcimpi version 'master' has extra arguments: 'branch'
Valid arguments for a url fetcher are:
'url', 'sha256', 'md5', 'sha1', 'sha224', 'sha384', 'sha512', and 'checksum'
Error: Process completed with exit code 1.
* style
* ARMCI-MPI needs depends_on; the rest seems fixed
* fix ARMCI selection per review feedback from @zzzoom
https://github.com/spack/spack/pull/43883#discussion_r1585014147
* address reviewer feedback from @yizeyi18
https://github.com/spack/spack/pull/43883#discussion_r1587228084
elaborate on what +extratce does, in terms of the NWChem build environment variables,
some of which are documented on https://nwchemgit.github.io/TCE.html.
* style
* Update var/spack/repos/builtin/packages/nwchem/package.py
---------
Signed-off-by: Jeff Hammond <jeff.science@gmail.com>
Co-authored-by: Carlos Bederián <4043375+zzzoom@users.noreply.github.com>
* [TAU package] Update with +rocprofv2. Updated some tests.
pdated with +rocprofv2 flag. Only works with rocm-core >= 6.0.0
Can be tested with (Omnia):
spack install tau@master +rocm+rocprofv2 %gcc@11
Needs the last commit in our local repository, can be done by modifying the "git = " line, or waiting until the public one is updated.
In the case that tests cause issues when building TAU, there is the flag:
disable_tests = False
The rocm test is disabled by default, as the PR regarding tests loading dependencies is not solved (PR#43682).
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
* py-tensorflow: fix aarch64 build
* [@spackbot] updating style on behalf of aweits
* patch format
* change patch strategy, actually get the logic correct in
the patch
* only patch 2.16.1 onwards for aarch64
* __CUDACC__ not __NVCC__
* !(defined(__NVCC__) && defined(__CUDACC__))
---------
Co-authored-by: aweits <aweits@users.noreply.github.com>
We recently switched to using the new ReadTheDocs with "addons". That includes its own
analytics, which is nice, but we also want to continue using our GA4 analytics.
Adding GA4 is no longer supported by RTD, so we have to add it manually.
- [x] re-add the gtag to all pages, manually
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
PR #43701 is broken, preventing scorep from being installed. As explained in #43700 the issue is internal to scorep and can not be fixed in the package, at least by the method being attempted here.
Adds a pre-concretization check for the Windows SDK and WGL (Windows
GL) packages as non-buildable externals.
This is a redo of https://github.com/spack/spack/pull/43459, but makes
sure to modify the configuration scope outside of the bootstrap scope:
whichever is highest-precedence in the user's environment at the time
the concretization runs, which should either be an env scope or the
~ scope.
Adds pytest fixture mocking the check for WGL and WSDK as if they were
present.
This PR gives users finer control over which specs are reused during concretization.
The value of the `concretizer:reuse` config option now can take an object with the following properties:
- `roots`: true if reusing roots, false if reusing just dependencies
- `exclude`: list of constraints used to select reusable specs
- `include`: list of constraints used to select reusable specs
- `from`: allows to select the sources of reused specs
### Examples
#### Reuse only specs compiled with GCC
```yaml
concretizer:
reuse:
roots: true
include:
- "%gcc"
```
#### `openmpi` must be used from externals, and it must be the only external used
```yaml
concretizer:
reuse:
roots: true
from:
- type: local
exclude:
- "openmpi"
- type: buildcache
exclude:
- "openmpi"
- type: external
include:
- "openmpi"
```
With v2.10+, ADIOS added a campaign manager. This is auto-enabled
if SQLite3 is found.
Add explicit control for it now and disables it by default, to avoid
picking up system dependencies or bloating by default the ADIOS2
dependencies. Also, not yet fully mature to be used by default:
https://github.com/ornladios/ADIOS2/issues/4148
* pypi build of py-simpy
* [py-simpy] toml explicitly called out
* py-simpy: New version
* py-setuptools-scm: Added new version
* py-setuptools-scm: add url_for_version
Because versions @:7 have an underscore (setuptools_scm) in the URL, and
newer versions have a dash (setuptools-scm).
* py-setuptools-scm: fix flake8 line too long
* py-simpy: Fix hash
---------
Co-authored-by: Sid Pendelberry <sid@rit.edu>
Co-authored-by: Jen Herting <jen@herting.cc>
* PackageStillNeededError: add pkg that needs spec to exception msg
* PackageStillNeededError: f-string with short fmt and hash
* PackageStillNeededError: split long string
The old concretizer creates a cyclic graph when expanding virtuals for
`iconv`, which is a bug. This hack drops glibc and musl as possible
providers for `iconv` in the old concretizer to work around it.
* Added package to build Ollama
* Update package.py
Add license and documentation
* [@spackbot] updating style on behalf of teaguesterling
* We can now use OVERRIDE_GIT_DESCRIBE to set the version in DuckDB
* Update duckdb/package.py
- use f-string
- fix version specs to address inconsistencies
* Update package.py
Fix spec definitions further
* [@spackbot] updating style on behalf of teaguesterling
---------
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
The "DOTNET_CLI_TELEMETRY_OPTOUT" environment variable should be defined when using the product, not when installing it (the installation phase is just extract the files anyway).
Also use `~` instead of `-` to check for the variant and fix the second argument for `env.set` which should also be a string.
Add debug log for external detection tests. The debug log
is used to print which test is being executed.
Skip version audit on Windows where appropriate
* Update package.py to osg 1.119 and igtf 1.128
* Remove unnecessary white space
* Add missing comma
* change url to use git and use commit hash from repository instead of ssh256sum of tar.gz
* [@spackbot] updating style on behalf of rahmans1
---------
Co-authored-by: rahmans1 <rahmans1@users.noreply.github.com>
* * Add initial tests-sos package
* Remove failing call of missing setup_compiler_environment from sos
package
* Add several variants for sos package
* [@spackbot] updating style on behalf of jack-morrison
We run `extend spack_flags_list SPACK_LDFLAGS` for `$mode in ld|ccld`.
That's problematic, cause `ccld` needs `-Wl,--flag` whereas `ld` needs
`--flag` directly. Only `-L` and `-l` are common to compiler & linker.
In all build systems `LDFLAGS` is for the compiler not the linker, cause
any linker flag `-x` can be passed as a compiler flag `-Wl,-x`, and there
are many compiler flags that affect the linker invocation, like `-fopenmp`,
`-fuse-ld=`, `-fsanitize=` etc.
So don't pass `LDFLAGS` to the linker directly.
This way users can set `ldflags: -Wl,--allow-shlib-undefined` in compilers.yaml
to work around an issue where the linker tries to resolve the `libcuda.so.1`
stub lib which cannot be located by design in `cuda`.
* Updates for rocm
Updated for rocm@6
Added conflict between rocprofiler and roctracer.
Request either +rocprofiler or +roctracer when +rocm. In this case, it automatically builds for one, instead of displaying the message.
Request +rocm when using either +rocprofiler or +roctracer. In this case, it automatically builds with +rocm, instead of displaying the message.
Disabled the tests. Will update them with the new test method.
* [@spackbot] updating style on behalf of jordialcaraz
---------
Co-authored-by: jordialcaraz <jordialcaraz@users.noreply.github.com>
Some packages can't be redistributed in source or binary form. We need an explicit way to say that in a package.
This adds a `redistribute()` directive so that package authors can write, e.g.:
```python
redistribute(source=False, binary=False)
```
You can also do this conditionally with `when=`, as with other directives, e.g.:
```python
# 12.0 and higher are proprietary
redistribute(source=False, binary=False, when="@12.0:")
# can't redistribute when we depend on some proprietary dependency
redistribute(source=False, binary=False, when="^proprietary-dependency")
```
To prevent Spack from adding either their sources or binaries to public mirrors and build caches. You can still unconditionally add things *if* you run either:
* `spack mirror create --private`
* `spack buildcache push --private`
But the default behavior for build caches is not to include non-redistributable packages in either mirrors or build caches. We have previously done this manually for our public buildcache, but with this we can start maintaining redistributability directly in packages.
Caveats: currently the default for `redistribute()` is `True` for both `source` and `binary`, and you can only set either of them to `False` via this directive.
- [x] add `redistribute()` directive
- [x] add `redistribute_source` and `redistribute_binary` class methods to `PackageBase`
- [x] add `--private` option to `spack mirror`
- [x] add `--private` option to `spack buildcache push`
- [x] test exclusion of packages from source mirror (both as a root and as a dependency)
- [x] test exclusion of packages from binary mirror (both as a root and as a dependency)
If there's no compiler we currently don't have any external libc for the solver.
This commit adds a fallback on libc from the current Python process, which works if it is dynamically linked.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
When looking at where we spend our time in solver setup, I noticed a fair bit of time is spent
in `Spec.format()`, and `Spec.format()` is a pretty old, slow, convoluted method.
This PR does a number of things:
- [x] Consolidate most of what was being done manually with a character loop and several
regexes into a single regex.
- [x] Precompile regexes where we keep them
- [x] Remove the `transform=` argument to `Spec.format()` which was only used in one
place in the code (modules) to uppercase env var names, but added a lot of complexity
- [x] Avoid escaping and colorizing specs unless necessary
- [x] Refactor a lot of the colorization logic to avoid unnecessary object construction
- [x] Add type hints and remove some spots in the code where we were using nonexistent
arguments to `format()`.
- [x] Add trivial cases to `__str__` in `VariantMap` and `VersionList` to avoid sorting
- [x] Avoid calling `isinstance()` in the main loop of `Spec.format()`
- [x] Don't bother constructing a `string` representation for the result of `_prev_version`
as it is only used for comparisons.
In my timings (on all the specs formatted in a solve of `hdf5`), this is over 2.67x faster than the
original `format()`, and it seems to reduce setup time by around a second (for `hdf5`).
The reverse provider lookup may have stale entries for deleted packages, which used to cause errors. It's hard to invalidate those cache entries, so this commit simply drops entries w/o invalidating the cache.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This commit differentiate linux from other platforms by
using libc compatibility as a criterion for deciding
which buildcaches / binaries can be reused. Other
platforms still use OS compatibility.
On linux a libc is injected by all compilers as an implicit
external, and the compatibility criterion is that a libc is
compatible with all other libcs with the same name and a
version that is lesser or equal.
Some concretization unit tests use libc when run on linux.
Some logic to detect what libc the c / cxx compilers use by default,
based on `-dynamic-linker`.
The function `compiler.default_libc()` returns a `Spec` of the form
`glibc@x.y` or `musl@x.y` with the `external_path` property set.
The idea is this can be injected as a dependency.
If we can't run the dynamic linker directly, fall back to `ldd` relative
to the prefix computed from `ld.so.`
In the future we may transform the database from a single JSON object to
a stream of JSON objects.
This paves the way for constant time writes and constant time rereads
when only O(1) changes are made. Currently both are linear time.
This commit gives just enough forward compat for Spack to produce a
friendly error when we would move to a stream of json objects, and a db
would look like this:
```json
{"database": {"version": "<something newer>"}}
```
* compiler wrapper: prioritize spack managed paths in search order
This commit partitions search paths of -L, -I (and -rpath) into three
groups, from highest priority to lowest:
1. Spack managed directories: these include absolute paths such as
stores and the stage dir, as well as all relative paths since they
are relative to a Spack owned dir
2. Non-system dirs: these are for externals that live in non-system
locations
3. System dirs: your typical `/usr/lib` etc.
It's very easy for Spack to known the prefixes it owns, it's much more
difficult to tell system dirs from non-system dirs. Before this commit
Spack tried to distinguish only system and non-system dirs, and failed
for very trivial cases like `/usr/lib/x/..` which comes up often, since
build systems sometimes copy search paths from `gcc -print-search-dirs`.
Potentially this implementation is even faster than the current state of
things, since a loop over paths is replaced with an eval'ed `case ...`.
* Trigger a pipeline
* Revert "Trigger a pipeline"
This reverts commit 5d7fa863de.
* remove redudant return statement
* Add conflicts for some blas implementations that don't build on
Windows (or with %msvc)
* Need to enclose CC/CXX variables in quotes in case those paths
have spaces, otherwise Meson runs into errors
* On Windows, Python dependencies now add <prefix>/Scripts to the
PATH (this is established as a standard in PEP 370)
* Later versions of oneAPI have moved, so update detection to find it
in both old and new location
* Remove reliance on ONEAPI_ROOT env variable when determining Fortran
compiler version for %msvc
* When finding a Fortran compiler for MSVC, there was logic enforcing
a maximum MSVC version for a given oneAPI Fortran version. This
mapping was out of date and excluding valid combinations, so has
been removed (the logic now just picks the latest available
oneAPI Fortran compiler for any given MSVC version).
On Windows, bootstrapping logic now searches for and adds the win-sdk
and wgl packages to the user's top scope as externals if they are not
present.
These packages are generally required to install most packages with
Spack on Windows, and are only available as externals, so it is
assumed that doing this automatically would be useful and avoid
a mandatory manual step for each new Spack instance.
Note this is the first case of bootstrapping logic modifying
configuration other than the bootstrap configuration.
* gxsview: compiles againts system qt and vtk on rhel8
* Update gxsview/package.py for blanks around operator
* Update gxsview/package.py import blank line
* Update gxsview/package.py for style
* Update gxsview/package.py checking vtk version
Score-P does not accept "--with-foo=yes", but only "--with-foo" or "--with-foo=some-valid-specific-choice-or-path". This keeps Spack from generating config flags that will cause Score-P to barf.
This adds some improvements to `spack find` output when in environments based
around some thoughts about what users want to know when they're in an env.
If you're working in an enviroment, you mostly care about:
* What are the roots
* Which ones are installed / not installed
* What's been added that still needs to be concretized
So, this PR adds a couple tweaks to display that information more clearly:
- [x] We now display install status next to every root. You can easily see
which are installed and which aren't.
- [x] When you run `spack find -l` in an env, the roots now show their concrete
hash (if they've been concretized). They previously would show `-------`
(b/c the root spec itself is abstract), but showing the concretized root's
hash is a lot more useful.
- [x] Newly added/unconcretized specs still show `-------`, which now makes more
sense, b/c they are not concretized.
- [x] There is a new option, `-r` / `--only-roots` to *only* show env roots if
you don't want to look at all the installed specs.
- [x] Roots in the installed spec list are now highlighted as bold. This is
actually an old feature from the first env implementation , but various
refactors had disabled it inadvertently.
Reduce incidence of spurious errors by:
* Ensuring we're passing the buffer by reference
* Get the correct short string size from Windows API instead of computing ourselves
* Ensure sufficient space for null terminator character
Add test for `windows_sfn`
Currently if you request pkg +example where example is a conditional
variant, and you have a pkg in the database for which the condition
did not hold (so no +example nor ~example), the solver would reuse it
regardless, not imposing +example.
The change rules out exactly one thing: variant_set without variant_value,
which in practice could only happen when not node_has_variant (i.e. when
under the current package.py rules the variant's when condition did not
trigger).
Add a "default" option that passes no option to configure. Existing
options changed in the MPICH 4.2.0 release, so update the package to
reflect those changes.
Currently, some of the tests in `spec_format` and `spec_semantics` fetch
the actual zlib repository when run, because they call `str()` on specs
like `zlib@foo/bar`, which at least currently requires a remote git clone
to resolve.
This doesn't change the behavior of git versions, but it uses our mock git
repo infrastructure and clones the `git-test` package instead of the *real*
URL from the mock `zlib` package.
This should speed up tests. We could probably refactor more so that the git
tests *all* use such a fixture, but the `checks` field that unfortunately
tightly couples the mock git repository and the `git_fetch` tests complicates
this. We could also consider *not* making `str()` resolve git versions, but
I did not dig into that here.
- [x] add a mock_git_test_package fixture that sets up a mock git repo *and*
monkeypatches the `git-test` package (like our git test packages do)
- [x] use fixture in `test_spec_format_path`
- [x] use fixture in `test_spec_format_path_posix`
- [x] use fixture in `test_spec_format_path_windows`
- [x] use fixture in `test_parse_single_spec`
Commit 330a9a7c9a aimed at preventing
generation of .cfg files when a given compiler does not exist
in the particular release. However the check does not
contain the full paths so it always fails resulting in empty
.cfg files. This commit fixes it.
* add new axom releases, sync changes between repos
* add new version of axom and add fallback depends for umpire/raja
* remove now redundant flags, add flags to cuda flags to flags output by cachedcmakepackage
Co-authored-by: white238 <white238@users.noreply.github.com>
* w3nco %oneapi cflags: append -Wno-error=implicit-function-declaration
* update flag for %apple-clang, %clang
* [@spackbot] updating style on behalf of eugeneswalker
Upon close inspection of clingo answer sets, in some cases we have "equivalent" (i.e. same hash for the concrete spec) duplicates that differ only because of virtual nodes that are added to the answer set, without any edge using them.
This commit adds a property `autopush` to mirrors. When true, every source build is immediately followed by a push to the build cache. This is useful in ephemeral environments such as CI / containers.
To enable autopush on existing build caches, use `spack mirror set --autopush <name>`. The same flag can be used in `spack mirror add`.
* Simplify config command and add BLAS/LAPACK location
* Use BLAS_ROOT and LAPACK_ROOT and disable use of system
package registry
* Adds location of BLAS_LIBRARIES and LAPACK_LIBRARIES to CMake
* Adds CMake variables to prevent picking up system installations
of BLAS/LAPACK. Fixes previous PR #43328 that was picking up
incorrect installations
* Adds versions 7.2.1
* libzip: add up to v1.10.1
- update homepage and change download url to GitHub
- change build system to CMake for releases starting with 1.4
* [@spackbot] updating style on behalf of aumuell
* libzip: fix urls
* [@spackbot] updating style on behalf of aumuell
* libzip: do not add versions from libzip.org
these are old, and urllib refuses to fetch them
* libzip: deprecate versions from libzip.org
urllib refuses to fetch them, only curl would work
---------
Co-authored-by: aumuell <aumuell@users.noreply.github.com>
* acts: new version 33.1.0
* actsvg: new version 0.4.41
* geomodel: new package
* [@spackbot] updating style on behalf of wdconinc
* acts: plugin_cmake_variant(geomodel)
* geomodel: with when(+visualization)
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
* Include recent change for Ubuntu
Select option -disable-no-pie-on-ubuntu for some Ubuntu systems
823971df01
* Added conflict for new variant
* Updated conflict version
* Added mention of Ubuntu to variant description
* Kokkos Kernels: adding missing TPLs and pre-conditions
Adding variants and dependencies for rocBLAS and rocSPARSE.
Also adding a "when=" close to the TPL variants that prevents
enabling the TPLs in versions of the library when it was not
yet available.
* Kokkos Kernels: remove comment for better format
* Kokkos Kernels: adding cusolver and rocsolver for at version 4.3.00
* Kokkos Ecosystem: updating packages for release 4.3.00
* Kokkos: adding arch for SG2042
* Removing sg2042 from spack_micro_arch_map
Removing it here and will work to add it in the proper generic spack location, likely: `spack/lib/spack/external/archspec/json/cpu/microarchitectures.json` ?
Allow reuse of specs that were built with compilers not in the current configuration. This means that specs from build caches don't need to have a matching compiler locally to be reused. Similarly when updating a distro. If a node needs to be built, only available compilers will be considered as candidates.
* add c++11 header to gold for compiler not defaulting to c++11
* glibc: add 2.39
* llvm: add 18.1.3
* fix#42314, remove cxx11 flag for llvm; should be controlled by cmake.
* modify patch
* llvm version
* add gmake version request
* yoda: new versions 1.9.9, 1.9.10, and 2.0.0
* [@spackbot] updating style on behalf of wdconinc
---------
Co-authored-by: wdconinc <wdconinc@users.noreply.github.com>
As in the original makefile "FFLAGS_PROMOTION = -fdefault-real-8
-fdefault-double-8" is only used when `precision=double`. This is the default
for the Spack package, so no change if `precision` is left unset.
* Generally use os.replace on Windows and Linux
* Windows behavior for os.replace differs when the destination exists
and is a symlink to a directory: on Linux the dst is replaced and
on Windows this fails - this PR makes Windows behave like Linux
(by deleting the dst before doing the rename unless src and dst
are the same)
* Nalu: updating Trilinos recipe a bit
Basic changes to build/install nalu properly using Spack.
Some more changes would be nice for instance adding an
option to build against Trilinos master or develop. Adding
a dependency on googletest to avoid the annoying build
failures in the unit-tests.
* Nalu: adding release 1.6.0
Nalu v1.6.0 can build cleanly against Trilinos 14.0.0 with the
proposed changes. The only other combo is master / master but
than one is "floating" as these branch evolve over time. When a
new Nalu comes out we might want to add another fixed version to
keep this recipes up to date!
the autotools build system does something funky which causes a link line
where gccs default link dirs are explicitly added and end up before the
-L from spack's libunwind, so that ultimately it links against system
libunwind.
the cmake build system does better.
* upgrade new versions
* style fix
* update jaxlib deps (not cuda and bazel yet)
* update jaxlib cuda versions
* update jaxlib cuda versions
* update jaxlib cuda versions
* chore: style fix
* Update package.py
* Update package.py
* fix: typo
* docs: add source for cuda version
* py-jaxlib 0.4.14 also doesn't build on ppc64le
* Add 0.4.26
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ParaView: Update version 5.12.0
Add 5.12.0 release
Update default to 5.12.0
* Add patch for building ParaView 5.12 with kits
* Drop VTKm from neoverse
* For avx build, the start address of values_ buffer in KernelParameters
is not a correct as it is computed based on 16-byte alignment.
* Style check error fix
* updating package.py for py-celery, py-kombu, py-amq
* added more py-kombu package versions
* fix copyrights and stype on py-kombu/package.py
* removed extra spaces
* added py-billiard 4.2.0 and added back the license('BSD-3-Clause')
* removed extra spaces in py-celery/package.py
* fixed py-amqp 2.4.0 sha; fixed py-celery's dependency of py-click (when version restrictions)
* more clean up on specifying version bounds
* Relax compiler and target mismatches
The mismatch occurs on an edge. Previously it was assigned
the parent priority, now it is assigned the child priority.
This should make reuse from buildcaches or store more likely,
since most mismatches will be counted with "reused" priority.
* Optimize version badness for runtimes at very low priority
We don't want to e.g. switch other attributes because we
cannot reuse an old installed runtime.
* Optimize runtime attributes at very low priority
This is such that the version of the runtime would
not influence whether we should reuse a spec.
Compiler mismatches are considered for runtimes,
to avoid situations where compiling foo%gcc@9
brings in gcc-runtime%gcc@13 if gcc@13 is among
the available compilers
* Exclude specs without runtimes from reuse
This should ensure that we do not reuse specs that
could be broken, as they expect the compiler to be
installed in a specific place.
The installer runs `get_dependent_ids`, which follows edges outside the
subdag that's being installed, so it returns a superset of the actual
dependents.
That's generally fine, except that it calls `s.package` on every
dependent, which triggers a package class to be instantiated, which is a
lot of work.
Instead, compute the package id from the spec, since that's all that's
used anyways and does not trigger *lots* of slow and redundant
instantiations of package objects.
If ONEAPI_ROOT is not set as an environment variable, the current approach will raise an error.
Instead we can compute the OneAPI_ROOT from the compiler paths like we do with vcvarsall.
The installation mechanism used on Linux to install py-pip (using pip
from the downloaded wheel to install the wheel) does not work on Windows.
This updates the installation of py-pip on Windows to download and
use a zipapp of a specific pip version in order to install the wheel
pip version that is requested.
`dpcpp` is deprecated by intel and has been superseded by `oneapi` compilers for a very long time.
---------
Co-authored-by: becker33 <becker33@users.noreply.github.com>
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* SEACAS, EXODUSII: New version; deprecate older versions; better variant descriptions
* [@spackbot] updating style on behalf of gsjaardema
* Fix long lines reported by flake8
---------
Co-authored-by: gsjaardema <gsjaardema@users.noreply.github.com>
* Add release 2.8.0
* Changing C compiler to hipcc
* Final release version
* Adding new cmake definition for rocm support
* Enabling rocm version support
* Update sha256
* Updating website URL
* Removing unnecessary C compiler spec
* Adding rocm-core dependency
* fixing rocm-core version
* fixing rocm-core version
* fixing style
* bugfix
* Add systemd
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* gobject-introspection: Correct glib versions
- The meson.build requirement that the glib version
is >= the gobject-introspective version is not in place
until v1.76.1.
- Prior to that, the requirement was glib >= 2.58.0.
- Bug introduced in acbf0d99c4, PR #42222.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* util-linux: add v2.39.3
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* py-natsort: add new versions
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: default systemd support to true
- Make the dependency sticky to force a failure
if systemd compilation fails, or force
the user to disable the option.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: Add initial multi-architecture support
- Restrict arch conflicts to 3.0.1
- Disable cpuid at configure time on non-x86_64 platforms.
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* geopm-service: update docstrings
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* Add py-geopmdpy
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
* Add geopm-runtime recipe
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
---------
Signed-off-by: Brad Geltz <brad.geltz@intel.com>
This PR allows the user to specify a path to a custom cert file (or directory) in
Spack's config:
```yaml
# This is where custom certs for proxy/firewall are stored.
# It can be a path or environment variable. To match ssl env configuration
# the default is the environment variable SSL_CERT_FILE
ssl_certs: $SSL_CERT_FILE
```
`config:ssl_certs` can be a path to a file or a directory, or it can be and environment
variable that resolves to one of those. When it posts to something valid, Spack will
update the ssl context to include custom certs, and fetching via `urllib` and `curl`
will trust the provided certs.
This should resolve many issues with fetching behind corporate firewalls.
---------
Co-authored-by: psakievich <psakievich@users.noreply.github.com>
Co-authored-by: Alec Scott <alec@bcs.sh>
After #41373, where we stopped considering the source directory to be the stage for develop builds,
we resumed *deleting* the stage even after a successful build.
We don't want this for develop builds because developers need to iterate; we should keep the artifacts
unless they explicitly run `spack clean`.
Now:
- [x] Build artifacts for develop packages are not removed after a successful install
- [x] They are also not removed before an install starts, i.e. develop packages always
reuse prior artifacts, if available.
- [x] They can be deleted in any other context, e.g. by running `spack clean --stage`
Prior FleCSI releases relied on commits on the control-replication branch (cr)
of Legion. That branch has now been merged into master and is part of the
24.03.0 Legion release.
* (py-)onnx: new version 1.14.{0,1}, 1.15.0
Notes on `onnx`:
- The C++ standard was changed to 14 in 1.15, so no more filter_file is needed. (The C++ standard has since changed to 17 in master.)
Notes on `py-onnx`:
- `py-pybind11` was an unlisted requirement in CMakeLists.txt since 1.3 or so (before earliest spack package).
* py-onnx: depends_on pybind11 with type link, not run
* py-onnx: depends_on py-setuptools@64:
Users requested an option to filter between local/upstream results in `spack find` output.
```
# default behavior, same as without --install-tree argument
$ spack find --install-tree all
# show only local results
$ spack find --install-tree local
# show results from all upstreams
$ spack find --install-tree upstream
# show results from a particular upstream or the local install_tree
$ spack find --install-tree /path/to/install/tree/root
```
---------
Co-authored-by: becker33 <becker33@users.noreply.github.com>
* sundials: add new version
* note previous default
* update when clause for removed options
---------
Co-authored-by: David J. Gardner <gardner48@llnl.gov>
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* exodusii -- refactor and bring up-to-date
* Add missed patch file
* [@spackbot] updating style on behalf of gsjaardema
* Apply seacas windows patch here also.
* Update url so old checksums valid; redo new checksums
---------
Co-authored-by: gsjaardema <gsjaardema@users.noreply.github.com>
* Initial commit to bump rocprofiler-dev to 6.0 and add aqlprofile recipe
* bump to 6.0.2 and extracting binaries from deb pkg
* fixes for hpctoolkit build errors
* add yum and zyp aqlprofile packages
* fix style issues
* Allow compilers to function across compatible OS's
* Add documentation in the default yaml
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Gregory Becker <becker33@llnl.gov>
* py-libsonata: new package starting at 0.1.25
* align spack version with the git branch
* Fix min required versions / use spack packages instead of submodules
* Create devcontainer.json
* Ensure codespace can be setup for current branch
* fix: find compilers in site scope
* fix: use cloud_pipelines ubuntu20.04 image
* fix: spack config --scope site add
* fix: use develop, not develop-root mirror
This adds a dependency on pkg-config which in turn builds pkg-config
on pipelines using %onapi/%cce: update the pkg-config build to disable
specific warnings-as-errors from these compilers.
Co-authored-by: Reid Priedhorsky <1682574+reidpr@users.noreply.github.com>
* Add macos-14 as a runner (Apple M1)
* Mark a test xfail
We need to check later if this test needs modifications
on Apple Silicon chips.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Co-authored-by: alalazo <alalazo@users.noreply.github.com>
* buildcache sync: manifest-glob with arbitrary destination
The current implementation of the --manifest-glob is a bit restrictive
requiring the destination to be known by the generation stage of CI.
This allows specifying an arbitrary destination mirror URL.
* Add unit test for buildcache sync with manifest
* Fix test and arguments for manifest-glob with override destination
* Add testing path for unused mirror argument
* Remove a few compilers from static test data
These compilers were used only in a bunch of tests, so
they are added only there.
* Remove clang@3.3 from unit test configuration
* Parametrize compilers.yaml
* Remove specially named gcc from static data
The compilers are used in two tests
* Remove apple-clang and macOS compilers from static data
The compiler was used only in multimethod tests
* Remove clang@3.5 (compiler seems to be unused)
* Remove gcc@4.4.0 (compiler seems to be unused)
* Exclude x86_64 tests on other architectures
* Mark two tests as for clingo only
* Update version syntax in compilers.yaml
* Parametrize tcl tests on architectures
* Parametrize lmod tests on architectures
* Substitute gcc@4.5.0 with gcc@4.8.0 so it can be used on aarch64
* Fix a few issues with aarch64 and unit-tests
It's now possible to add config on the command line with `spack -c <CONFIG_VARS> ...`, but the new `command_line` scope isn't reflected in the help output for `--scope`:
```bash
> spack help config
...
--scope {defaults,system,site,user}[/PLATFORM] or env:ENVIRONMENT
configuration scope to read/modify
...
```
This PR adds:
- A new runtime for `%oneapi` compilers, called `intel-oneapi-runtime`
- Information to both `gcc-runtime` and `intel-oneapi-runtime`, to ensure
that we don't mix compilers using different soname for either `libgfortran`
or `libifcore`
To do so, the following internal mechanisms have been implemented:
- Possibility to inject virtual dependencies from the `runtime_constraints`
callback on packages
Information has been added to `gcc-runtime` to provide the correct soname
under different conditions on its `%gcc`.
Rules injected into the solver looks like:
```prolog
% Add a dependency on 'gfortran@5' for nodes compiled with gcc@=13.2.0 and using the 'fortran' language
attr("dependency_holds", node(ID, Package), "gfortran", "link") :-
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
attr("virtual_node", node(RuntimeID, "gfortran")) :-
attr("depends_on", node(ID, Package), ProviderNode, "link"),
provider(ProviderNode, node(RuntimeID, "gfortran")),
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
attr("node_version_satisfies", node(RuntimeID, "gfortran"), "5") :-
attr("depends_on", node(ID, Package), ProviderNode, "link"),
provider(ProviderNode, node(RuntimeID, "gfortran")),
attr("node", node(ID, Package)),
attr("node_compiler", node(ID, Package), "gcc"),
attr("node_compiler_version", node(ID, Package), "gcc", "13.2.0"),
not external(node(ID, Package)),
not runtime(Package),
attr("language", node(ID, Package), "fortran").
```
The default url couldn't be the one with v0.0.0-aws since spack was
replacing v0.0.0-aws with v<version_number> for example, deleting the
-aws suffix. I used the url_for_version method to specify this suffix.
This adds support for prereleases. Alpha, beta and release candidate
suffixes are ordered in the intuitive way:
```
1.2.0-alpha < 1.2.0-alpha.1 < 1.2.0-beta.2 < 1.2.0-rc.3 < 1.2.0 < 1.2.0-xyz
```
Alpha, beta and rc prereleases are defined as follows: split the version
string into components like before (on delimiters and string boundaries).
If there's a string component `alpha`, `beta` or `rc` followed by an optional
numeric component at the end, then the version is prerelease.
So `1.2.0-alpha.1 == 1.2.0alpha1 == 1.2.0.alpha1` are all the same, as usual.
The strings `alpha`, `beta` and `rc` are chosen because they match semver,
they are sufficiently long to be unambiguous, and and all contain at least
one non-hex character so distinguish them from shasum/digest type suffixes.
The comparison key is now stored as `(release_tuple, prerelease_tuple)`, so in
the above example:
```
((1,2,0),(ALPHA,)) < ((1,2,0),(ALPHA,1)) < ((1,2,0),(BETA,2)) < ((1,2,0),(RC,3)) < ((1,2,0),(FINAL,)) < ((1,2,0,"xyz"), (FINAL,))
```
The version ranges `@1.2.0:` and `@:1.1` do *not* include prereleases of
`1.2.0`.
So for packaging, if the `1.2.0alpha` and `1.2.0` versions have the same constraints on
dependencies, it's best to write
```python
depends_on("x@1:", when="@1.2.0alpha:")
```
However, `@1.2:` does include `1.2.0alpha`. This is because Spack considers
`1.2 < 1.2.0` as distinct versions, with `1.2 < 1.2.0alpha < 1.2.0` as a consequence.
Alternatively, the above `depends_on` statement can thus be written
```python
depends_on("x@1:", when="@1.2:")
```
which can be useful too. A short-hand to include prereleases, but you
can still be explicit to exclude the prerelease by specifying the patch version
number.
### Concretization
Concretization uses a different version order than `<`. Prereleases are ordered
between final releases and develop versions. That way, users should not
have to set `preferred=True` on every final release if they add just one
prerelease to a package. The concretizer is unlikely to pick a prerelease when
final releases are possible.
### Limitations
1. You can't express a range that includes all alpha release but excludes all beta
releases. Only alternative is good old repeated nines: `@:1.2.0alpha99`.
2. The Python ecosystem defaults to `a`, `b`, `rc` strings, so translation of Python versions to
Spack versions requires expansion to `alpha`, `beta`, `rc`. It's mildly annoying, because
this means we may need to compute URLs differently (not done in this commit).
### Hash
Care is taken not to break hashes of versions that do not have a prerelease
suffix.
Generate CI scripts as powershell on Windows. This is intended to
output exactly the same bash scripts as before on Linux.
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
As the cmake build is triggered by scikit build, the usual spack option
for enabling tests had no effect and the heavy test suite ran all the time.
Used https://github.com/scikit-build/cmake-python-distributions/issues/172#issuecomment-890322263
to implement how to pass options to the actual `cmake` build.
I also excluded some tests that failed for me on alma9 (gcc 11.4.1),
so the rest of the test suite can be run.
Enable OpenBLAS's built-in CPU capability detection and kernel selection.
This allows run-time selection of the "best" kernels for the running CPU, rather
than what is specified at build time. For example, it allows OpenBLAS to use
AVX512 kernels when running on ZEN4, and built targeting the "ZEN" architecture.
Co-authored-by: Branden Moore <branden.moore@amd.com>
* Changes to re-enable aws-pcluster pipelines
- Use compilers from pre-installed spack store such that compiler path relocation works when downloading from buildcache.
- Install gcc from hash so there is no risk of building gcc from source in pipleine.
- `packages.yam` files are now part of the pipelines.
- No more eternal `postinstall.sh`. The necessary steps are in `setup=pcluster.sh` and will be version controlled within this repo.
- Re-enable pipelines.
* Add and
* Debugging output & mv skylake -> skylake_avx512
* Explicilty check for packages
* Handle case with no intel compiler
* compatibility when using setup-pcluster.sh on a pre-installed cluster.
* Disable palace as parser cannot read require clause at the moment
* ifort cannot build superlu in buildcache
`ifort` is unable to handle such long file names as used when cmake compiles
test programs inside build cache.
* Fix spack commit for intel compiler installation
* Need to fetch other commits before using them
* fix style
* Add TODO
* Update packages.yaml to not use 'compiler:', 'target:' or 'provider:'
Synchronize with changes in https://github.com/spack/spack-configs/blob/main/AWS/parallelcluster/
* Use Intel compiler from later version (orig commit no longer found)
* Use envsubst to deal with quoted newlines
This is cleaner than the `eval` command used.
* Need to fetch tags for checkout on version number
* Intel compiler needs to be from version that has compatible DB
* Install intel compiler with commit that has DB ver 7
* Decouple the intel compiler installation from current commit
- Use a completely different spack installation such that this current pipeline
commit remains untouched.
- Make the script suceed even if the compiler installation fails (e.g. because
the Database version has been updated)
- Make the install targets fall back to gcc in case the compiler did not install
correctly.
* Use generic target for x86_64_vX
There is no way to provision a skylake/icelake/zen runner. They are all in the
same pools under x86_64_v3 and x86_64_v4.
* Find the intel compiler in the current spack installation
* Remove SPACK_TARGET_ARCH
* Fix virtual package index & use package.yaml for intel compiler
* Use only one stack & pipeline per generic architecture
* Fix yaml format
* Cleanup typos
* Include fix for ifx.cfg to get the right gcc toolchain when linking
* [removeme] Adding timeout to debug hang in make (palace)
* Revert "[removeme] Adding timeout to debug hang in make (palace)"
This reverts commit fee8a01580489a4ea364368459e9353b46d0d7e2.
* palace x86_64_v4 gets stuck when compiling try newer oneapi
* Update comment
* Use the latest container image
* Update gcc_hashes to match new container
* Use only one tag providing tags per extends call
Also removed an unnecessary tag.
* Move generic setup script out of individual stack
* Cleanup from last commit
* Enable checking signature for packages available on the container
* Remove commented packages / Add comment for palace
* Enable openmpi@5 which needs pmix>3
* don't look for intel compiler on aarch64
* py-python-lsp-server: add v1.10.0
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Remove py-wheel from package
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Running a `spack-python` script like this:
```python
import spack
import multiprocessing
def echo(args):
print(args)
if __name__ == "__main__":
pool = multiprocessing.Pool(2)
pool.map(echo, range(10))
```
will fail in `develop` with an error like this:
```console
_pickle.PicklingError: Can't pickle <function echo at 0x104865820>: attribute lookup echo on __main__ failed
```
Python expects to be able to look up the method `echo` in `sys.path["__main__"]` in
subprocesses spawned by `multiprocessing`, but because we use `InteractiveConsole` to
run `spack python`, the executed file isn't considered to be the `__main__` module, and
lookups in subprocesses fail. We tried to fake this by setting `__name__` to `__main__`
in the `spack python` command, but that doesn't fix the fact that no `__main__` module
exists.
Another annoyance with `InteractiveConsole` is that `__file__` is not defined in the
main script scope, so you can't use it in your scripts.
We can use the [runpy.run_path()](https://docs.python.org/3/library/runpy.html#runpy.run_path) function,
which has been around since Python 3.2, to fix this.
- [x] Use `runpy` module to launch non-interactive `spack python` invocations
- [x] Only use `InteractiveConsole` for interactive `spack python`
Often in containers, the files we use to detect whether a cray system supports new features are not available.
Given that the cray containers only support the newer versions, and that these versions have been
around for a while at this point and few sites don't support them, this PR changes the logic for
detecting cray systems so that:
1. Don't even consider whether something is the `cray` platform if `opt/cray` is not in `MODULEPATH`
2. Only use the `cray` platform if we can read files in /opt/cray/pe and positively detect an older version
3. Otherwise, assume we're *not* on a cray (includes newer Cray PE's, which we treat as Linux)
Compilation with the old flags fails on PowerPC (power8le) due to syntax
errors in the output from the preprocessor. Compilation with the
extended set of flags works both on PowerPC and x86_64.
The correct set of flags was suggested from the berkeleygw developers:
https://groups.google.com/a/berkeleygw.org/g/help/c/ewi3RZgOyeE/m/jSIoe45PAgAJ
Uses spack-provided compiler prefix to call cpp when compiling berkeleygw with gcc.
Adds variants to turn off tests
Add variants for some missing TPL options
Add the variables required to build in ~shared
* Add pamgen to Trilinos as a variant to support SEACAS
This adds the ability to turn off and on pamgen as needed
through the variant interface for the Trilinos package.py.
Add changes for seacas package.py to build the appropriate
Trilinos variants.
Add zlib-api as depends_on instead of zlib directly for SEACAS
package.py
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* Add variant typescript for py-jupyter-server@:1, which then requires npm/node. Patch the build system for ~typescript so that it doesn't find any npm/node installations and attempts to build the typescript extension even though it shouldn't
* Fix formatting in var/spack/repos/builtin/packages/py-jupyter-server/package.py
* Constrain typescript variant to py-jupyter-server versions 1.10.2:1
* with when not needed if variant doesn't exist for other versions
This should fix issue #43192
Basically, had issue where a make variable was set to the output
of a shell function which included cd commands, and then the value
of that variable used as a makefile target.
The cd commands in the shell function caused assorted informational
messages (e.g. "Entering directory ...") which got included in the
return of the shell function, corrupting the value of the variable.
The presence of colons in the corrupted value caused make to issue
"multiple target" erros.
This fix adds --no-print-directory flags to the calls to the
make function in the package's build method, which resolves the
issue above.
It is admittedly a crude fix, and will remove *all* informational
messages re directory changes, thereby potentially making it more
difficult to diagnose/debug future issues building this package.
However, I do not see a way for to turn off these messages in a
more surgical manner.
* mgard: don't restrict protobuf version more than necessary
successfully built:
mgard@2022-11-18 ^protobuf@3.{4,21,25}
mgard@2023-01-10 ^protobuf@3.{4,25}
mgard@2023-03-31 ^protobuf@3.{4,25}
compile failures:
mgard@2022-11-18 ^protobuf@3.3
mgard@2023-01-10 ^protobuf@3.3
mgard@2023-03-31 ^protobuf@3.3
* mgard: add conflicts to address CI errors
* mgard: conflict between cuda and abseil@20240116.1
compiling mgard+cuda with gcc@12.3.0 and nvcc from cuda@12.3.0 against
protobuf pulling in abseil-cpp@20240116.1 results in the errors reported
here: https://github.com/abseil/abseil-cpp/issues/1629
* py-shacl: new version, update dependencies
Also updates the dependencies py-prettytable and py-rdflib.
* review comments
* Update var/spack/repos/builtin/packages/py-pyshacl/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-poetry-core: add required 1.8.1
---------
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
`jinja2` can be a costly import, and right now it happens at startup every time we run
Spack. This slows down `spack --print-shell-vars` a bit, which is needed by `setup-env.*sh`.
@@ -865,7 +865,7 @@ There are several different ways to use Spack packages once you have
installed them. As you've seen, spack packages are installed into long
paths with hashes, and you need a way to get them into your path. The
easiest way is to use :ref:`spack load <cmd-spack-load>`, which is
described in the next section.
described in this section.
Some more advanced ways to use Spack packages include:
@@ -959,7 +959,86 @@ use ``spack find --loaded``.
You can also use ``spack load --list`` to get the same output, but it
does not have the full set of query options that ``spack find`` offers.
We'll learn more about Spack's spec syntax in the next section.
We'll learn more about Spack's spec syntax in :ref:`a later section <sec-specs>`.
.._extensions:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python packages and virtual environments
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Spack can install a large number of Python packages. Their names are
typically prefixed with ``py-``. Installing and using them is no
different from any other package:
..code-block::console
$ spack install py-numpy
$ spack load py-numpy
$ python3
>>> import numpy
The ``spack load`` command sets the ``PATH`` variable so that the right Python
executable is used, and makes sure that ``numpy`` and its dependencies can be
located in the ``PYTHONPATH``.
Spack is different from other Python package managers in that it installs
every package into its *own* prefix. This is in contrast to ``pip``, which
installs all packages into the same prefix, be it in a virtual environment
or not.
For many users, **virtual environments** are more convenient than repeated
``spack load`` commands, particularly when working with multiple Python
packages. Fortunately Spack supports environments itself, which together
with a view are no different from Python virtual environments.
The recommended way of working with Python extensions such as ``py-numpy``
is through :ref:`Environments <environments>`. The following example creates
a Spack environment with ``numpy`` in the current working directory. It also
puts a filesystem view in ``./view``, which is a more traditional combined
prefix for all packages in the environment.
..code-block::console
$ spack env create --with-view view --dir .
$ spack -e . add py-numpy
$ spack -e . concretize
$ spack -e . install
Now you can activate the environment and start using the packages:
..code-block::console
$ spack env activate .
$ python3
>>> import numpy
The environment view is also a virtual environment, which is useful if you are
sharing the environment with others who are unfamiliar with Spack. They can
either use the Python executable directly:
..code-block::console
$ ./view/bin/python3
>>> import numpy
or use the activation script:
..code-block::console
$ source ./view/bin/activate
$ python3
>>> import numpy
In general, there should not be much difference between ``spack env activate``
and using the virtual environment. The main advantage of ``spack env activate``
is that it knows about more packages than just Python packages, and it may set
additional runtime variables that are not covered by the virtual environment
activation script.
See :ref:`environments` for a more in-depth description of Spack
environments and customizations to views.
.._sec-specs:
@@ -1096,6 +1175,17 @@ unspecified version, but packages can depend on other packages with
could depend on ``mpich@1.2:`` if it can only build with version
``1.2`` or higher of ``mpich``.
..note:: Windows Spec Syntax Caveats
Windows has a few idiosyncrasies when it comes to the Spack spec syntax and the use of certain shells
Spack's spec dependency syntax uses the carat (``^``) character, however this is an escape string in CMD
so it must be escaped with an additional carat (i.e. ``^^``).
CMD also will attempt to interpret strings with ``=`` characters in them. Any spec including this symbol
must double quote the string.
Note: All of these issues are unique to CMD, they can be avoided by using Powershell.
For more context on these caveats see the related issues: `carat <https://github.com/spack/spack/issues/42833>`_ and `equals <https://github.com/spack/spack/issues/43348>`_
Below are more details about the specifiers that you can add to specs.
.._version-specifier:
@@ -1119,6 +1209,9 @@ and ``3.4.2``. Similarly, ``@4.2:`` means any version above and including
``4.2``. As a short-hand, ``@3`` is equivalent to the range ``@3:3`` and
includes any version with major version ``3``.
Versions are ordered lexicograpically by its components. For more details
on the order, see :ref:`the packaging guide <version-comparison>`.
Notice that you can distinguish between the specific version ``@=3.2`` and
the range ``@3.2``. This is useful for packages that follow a versioning
scheme that omits the zero patch version number: ``3.2``, ``3.2.1``,
@@ -1351,22 +1444,12 @@ the reserved keywords ``platform``, ``os`` and ``target``:
$ spack install libelf os=ubuntu18.04
$ spack install libelf target=broadwell
or together by using the reserved keyword ``arch``:
..code-block::console
$ spack install libelf arch=cray-CNL10-haswell
Normally users don't have to bother specifying the architecture if they
are installing software for their current host, as in that case the
values will be detected automatically. If you need fine-grained control
over which packages use which targets (or over *all* packages' default
target), see :ref:`package-preferences`.
..admonition:: Cray machines
The situation is a little bit different for Cray machines and a detailed
explanation on how the architecture can be set on them can be found at :ref:`cray-support`
.._support-for-microarchitectures:
@@ -1702,165 +1785,6 @@ check only local packages (as opposed to those used transparently from
``upstream`` spack instances) and the ``-j,--json`` option to output
machine-readable json data for any errors.
.._extensions:
---------------------------
Extensions & Python support
---------------------------
Spack's installation model assumes that each package will live in its
own install prefix. However, certain packages are typically installed
*within* the directory hierarchy of other packages. For example,
`Python <https://www.python.org>`_ packages are typically installed in the
@@ -220,6 +220,40 @@ section of the configuration:
.._binary_caches_oci:
---------------------------------
Automatic push to a build cache
---------------------------------
Sometimes it is convenient to push packages to a build cache as soon as they are installed. Spack can do this by setting autopush flag when adding a mirror:
..code-block::console
$ spack mirror add --autopush <name> <url or path>
Or the autopush flag can be set for an existing mirror:
..code-block::console
$ spack mirror set --autopush <name> # enable automatic push for an existing mirror
$ spack mirror set --no-autopush <name> # disable automatic push for an existing mirror
Then after installing a package it is automatically pushed to all mirrors with ``autopush: true``. The command
..code-block::console
$ spack install <package>
will have the same effect as
..code-block::console
$ spack install <package>
$ spack buildcache push <cache> <package> # for all caches with autopush: true
..note::
Packages are automatically pushed to a build cache only if they are built from source.
The first step to contribute new runners is to open an issue in the `spack infrastructure <https://github.com/spack/spack-infrastructure/issues/new?assignees=&labels=runner-registration&projects=&template=runner_registration.yml>`_
project. This will be reported to the spack infrastructure team who will guide users through the process
of registering new runners for Spack CI.
The information needed to register a runner is the motivation for the new resources, a semi-detailed description of
the runner, and finallly the point of contact for maintaining the software on the runner.
The point of contact will then work with the infrastruture team to obtain runner registration token(s) for interacting with
with Spack's GitLab instance. Once the runner is active, this point of contact will also be responsible for updating the
GitLab runner software to keep pace with Spack's Gitlab.
Tagging
~~~~~~~
In the initial stages of runner registration it is important to **exclude** the special tag ``spack``. This will prevent
the new runner(s) from being picked up for production CI jobs while it is configured and evaluated. Once it is determined
that the runner is ready for production use the ``spack`` tag will be added.
Because gitlab has no concept of tag exclustion, runners that provide specialized resource also require specialized tags.
For example, a basic CPU only x86_64 runner may have a tag ``x86_64`` associated with it. However, a runner containing an
CUDA capable GPU may have the tag ``x86_64-cuda`` to denote that it should only be used for packages that will benefit from
a CUDA capable resource.
OIDC
~~~~
Spack runners use OIDC authentication for connecting to the appropriate AWS bucket
which is used for coordinating the communication of binaries between build jobs. In
order to configure OIDC authentication, Spack CI runners use a python script with minimal
dependencies. This script can be configured for runners as seen here using the ``pre_build_script``.
An environment is used to group together a set of specs for the
purpose of building, rebuilding and deploying in a coherent fashion.
Environments provide a number of advantages over the *à la carte*
approach of building and loading individual Spack modules:
An environment is used to group a set of specs intended for some purpose
to be built, rebuilt, and deployed in a coherent fashion. Environments
define aspects of the installation of the software, such as:
#.Environments separate the steps of (a) choosing what to
install, (b) concretizing, and (c) installing. This allows
Environments to remain stable and repeatable, even if Spack packages
are upgraded: specs are only re-concretized when the user
explicitly asks for it. It is even possible to reliably
transport environments between different computers running
different versions of Spack!
#. Environments allow several specs to be built at once; a more robust
solution than ad-hoc scripts making multiple calls to ``spack
install``.
#. An Environment that is built as a whole can be loaded as a whole
into the user environment. An Environment can be built to maintain
a filesystem view of its packages, and the environment can load
that view into the user environment at activation time. Spack can
also generate a script to load all modules related to an
environment.
#.*which* specs to install;
#.*how* those specs are configured; and
#.*where* the concretized software will be installed.
Aggregating this information into an environment for processing has advantages
over the *à la carte* approach of building and loading individual Spack modules.
With environments, you concretize, install, or load (activate) all of the
specs with a single command. Concretization fully configures the specs
and dependencies of the environment in preparation for installing the
software. This is a more robust solution than ad-hoc installation scripts.
And you can share an environment or even re-use it on a different computer.
Environment definitions, especially *how* specs are configured, allow the
software to remain stable and repeatable even when Spack packages are upgraded. Changes are only picked up when the environment is explicitly re-concretized.
Defining *where* specs are installed supports a filesystem view of the
environment. Yet Spack maintains a single installation of the software that
can be re-used across multiple environments.
Activating an environment determines *when* all of the associated (and
installed) specs are loaded so limits the software loaded to those specs
actually needed by the environment. Spack can even generate a script to
load all modules related to an environment.
Other packaging systems also provide environments that are similar in
some ways to Spack environments; for example, `Conda environments
<https://conda.io/docs/user-guide/tasks/manage-environments.html>`_ or
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
