Compare commits
7 Commits
develop
...
hs/test/tr
Author | SHA1 | Date | |
---|---|---|---|
![]() |
05fabbbab2 | ||
![]() |
c6d4037758 | ||
![]() |
08f1cf9ae2 | ||
![]() |
48dfa3c95e | ||
![]() |
e5c411d8f0 | ||
![]() |
020e30f3e6 | ||
![]() |
181c404af5 |
362
CHANGELOG.md
362
CHANGELOG.md
@ -1,3 +1,365 @@
|
|||||||
|
# v0.23.0 (2024-11-13)
|
||||||
|
|
||||||
|
`v0.23.0` is a major feature release.
|
||||||
|
|
||||||
|
We are planning to make this the last major release before Spack `v1.0`
|
||||||
|
in June 2025. Alongside `v0.23`, we will be making pre-releases (alpha,
|
||||||
|
beta, etc.) of `v1.0`, and we encourage users to try them and send us
|
||||||
|
feedback, either on GitHub or on Slack. You can track the road to
|
||||||
|
`v1.0` here:
|
||||||
|
|
||||||
|
* https://github.com/spack/spack/releases
|
||||||
|
* https://github.com/spack/spack/discussions/30634
|
||||||
|
|
||||||
|
## Features in this Release
|
||||||
|
|
||||||
|
1. **Language virtuals**
|
||||||
|
|
||||||
|
Your packages can now explicitly depend on the languages they require.
|
||||||
|
Historically, Spack has considered C, C++, and Fortran compiler
|
||||||
|
dependencies to be implicit. In `v0.23`, you should ensure that
|
||||||
|
new packages add relevant C, C++, and Fortran dependencies like this:
|
||||||
|
|
||||||
|
```python
|
||||||
|
depends_on("c", type="build")
|
||||||
|
depends_on("cxx", type="build")
|
||||||
|
depends_on("fortran", type="build")
|
||||||
|
```
|
||||||
|
|
||||||
|
We encourage you to add these annotations to your packages now, to prepare
|
||||||
|
for Spack `v1.0.0`. In `v1.0.0`, these annotations will be necessary for
|
||||||
|
your package to use C, C++, and Fortran compilers. Note that you should
|
||||||
|
*not* add language dependencies to packages that don't need them, e.g.,
|
||||||
|
pure python packages.
|
||||||
|
|
||||||
|
We have already auto-generated these dependencies for packages in the
|
||||||
|
`builtin` repository (see #45217), based on the types of source files
|
||||||
|
present in each package's source code. We *may* have added too many or too
|
||||||
|
few language dependencies, so please submit pull requests to correct
|
||||||
|
packages if you find that the language dependencies are incorrect.
|
||||||
|
|
||||||
|
Note that we have also backported support for these dependencies to
|
||||||
|
`v0.21.3` and `v0.22.2`, to make all of them forward-compatible with
|
||||||
|
`v0.23`. This should allow you to move easily between older and newer Spack
|
||||||
|
releases without breaking your packages.
|
||||||
|
|
||||||
|
2. **Spec splicing**
|
||||||
|
|
||||||
|
We are working to make binary installation more seamless in Spack. `v0.23`
|
||||||
|
introduces "splicing", which allows users to deploy binaries using local,
|
||||||
|
optimized versions of a binary interface, even if they were not built with
|
||||||
|
that interface. For example, this would allow you to build binaries in the
|
||||||
|
cloud using `mpich` and install them on a system using a local, optimized
|
||||||
|
version of `mvapich2` *without rebuilding*. Spack preserves full provenance
|
||||||
|
for the installed packages and knows that they were built one way but
|
||||||
|
deployed another.
|
||||||
|
|
||||||
|
Our intent is to leverage this across many key HPC binary packages,
|
||||||
|
e.g. MPI, CUDA, ROCm, and libfabric.
|
||||||
|
|
||||||
|
Fundamentally, splicing allows Spack to redeploy an existing spec with
|
||||||
|
different dependencies than how it was built. There are two interfaces to
|
||||||
|
splicing.
|
||||||
|
|
||||||
|
a. Explicit Splicing
|
||||||
|
|
||||||
|
#39136 introduced the explicit splicing interface. In the
|
||||||
|
concretizer config, you can specify a target spec and a replacement
|
||||||
|
by hash.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
concretizer:
|
||||||
|
splice:
|
||||||
|
explicit:
|
||||||
|
- target: mpi
|
||||||
|
replacement: mpich/abcdef
|
||||||
|
```
|
||||||
|
|
||||||
|
Here, every installation that would normally use the target spec will
|
||||||
|
instead use its replacement. Above, any spec using *any* `mpi` will be
|
||||||
|
spliced to depend on the specific `mpich` installation requested. This
|
||||||
|
*can* go wrong if you try to replace something built with, e.g.,
|
||||||
|
`openmpi` with `mpich`, and it is on the user to ensure ABI
|
||||||
|
compatibility between target and replacement specs. This currently
|
||||||
|
requires some expertise to use, but it will allow users to reuse the
|
||||||
|
binaries they create across more machines and environments.
|
||||||
|
|
||||||
|
b. Automatic Splicing (experimental)
|
||||||
|
|
||||||
|
#46729 introduced automatic splicing. In the concretizer config, enable
|
||||||
|
automatic splicing:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
concretizer:
|
||||||
|
splice:
|
||||||
|
automatic: true
|
||||||
|
```
|
||||||
|
|
||||||
|
or run:
|
||||||
|
|
||||||
|
```console
|
||||||
|
spack config add concretizer:splice:automatic:true
|
||||||
|
```
|
||||||
|
|
||||||
|
The concretizer will select splices for ABI compatibility to maximize
|
||||||
|
package reuse. Packages can denote ABI compatibility using the
|
||||||
|
`can_splice` directive. No packages in Spack yet use this directive, so
|
||||||
|
if you want to use this feature you will need to add `can_splice`
|
||||||
|
annotations to your packages. We are working on ways to add more ABI
|
||||||
|
compatibility information to the Spack package repository, and this
|
||||||
|
directive may change in the future.
|
||||||
|
|
||||||
|
See the documentation for more details:
|
||||||
|
* https://spack.readthedocs.io/en/latest/build_settings.html#splicing
|
||||||
|
* https://spack.readthedocs.io/en/latest/packaging_guide.html#specifying-abi-compatibility
|
||||||
|
|
||||||
|
3. Broader variant propagation
|
||||||
|
|
||||||
|
Since #42931, you can specify propagated variants like `hdf5
|
||||||
|
build_type==RelWithDebInfo` or `trilinos ++openmp` to propagate a variant
|
||||||
|
to all dependencies for which it is relevant. This is valid *even* if the
|
||||||
|
variant does not exist on the package or its dependencies.
|
||||||
|
|
||||||
|
See https://spack.readthedocs.io/en/latest/basic_usage.html#variants.
|
||||||
|
|
||||||
|
4. Query specs by namespace
|
||||||
|
|
||||||
|
#45416 allows a package's namespace (indicating the repository it came from)
|
||||||
|
to be treated like a variant. You can request packages from particular repos
|
||||||
|
like this:
|
||||||
|
|
||||||
|
```console
|
||||||
|
spack find zlib namespace=builtin
|
||||||
|
spack find zlib namespace=myrepo
|
||||||
|
```
|
||||||
|
|
||||||
|
Previously, the spec syntax only allowed namespaces to be prefixes of spec
|
||||||
|
names, e.g. `builtin.zlib`. The previous syntax still works.
|
||||||
|
|
||||||
|
5. `spack spec` respects environment settings and `unify:true`
|
||||||
|
|
||||||
|
`spack spec` did not previously respect environment lockfiles or
|
||||||
|
unification settings, which made it difficult to see exactly how a spec
|
||||||
|
would concretize within an environment. Now it does, so the output you get
|
||||||
|
with `spack spec` will be *the same* as what your environment will
|
||||||
|
concretize to when you run `spack concretize`. Similarly, if you provide
|
||||||
|
multiple specs on the command line with `spack spec`, it will concretize
|
||||||
|
them together if `unify:true` is set.
|
||||||
|
|
||||||
|
See #47556 and #44843.
|
||||||
|
|
||||||
|
6. Less noisy `spack spec` output
|
||||||
|
|
||||||
|
`spack spec` previously showed output like this:
|
||||||
|
|
||||||
|
```console
|
||||||
|
> spack spec /v5fn6xo
|
||||||
|
Input spec
|
||||||
|
--------------------------------
|
||||||
|
- /v5fn6xo
|
||||||
|
|
||||||
|
Concretized
|
||||||
|
--------------------------------
|
||||||
|
[+] openssl@3.3.1%apple-clang@16.0.0~docs+shared arch=darwin-sequoia-m1
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
But the input spec is redundant, and we know we run `spack spec` to concretize
|
||||||
|
the input spec. `spack spec` now *only* shows the concretized spec. See #47574.
|
||||||
|
|
||||||
|
7. Better output for `spack find -c`
|
||||||
|
|
||||||
|
In an environmnet, `spack find -c` lets you search the concretized, but not
|
||||||
|
yet installed, specs, just as you would the installed ones. As with `spack
|
||||||
|
spec`, this should make it easier for you to see what *will* be built
|
||||||
|
before building and installing it. See #44713.
|
||||||
|
|
||||||
|
8. `spack -C <env>`: use an environment's configuration without activation
|
||||||
|
|
||||||
|
Spack environments allow you to associate:
|
||||||
|
1. a set of (possibly concretized) specs, and
|
||||||
|
2. configuration
|
||||||
|
|
||||||
|
When you activate an environment, you're using both of these. Previously, we
|
||||||
|
supported:
|
||||||
|
* `spack -e <env>` to run spack in the context of a specific environment, and
|
||||||
|
* `spack -C <directory>` to run spack using a directory with configuration files.
|
||||||
|
|
||||||
|
You can now also pass an environment to `spack -C` to use *only* the environment's
|
||||||
|
configuration, but not the specs or lockfile. See #45046.
|
||||||
|
|
||||||
|
## New commands, options, and directives
|
||||||
|
|
||||||
|
* The new `spack env track` command (#41897) takes a non-managed Spack
|
||||||
|
environment and adds a symlink to Spack's `$environments_root` directory, so
|
||||||
|
that it will be included for reference counting for commands like `spack
|
||||||
|
uninstall` and `spack gc`. If you use free-standing directory environments,
|
||||||
|
this is useful for preventing Spack from removing things required by your
|
||||||
|
environments. You can undo this tracking with the `spack env untrack`
|
||||||
|
command.
|
||||||
|
|
||||||
|
* Add `-t` short option for `spack --backtrace` (#47227)
|
||||||
|
|
||||||
|
`spack -d / --debug` enables backtraces on error, but it can be very
|
||||||
|
verbose, and sometimes you just want the backtrace. `spack -t / --backtrace`
|
||||||
|
provides that option.
|
||||||
|
|
||||||
|
* `gc`: restrict to specific specs (#46790)
|
||||||
|
|
||||||
|
If you only want to garbage-collect specific packages, you can now provide
|
||||||
|
them on the command line. This gives users finer-grained control over what
|
||||||
|
is uninstalled.
|
||||||
|
|
||||||
|
* oci buildcaches now support `--only=package`. You can now push *just* a
|
||||||
|
package and not its dependencies to an OCI registry. This allows dependents
|
||||||
|
of non-redistributable specs to be stored in OCI registries without an
|
||||||
|
error. See #45775.
|
||||||
|
|
||||||
|
## Notable refactors
|
||||||
|
* Variants are now fully conditional
|
||||||
|
|
||||||
|
The `variants` dictionary on packages was previously keyed by variant name,
|
||||||
|
and allowed only one definition of any given variant. Spack is now smart
|
||||||
|
enough to understand that variants may have different values and defaults
|
||||||
|
for different versions. For example, `warpx` prior to `23.06` only supported
|
||||||
|
builds for one dimensionality, and newer `warpx` versions could be built
|
||||||
|
with support for many different dimensions:
|
||||||
|
|
||||||
|
```python
|
||||||
|
variant(
|
||||||
|
"dims",
|
||||||
|
default="3",
|
||||||
|
values=("1", "2", "3", "rz"),
|
||||||
|
multi=False,
|
||||||
|
description="Number of spatial dimensions",
|
||||||
|
when="@:23.05",
|
||||||
|
)
|
||||||
|
variant(
|
||||||
|
"dims",
|
||||||
|
default="1,2,rz,3",
|
||||||
|
values=("1", "2", "3", "rz"),
|
||||||
|
multi=True,
|
||||||
|
description="Number of spatial dimensions",
|
||||||
|
when="@23.06:",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Previously, the default for the old version of `warpx` was not respected and
|
||||||
|
had to be specified manually. Now, Spack will select the right variant
|
||||||
|
definition for each version at concretization time. This allows variants to
|
||||||
|
evolve more smoothly over time. See #44425 for details.
|
||||||
|
|
||||||
|
## Highlighted bugfixes
|
||||||
|
|
||||||
|
1. Externals no longer override the preferred provider (#45025).
|
||||||
|
|
||||||
|
External definitions could interfere with package preferences. Now, if
|
||||||
|
`openmpi` is the preferred `mpi`, and an external `mpich` is defined, a new
|
||||||
|
`openmpi` *will* be built if building it is possible. Previously we would
|
||||||
|
prefer `mpich` despite the preference.
|
||||||
|
|
||||||
|
2. Composable `cflags` (#41049).
|
||||||
|
|
||||||
|
This release fixes a longstanding bug that concretization would fail if
|
||||||
|
there were different `cflags` specified in `packages.yaml`,
|
||||||
|
`compilers.yaml`, or on `the` CLI. Flags and their ordering are now tracked
|
||||||
|
in the concretizer and flags from multiple sources will be merged.
|
||||||
|
|
||||||
|
3. Fix concretizer Unification for included environments (#45139).
|
||||||
|
|
||||||
|
## Deprecations, removals, and syntax changes
|
||||||
|
|
||||||
|
1. The old concretizer has been removed from Spack, along with the
|
||||||
|
`config:concretizer` config option. Spack will emit a warning if the option
|
||||||
|
is present in user configuration, since it now has no effect. Spack now
|
||||||
|
uses a simpler bootstrapping mechanism, where a JSON prototype is tweaked
|
||||||
|
slightly to get an initial concrete spec to download. See #45215.
|
||||||
|
|
||||||
|
2. Best-effort expansion of spec matrices has been removed. This feature did
|
||||||
|
not work with the "new" ASP-based concretizer, and did not work with
|
||||||
|
`unify: True` or `unify: when_possible`. Use the
|
||||||
|
[exclude key](https://spack.readthedocs.io/en/latest/environments.html#spec-matrices)
|
||||||
|
for the environment to exclude invalid components, or use multiple spec
|
||||||
|
matrices to combine the list of specs for which the constraint is valid and
|
||||||
|
the list of specs for which it is not. See #40792.
|
||||||
|
|
||||||
|
3. The old Cray `platform` (based on Cray PE modules) has been removed, and
|
||||||
|
`platform=cray` is no longer supported. Since `v0.19`, Spack has handled
|
||||||
|
Cray machines like Linux clusters with extra packages, and we have
|
||||||
|
encouraged using this option to support Cray. The new approach allows us to
|
||||||
|
correctly handle Cray machines with non-SLES operating systems, and it is
|
||||||
|
much more reliable than making assumptions about Cray modules. See the
|
||||||
|
`v0.19` release notes and #43796 for more details.
|
||||||
|
|
||||||
|
4. The `config:install_missing_compilers` config option has been deprecated,
|
||||||
|
and it is a no-op when set in `v0.23`. Our new compiler dependency model
|
||||||
|
will replace it with a much more reliable and robust mechanism in `v1.0`.
|
||||||
|
See #46237.
|
||||||
|
|
||||||
|
5. Config options that deprecated in `v0.21` have been removed in `v0.23`. You
|
||||||
|
can now only specify preferences for `compilers`, `targets`, and
|
||||||
|
`providers` globally via the `packages:all:` section. Similarly, you can
|
||||||
|
only specify `versions:` locally for a specific package. See #44061 and
|
||||||
|
#31261 for details.
|
||||||
|
|
||||||
|
6. Spack's old test interface has been removed (#45752), having been
|
||||||
|
deprecated in `v0.22.0` (#34236). All `builtin` packages have been updated
|
||||||
|
to use the new interface. See the [stand-alone test documentation](
|
||||||
|
https://spack.readthedocs.io/en/latest/packaging_guide.html#stand-alone-tests)
|
||||||
|
|
||||||
|
7. The `spack versions --safe-only` option, deprecated since `v0.21.0`, has
|
||||||
|
been removed. See #45765.
|
||||||
|
|
||||||
|
* The `--dependencies` and `--optimize` arguments to `spack ci` have been
|
||||||
|
deprecated. See #45005.
|
||||||
|
|
||||||
|
## Binary caches
|
||||||
|
1. Public binary caches now include an ML stack for Linux/aarch64 (#39666)We
|
||||||
|
now build an ML stack for Linux/aarch64 for all pull requests and on
|
||||||
|
develop. The ML stack includes both CPU-only and CUDA builds for Horovod,
|
||||||
|
Hugging Face, JAX, Keras, PyTorch,scikit-learn, TensorBoard, and
|
||||||
|
TensorFlow, and related packages. The CPU-only stack also includes XGBoost.
|
||||||
|
See https://cache.spack.io/tag/develop/?stack=ml-linux-aarch64-cuda.
|
||||||
|
|
||||||
|
2. There is also now an stack of developer tools for macOS (#46910), which is
|
||||||
|
analogous to the Linux devtools stack. You can use this to avoid building
|
||||||
|
many common build dependencies. See
|
||||||
|
https://cache.spack.io/tag/develop/?stack=developer-tools-darwin.
|
||||||
|
|
||||||
|
## Architecture support
|
||||||
|
* archspec has been updated to `v0.2.5`, with support for `zen5`
|
||||||
|
* Spack's CUDA package now supports the Grace Hopper `9.0a` compute capability (#45540)
|
||||||
|
|
||||||
|
## Windows
|
||||||
|
* Windows bootstrapping: `file` and `gpg` (#41810)
|
||||||
|
* `scripts` directory added to PATH on Windows for python extensions (#45427)
|
||||||
|
* Fix `spack load --list` and `spack unload` on Windows (#35720)
|
||||||
|
|
||||||
|
## Other notable changes
|
||||||
|
* Bugfix: `spack find -x` in environments (#46798)
|
||||||
|
* Spec splices are now robust to duplicate nodes with the same name in a spec (#46382)
|
||||||
|
* Cache per-compiler libc calculations for performance (#47213)
|
||||||
|
* Fixed a bug in external detection for openmpi (#47541)
|
||||||
|
* Mirror configuration allows username/password as environment variables (#46549)
|
||||||
|
* Default library search caps maximum depth (#41945)
|
||||||
|
* Unify interface for `spack spec` and `spack solve` commands (#47182)
|
||||||
|
* Spack no longer RPATHs directories in the default library search path (#44686)
|
||||||
|
* Improved performance of Spack database (#46554)
|
||||||
|
* Enable package reuse for packages with versions from git refs (#43859)
|
||||||
|
* Improved handling for `uuid` virtual on macos (#43002)
|
||||||
|
* Improved tracking of task queueing/requeueing in the installer (#46293)
|
||||||
|
|
||||||
|
## Spack community stats
|
||||||
|
|
||||||
|
* Over 2,000 pull requests updated package recipes
|
||||||
|
* 8,307 total packages, 329 new since `v0.22.0`
|
||||||
|
* 140 new Python packages
|
||||||
|
* 14 new R packages
|
||||||
|
* 373 people contributed to this release
|
||||||
|
* 357 committers to packages
|
||||||
|
* 60 committers to core
|
||||||
|
|
||||||
|
|
||||||
# v0.22.2 (2024-09-21)
|
# v0.22.2 (2024-09-21)
|
||||||
|
|
||||||
## Bugfixes
|
## Bugfixes
|
||||||
|
@ -11,7 +11,7 @@
|
|||||||
import spack.util.git
|
import spack.util.git
|
||||||
|
|
||||||
#: PEP440 canonical <major>.<minor>.<micro>.<devN> string
|
#: PEP440 canonical <major>.<minor>.<micro>.<devN> string
|
||||||
__version__ = "0.23.0.dev0"
|
__version__ = "0.23.0"
|
||||||
spack_version = __version__
|
spack_version = __version__
|
||||||
|
|
||||||
|
|
||||||
|
@ -25,6 +25,7 @@
|
|||||||
import spack.extensions
|
import spack.extensions
|
||||||
import spack.parser
|
import spack.parser
|
||||||
import spack.paths
|
import spack.paths
|
||||||
|
import spack.repo
|
||||||
import spack.spec
|
import spack.spec
|
||||||
import spack.store
|
import spack.store
|
||||||
import spack.traverse as traverse
|
import spack.traverse as traverse
|
||||||
|
@ -3,7 +3,6 @@
|
|||||||
#
|
#
|
||||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||||
|
|
||||||
import datetime
|
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
@ -97,7 +96,7 @@ def list_files(args):
|
|||||||
OLD_LICENSE, SPDX_MISMATCH, GENERAL_MISMATCH = range(1, 4)
|
OLD_LICENSE, SPDX_MISMATCH, GENERAL_MISMATCH = range(1, 4)
|
||||||
|
|
||||||
#: Latest year that copyright applies. UPDATE THIS when bumping copyright.
|
#: Latest year that copyright applies. UPDATE THIS when bumping copyright.
|
||||||
latest_year = datetime.date.today().year
|
latest_year = 2024 # year of 0.23 release
|
||||||
strict_date = r"Copyright 2013-%s" % latest_year
|
strict_date = r"Copyright 2013-%s" % latest_year
|
||||||
|
|
||||||
#: regexes for valid license lines at tops of files
|
#: regexes for valid license lines at tops of files
|
||||||
|
@ -82,14 +82,6 @@ def spec(parser, args):
|
|||||||
if args.namespaces:
|
if args.namespaces:
|
||||||
fmt = "{namespace}." + fmt
|
fmt = "{namespace}." + fmt
|
||||||
|
|
||||||
tree_kwargs = {
|
|
||||||
"cover": args.cover,
|
|
||||||
"format": fmt,
|
|
||||||
"hashlen": None if args.very_long else 7,
|
|
||||||
"show_types": args.types,
|
|
||||||
"status_fn": install_status_fn if args.install_status else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
# use a read transaction if we are getting install status for every
|
# use a read transaction if we are getting install status for every
|
||||||
# spec in the DAG. This avoids repeatedly querying the DB.
|
# spec in the DAG. This avoids repeatedly querying the DB.
|
||||||
tree_context = lang.nullcontext
|
tree_context = lang.nullcontext
|
||||||
@ -99,46 +91,35 @@ def spec(parser, args):
|
|||||||
env = ev.active_environment()
|
env = ev.active_environment()
|
||||||
|
|
||||||
if args.specs:
|
if args.specs:
|
||||||
input_specs = spack.cmd.parse_specs(args.specs)
|
concrete_specs = spack.cmd.parse_specs(args.specs, concretize=True)
|
||||||
concretized_specs = spack.cmd.parse_specs(args.specs, concretize=True)
|
|
||||||
specs = list(zip(input_specs, concretized_specs))
|
|
||||||
elif env:
|
elif env:
|
||||||
env.concretize()
|
env.concretize()
|
||||||
specs = env.concretized_specs()
|
concrete_specs = env.concrete_roots()
|
||||||
|
|
||||||
if not args.format:
|
|
||||||
# environments are printed together in a combined tree() invocation,
|
|
||||||
# except when using --yaml or --json, which we print spec by spec below.
|
|
||||||
tree_kwargs["key"] = spack.traverse.by_dag_hash
|
|
||||||
tree_kwargs["hashes"] = args.long or args.very_long
|
|
||||||
print(spack.spec.tree([concrete for _, concrete in specs], **tree_kwargs))
|
|
||||||
return
|
|
||||||
else:
|
else:
|
||||||
tty.die("spack spec requires at least one spec or an active environment")
|
tty.die("spack spec requires at least one spec or an active environment")
|
||||||
|
|
||||||
for input, output in specs:
|
# With --yaml, --json, or --format, just print the raw specs to output
|
||||||
# With --yaml or --json, just print the raw specs to output
|
|
||||||
if args.format:
|
if args.format:
|
||||||
|
for spec in concrete_specs:
|
||||||
if args.format == "yaml":
|
if args.format == "yaml":
|
||||||
# use write because to_yaml already has a newline.
|
# use write because to_yaml already has a newline.
|
||||||
sys.stdout.write(output.to_yaml(hash=ht.dag_hash))
|
sys.stdout.write(spec.to_yaml(hash=ht.dag_hash))
|
||||||
elif args.format == "json":
|
elif args.format == "json":
|
||||||
print(output.to_json(hash=ht.dag_hash))
|
print(spec.to_json(hash=ht.dag_hash))
|
||||||
else:
|
else:
|
||||||
print(output.format(args.format))
|
print(spec.format(args.format))
|
||||||
continue
|
return
|
||||||
|
|
||||||
with tree_context():
|
with tree_context():
|
||||||
# Only show the headers for input specs that are not concrete to avoid
|
print(
|
||||||
# repeated output. This happens because parse_specs outputs concrete
|
spack.spec.tree(
|
||||||
# specs for `/hash` inputs.
|
concrete_specs,
|
||||||
if not input.concrete:
|
cover=args.cover,
|
||||||
tree_kwargs["hashes"] = False # Always False for input spec
|
format=fmt,
|
||||||
print("Input spec")
|
hashlen=None if args.very_long else 7,
|
||||||
print("--------------------------------")
|
show_types=args.types,
|
||||||
print(input.tree(**tree_kwargs))
|
status_fn=install_status_fn if args.install_status else None,
|
||||||
print("Concretized")
|
hashes=args.long or args.very_long,
|
||||||
print("--------------------------------")
|
key=spack.traverse.by_dag_hash,
|
||||||
|
)
|
||||||
tree_kwargs["hashes"] = args.long or args.very_long
|
)
|
||||||
print(output.tree(**tree_kwargs))
|
|
||||||
|
@ -24,7 +24,7 @@
|
|||||||
|
|
||||||
|
|
||||||
# tutorial configuration parameters
|
# tutorial configuration parameters
|
||||||
tutorial_branch = "releases/v0.22"
|
tutorial_branch = "releases/v0.23"
|
||||||
tutorial_mirror = "file:///mirror"
|
tutorial_mirror = "file:///mirror"
|
||||||
tutorial_key = os.path.join(spack.paths.share_path, "keys", "tutorial.pub")
|
tutorial_key = os.path.join(spack.paths.share_path, "keys", "tutorial.pub")
|
||||||
|
|
||||||
|
@ -180,7 +180,7 @@ def ensure_mirror_usable(self, direction: str = "push"):
|
|||||||
if errors:
|
if errors:
|
||||||
msg = f"invalid {direction} configuration for mirror {self.name}: "
|
msg = f"invalid {direction} configuration for mirror {self.name}: "
|
||||||
msg += "\n ".join(errors)
|
msg += "\n ".join(errors)
|
||||||
raise spack.mirror.MirrorError(msg)
|
raise MirrorError(msg)
|
||||||
|
|
||||||
def _update_connection_dict(self, current_data: dict, new_data: dict, top_level: bool):
|
def _update_connection_dict(self, current_data: dict, new_data: dict, top_level: bool):
|
||||||
# Only allow one to exist in the config
|
# Only allow one to exist in the config
|
||||||
|
@ -3839,12 +3839,21 @@ def splice_at_hash(
|
|||||||
self._splices.setdefault(parent_node, []).append(splice)
|
self._splices.setdefault(parent_node, []).append(splice)
|
||||||
|
|
||||||
def _resolve_automatic_splices(self):
|
def _resolve_automatic_splices(self):
|
||||||
"""After all of the specs have been concretized, apply all immediate
|
"""After all of the specs have been concretized, apply all immediate splices.
|
||||||
splices in size order. This ensures that all dependencies are resolved
|
|
||||||
|
Use reverse topological order to ensure that all dependencies are resolved
|
||||||
before their parents, allowing for maximal sharing and minimal copying.
|
before their parents, allowing for maximal sharing and minimal copying.
|
||||||
|
|
||||||
"""
|
"""
|
||||||
fixed_specs = {}
|
fixed_specs = {}
|
||||||
for node, spec in sorted(self._specs.items(), key=lambda x: len(x[1])):
|
|
||||||
|
# create a mapping from dag hash to an integer representing position in reverse topo order.
|
||||||
|
specs = self._specs.values()
|
||||||
|
topo_order = list(traverse.traverse_nodes(specs, order="topo", key=traverse.by_dag_hash))
|
||||||
|
topo_lookup = {spec.dag_hash(): index for index, spec in enumerate(reversed(topo_order))}
|
||||||
|
|
||||||
|
# iterate over specs, children before parents
|
||||||
|
for node, spec in sorted(self._specs.items(), key=lambda x: topo_lookup[x[1].dag_hash()]):
|
||||||
immediate = self._splices.get(node, [])
|
immediate = self._splices.get(node, [])
|
||||||
if not immediate and not any(
|
if not immediate and not any(
|
||||||
edge.spec in fixed_specs for edge in spec.edges_to_dependencies()
|
edge.spec in fixed_specs for edge in spec.edges_to_dependencies()
|
||||||
|
@ -1431,8 +1431,6 @@ def tree(
|
|||||||
class Spec:
|
class Spec:
|
||||||
#: Cache for spec's prefix, computed lazily in the corresponding property
|
#: Cache for spec's prefix, computed lazily in the corresponding property
|
||||||
_prefix = None
|
_prefix = None
|
||||||
#: Cache for spec's length, computed lazily in the corresponding property
|
|
||||||
_length = None
|
|
||||||
abstract_hash = None
|
abstract_hash = None
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
@ -3702,18 +3700,6 @@ def __getitem__(self, name: str):
|
|||||||
|
|
||||||
return child
|
return child
|
||||||
|
|
||||||
def __len__(self):
|
|
||||||
if not self.concrete:
|
|
||||||
raise spack.error.SpecError(f"Cannot get length of abstract spec: {self}")
|
|
||||||
|
|
||||||
if not self._length:
|
|
||||||
self._length = 1 + sum(len(dep) for dep in self.dependencies())
|
|
||||||
return self._length
|
|
||||||
|
|
||||||
def __bool__(self):
|
|
||||||
# Need to define this so __len__ isn't used by default
|
|
||||||
return True
|
|
||||||
|
|
||||||
def __contains__(self, spec):
|
def __contains__(self, spec):
|
||||||
"""True if this spec or some dependency satisfies the spec.
|
"""True if this spec or some dependency satisfies the spec.
|
||||||
|
|
||||||
@ -4472,7 +4458,7 @@ def clear_caches(self, ignore=()):
|
|||||||
if h.attr not in ignore:
|
if h.attr not in ignore:
|
||||||
if hasattr(self, h.attr):
|
if hasattr(self, h.attr):
|
||||||
setattr(self, h.attr, None)
|
setattr(self, h.attr, None)
|
||||||
for attr in ("_dunder_hash", "_prefix", "_length"):
|
for attr in ("_dunder_hash", "_prefix"):
|
||||||
if attr not in ignore:
|
if attr not in ignore:
|
||||||
setattr(self, attr, None)
|
setattr(self, attr, None)
|
||||||
|
|
||||||
|
@ -10,9 +10,6 @@
|
|||||||
|
|
||||||
import spack.config
|
import spack.config
|
||||||
import spack.deptypes as dt
|
import spack.deptypes as dt
|
||||||
import spack.package_base
|
|
||||||
import spack.paths
|
|
||||||
import spack.repo
|
|
||||||
import spack.solver.asp
|
import spack.solver.asp
|
||||||
from spack.installer import PackageInstaller
|
from spack.installer import PackageInstaller
|
||||||
from spack.spec import Spec
|
from spack.spec import Spec
|
||||||
@ -232,3 +229,19 @@ def test_spliced_build_deps_only_in_build_spec(splicing_setup):
|
|||||||
assert _has_build_dependency(build_spec, "splice-z")
|
assert _has_build_dependency(build_spec, "splice-z")
|
||||||
# Spliced build dependencies are removed
|
# Spliced build dependencies are removed
|
||||||
assert len(concr_goal.dependencies(None, dt.BUILD)) == 0
|
assert len(concr_goal.dependencies(None, dt.BUILD)) == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_spliced_transitive_dependency(splicing_setup):
|
||||||
|
cache = ["splice-depends-on-t@1.0 ^splice-h@1.0.1"]
|
||||||
|
goal_spec = Spec("splice-depends-on-t^splice-h@1.0.2")
|
||||||
|
|
||||||
|
with CacheManager(cache):
|
||||||
|
spack.config.set("packages", _make_specs_non_buildable(["splice-depends-on-t"]))
|
||||||
|
_enable_splicing()
|
||||||
|
concr_goal = goal_spec.concretized()
|
||||||
|
# Spec has been spliced
|
||||||
|
assert concr_goal._build_spec is not None
|
||||||
|
assert concr_goal["splice-t"]._build_spec is not None
|
||||||
|
assert concr_goal.satisfies(goal_spec)
|
||||||
|
# Spliced build dependencies are removed
|
||||||
|
assert len(concr_goal.dependencies(None, dt.BUILD)) == 0
|
||||||
|
@ -4,9 +4,11 @@
|
|||||||
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
import spack.config
|
||||||
import spack.environment as ev
|
import spack.environment as ev
|
||||||
import spack.error
|
import spack.error
|
||||||
import spack.solver.asp as asp
|
import spack.solver.asp as asp
|
||||||
|
import spack.store
|
||||||
from spack.cmd import (
|
from spack.cmd import (
|
||||||
CommandNameError,
|
CommandNameError,
|
||||||
PythonNameError,
|
PythonNameError,
|
||||||
|
@ -315,6 +315,7 @@ def test_pkg_grep(mock_packages, capfd):
|
|||||||
"depends-on-manyvariants",
|
"depends-on-manyvariants",
|
||||||
"manyvariants",
|
"manyvariants",
|
||||||
"splice-a",
|
"splice-a",
|
||||||
|
"splice-depends-on-t",
|
||||||
"splice-h",
|
"splice-h",
|
||||||
"splice-t",
|
"splice-t",
|
||||||
"splice-vh",
|
"splice-vh",
|
||||||
|
@ -7,6 +7,7 @@
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
|
import spack.config
|
||||||
import spack.environment as ev
|
import spack.environment as ev
|
||||||
import spack.error
|
import spack.error
|
||||||
import spack.spec
|
import spack.spec
|
||||||
|
@ -0,0 +1,22 @@
|
|||||||
|
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
|
||||||
|
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
||||||
|
#
|
||||||
|
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
||||||
|
|
||||||
|
from spack.package import *
|
||||||
|
|
||||||
|
|
||||||
|
class SpliceDependsOnT(Package):
|
||||||
|
"""Package that depends on splice-t"""
|
||||||
|
|
||||||
|
homepage = "http://www.example.com"
|
||||||
|
url = "http://www.example.com/splice-depends-on-t-1.0.tar.gz"
|
||||||
|
|
||||||
|
version("1.0", md5="0123456789abcdef0123456789abcdef")
|
||||||
|
|
||||||
|
depends_on("splice-t")
|
||||||
|
|
||||||
|
def install(self, spec, prefix):
|
||||||
|
with open(prefix.join("splice-depends-on-t"), "w") as f:
|
||||||
|
f.write("splice-depends-on-t: {0}".format(prefix))
|
||||||
|
f.write("splice-t: {0}".format(spec["splice-t"].prefix))
|
Loading…
Reference in New Issue
Block a user