This implements Package API v2.0, and is an opt-in feature for repos. It can be enabled with
```yaml
repo:
...
api: v2.0
```
It differs from the current default v1.0 as follows:
1. Package names can only contain `-` as a separator.
2. Package names can only be lowercase.
3. Package directory names are valid Python module names.
4. The repo namespace and its directory name are the same.
5. The `packages` subdir, which is configurable, should be a directory
name that is also a valid Python module name.
6. There is a one to one mapping between Spack package names and Python
module names.
7. Import statements `import spack.pkg.namespace.package_module` in
`package.py` files need to specify the canonical package module.
To go from Spack package name to Python module name:
- Replace `-` by `_`
- Add a leading `_` if the package name starts with a digit
To go from Python module name to Spack package name:
- Strip leading `_`
- Replace `_` by `-`.
This PR improves compatibility with specs installed before #45189, and with externals specifying a compiler, by using the annotated compiler to "satisfy" a spec query.
On top of that, the PR adds a new flag for:
```console
$ spack find --specfile-format -I %gcc
-- linux-ubuntu20.04-icelake / gcc@10.5.0 -----------------------
[+] [v4] ca-certificates-mozilla@2023-05-30 [e] [v4] cmake@3.31.6 [+] [v4] gcc-runtime@10.5.0 [e] [v4] glibc@2.31 [+] [v4] gmake@4.4.1 [+] [v4] hdf5@1.14.5 [+] [v4] pkgconf@2.2.0 [+] [v4] zlib-ng@2.2.1
==> 8 installed packages
```
which shows the specfile format of the specs being retrieved.
Fixes a logic bug where a -> b was assumed to imply not a -> not b
in conditional requirements.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
When trying to use Spack to build Intel TBB on the EOS network file
system we use at CERN, I am facing the following error:
```
Launching the installer...
Installation directory is not empty.
The product cannot be installed into nonempty directory
'/eos/home-s/sswatman/spack/opt/spack/linux-x86_64_v2/...
intel-oneapi-tbb-2021.12.0-3jlx6hlr3z6di42f3qy35eizcse7u2tk'.
```
This error appears to happen because Spack tries to set `$HOME` to the
prefix of the package, into which the Intel installer is also trying to
install it's software. I've found that an easy fix is to set `$HOME` to
a directory in the build stage instead, as this can be guaranteed to
remain empty.
* `x=*` constrained by `+x` now produces a boolean valued variant instead of a multi-valued variant.
* Values are now always stored as a tuple internally, whether bool, single or multi-valued.
* Value assignment has a stricter api to prevent ambiguity / type issues related to
`variant.value = "x"` / `variant.value = ["x"]` / `variant.value = ("x",)`. It's now `variant.set("x", ...)` for
single and multi-valued variants.
* The `_original_value` prop is dropped, since it was unused.
* The wildcard `*` is no longer a possible variant value in any type of variant, since the *parser*
deals with it and creates a variant with no values.
The second change technically affects non-Windows, but the
behavior should be exactly the same:
* Packages no longer have access to `.msbuild` and `.nmake`
automatically; they now get them via a dependency on `msvc`.
* Update two CMake-based packages that call `make test` to
instead call `ctest` (`netcdf-cxx4` and `pegtl`).
CMake-based packages should do this because on Windows
`make test` will not generally work, but `ctest` does.
* Fix `openssl` "make test" on Windows (WRT prior point: not
a CMake-based package).
Show progress meter for fetches when `stdout` is a `tty`.
* fetch_strategy.py: show progress
* "Fetched: x MB at y MB/s"
* add tests, show % if content-length
This PR fixes the issues with `%` and reused specs, due to https://github.com/spack/spack/issues/49847#issuecomment-2774640234
It does so by adding another layer of indirection, so that whenever a spec
`foo %bar` is encountered, the `%bar` part is encoded as an
`attr("build_requirement", ...)`.
Then:
1. If `foo` is a node to be built, then the build requirement implies a dependency
2. Otherwise it implies looking e.g. reused specs metadata, and ensure it matches
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
When the solver produces specs that do not satisfy the input
constraints, dump both input and output specs as json in an temporary
dir and ask the user to upload these files in a bug report.
Adding binary provenance requires we track the commit.
Typically this has been an optional form for the encompassing git ref.
Moving towards always defining a commit means we need to have space
to store a user requested ref that will then be paired with a commit
sha.
Currently we inject runtimes when a package has a direct build dep on a
compiler, but what matters is whether the package depends on a language.
That way we can avoid recursion of injecting runtimes to runtimes
without a rule in the solver: runtimes don't depend on languages, they
just have a build dep on the same compiler.
In this way, to run them, we just need to run:
spack unit-test lib/spack/spack/test/concretization
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* TestSuite: add type hints
* spack test run: add a --timeout argument
* pipelines: allow 2 minutes to run tests
* Fix docstrings, increase maximum pipelines time for tests to 5 mins.
* Use SIGTERM first, SIGKILL shortly after
* Add unit-tests for "start_build_process"
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Multiple build systems have been part of Spack for a long
time now, and they are rarely the cause of a missing attribute.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Similar to the range-or-specific-version ambiguity of `@1.2` in the past,
which was solved with `@1.2` vs `@=1.2` we still have the ambiguity of
`name=a,b,c` in multi-valued variants. Do they mean "at least a,b,c" or
"exactly a,b,c"?
This issue comes up in for example `gcc languages=c,cxx`; there's no
way to exclude `fortran`.
The ambiguity is resolved with syntax `:=` to distinguish concrete from
abstract.
The following strings parse as **concrete** variants:
* `name:=a,b,c` => values exactly {a, b, c}
* `name:=a` => values exactly {a}
* `+name` => values exactly {True}
* `~name` => values exactly {False}
The following strings parse as **abstract** variants:
* `name=a,b,c` values at least {a, b, c}
* `name=*` special case for testing existence of a variant; values are at
least the empty set {}
As a reminder
* `satisfies(lhs, rhs)` means `concretizations(lhs)` ⊆ `concretizations(rhs)`
* `intersects(lhs, rhs)` means `concretizations(lhs)` ∩ `concretizations(rhs)` ≠ ∅
where `concretizations(...)` is the set of sets of variant values in this case.
The satisfies semantics are:
* rhs abstract: rhs values is a subset of lhs values (whether lhs is abstract or concrete)
* lhs concrete, rhs concrete: set equality
* lhs abstract, rhs concrete: false
and intersects should mean
* lhs and rhs abstract: true (the union is a valid concretization under both)
* lhs or rhs abstract: true iff the abstract variant's values are a subset of the concrete one
* lhs concrete, rhs concrete: set equality
Concrete specs with single-valued variants are printed `+foo`, `~foo` and `foo=bar`;
only multi-valued variants are printed with `foo:=bar,baz` to reduce the visual noise.
Return a single scope from environment.env_config_scope
Return a single scope from environment_path_scope
---------
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
This commit reorders ASP setup, so that rules from
possible compilers are collected first.
This allows us to know the dependencies that may be
injected before counting the possible dependencies,
so we can account for them too.
Proceeding this way makes it easier to inject
complex runtimes, like hip.
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Deal with the "issue" that passing a str instance does not cause a
type check failure, because str is a subset of Sequence[str] and
Iterable[str]. Instead fix it by special casing the str instance.
