In #30882, we made Spack ignore `-Werror` calls so that it could more easily build
projects that inject `-Werror` into their builds. We did this by translating them to
`-Wno-error` in the compiler wrapper. However, some compilers (like `nvhpc`) do not
support `-Wno-error`. We need to exclude them from this feature until they do.
- [x] make a property on `PackageBase` for `keep_werror` that knows not to use it for
`nvhpc`.
- [x] update property so that it keeps only the specific `-Werror=...` args for newer nvhpc's,
which support `-Wno-error` but not `-Wno-error=...`
---------
Co-authored-by: William Mou <william.mou1024@gmail.com>
Co-authored-by: Tom Scogland <scogland1@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
When Spack concretizes environments, it prints every (newly concretized) root spec
individually with all of its dependencies. For most reasonably sized environments, this
is too much output. This is true for three commands:
* `spack concretize` when concretizing an environment with newly added specs
* `spack install` when installing an environment with newly added specs
* `spack spec` with no arguments in an environment
The output dates back to before we had unified environments or nicer spec traversal
routines, and we can improve it.
This PR makes environment concretization output analogous to what we do for regular
specs. Just like `spack spec` for a single spec, we show all root specs with no
indentation, so you can easily see the specs you explicitly requested. Dependencies are
shown:
1. With indentation according to their depth in a breadth-first traversal starting at
the roots;
2. Only once if they appear on paths from multiple roots
So, the default is now consistent with `spack spec` for one spec--it's `--cover=nodes`.
i.e., if there are 100 specs in your environment, you'll get 100 lines of output.
If you want to see more details, you can do that with `spack spec` using the arguments
you're already familiar with. For example, if you wanted to see dependency types and
*all* dependencies, you could use `spack spec -l --cover=edges`. Or you could add
deptypes and namespaces with, e.g. `spack spec -ltN`.
With no arguments in an environment, `spack spec` concretizes (if necessary) and shows
the concretized environment. If you run `spack concretize` *first*, inspecting the
environment repeatedly with `spack spec` will be fast, as everything is already in the
`spack.lock` file.
- [x] factor most logic of `Spec.tree()` out of `Spec` class into `spack.spec.tree()`,
which can take multiple specs as roots.
- [x] make `Spec.tree()` call `spack.spec.tree()`
- [x] `spack.environment.display_specs()` now uses `spack.spec.tree()`
- [x] Update `spack concretize`
- [x] Update `spack install`
- [x] Update `spack spec` to call `spack.spec.tree()` for environments.
- [x] Continue to output specs individually for `spack spec` when using
`--yaml` or `--json`
`BuildcacheBootstrapper` uses `Spec.intersects` to match specs needed
for bootstrapping against the binary cache. The specs were not
sufficiently-detailed to prevent matching e.g. cached binaries for
Mac OS on Windows; this commit adds the platform to each requested
bootstrap spec to prevent that.
Remove support for `cray` as a separate platform.
Any platform previously detected as `cray` is now detected as `linux`.
Users who still need platform=cray have to stick to Spack 0.22
Fixes a bug in the concretizer where specs depending on a host
incompatible libc would be used. This bug triggers when nothing is
built.
In the case where everything is reused, there is no libc provider from
the perspective of the solver, there is only compatible_libc. This
commit ensures that we require a host compatible libc on any reused
spec, additionally to requiring compat with the chosen libc provider.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Change the installer to take `([pkg], args)` in the constructor instead
of `[(pkg, args)]`. The reason is that certain arguments are global
settings, and the new API ensures that those arguments cannot be
different across different "build requests".
The `explicit` install arg is now a list of hashes, and the installer is
no longer responsible for determining what package is installed
explicitly. This way environment installs can simply pass the list of
environment roots, without them necessarily being explicit build
requests. For example an env with two roots [a, b], where b depends on
a, would not always cause spack install to mark b as explicit.
Notice that `overwrite` already took a list of hashes, this makes
`explicit` consistent.
`package.do_install(explicit=True)` continues to take a boolean.
The windows wrappers for basic functions like `os.symlink`,
`os.readlink` and `os.path.islink` in the `llnl.util.symlink` module
have bugs, and trigger more file system operations on non-windows than
they should.
This commit just binds `llnl.util.symlink.symlink = os.symlink` etc so
built-in functions are used on non-windows
`spack clean <spec>` will now resolve specs based on the active environment if one is active.
If an env is active but no matching spec is found, this will fall back on fully concretizing.
Before this PR, if Spack could see a possibility to reuse a spec that
doesn't match a strong preference, it would do so. After the PR, a
strong preference would take precedence.
avoid calling `spec.target` when None.
When an external compiler package has an `os` set but no `target` set, Spack
currently falls into a codepath that calls `spec.target` (which itself calls
`spec.architecture.target.Microarchitecture`) when `spec.architecture.target`
is None, throwing an error.
e.g.
```
packages:
gcc:
externals:
- spec: gcc@12.3.1 os=rhel7
prefix: /usr
```
---------
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
This fixes a bug occurring when two root specs need to select
old versions, and these versions have the same penalty in the
optimization. This sometimes caused an older version to be
preferred to a more recent one.
The issue was the omission of `PackageNode` in the optimization
tuple.
Use correct path separator in get_all_package_diffs for all platforms.
Ensures correct package change computation on Windows when pruning unchanged specs in Gitlab CI
This fixes an issue where ghcr, gitlab and possibly other container registries paginate tags by default, which violates the OCI spec v1.0, but is common practice (the spec was broken itself). After this commit, you can create build cache indices of > 100 specs on ghcr.
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
Add support for Gitlab CI on Windows
This PR adds the config changes required to configure and execute
Gitlab pipelines running Windows builds on Windows runners using
the existing Gitlab CI infrastructure (and newly added Windows
infrastructure).
* Adds support for generating child pipelines dispatched to Windows runners
* Refactors the relevant pre-scripts, scripts, and post scripts to be compatible with Windows
* Adds Windows config section describing Windows jobs
* Adds VTK as Windows build stack (to be expanded later)
* Modifies proj to build on Windows
* Refactors Windows rpath symlinking to avoid system libs and externals
---------
Co-authored-by: Ryan Krattiger <ryan.krattiger@kitware.com>
Co-authored-by: Mike VanDenburgh <michael.vandenburgh@kitware.com>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
Symlinks on Windows can use longpath prefixes (\\?\); these are fine
in the context of win32 API interactions but break numerous facets of
Spack behavior that rely on string parsing/matching (archiving,
binary distributions, tarball extraction, view regen, etc).
Spack's internal readlink method (llnl.util.symlink.readlink)
gracefully handles this by removing the prefix and otherwise behaving
exactly as os.readlink does, so we should prefer that in all cases.
Apparently urllib can throw a range of different exceptions:
1. HTTPError
2. URLError with e.reason set to the actual exception
3. TimeoutError from getresponse, which is not wrapped
* archive: relative links only
Ensure all links written into tarfiles generated from Spack prefixes do not contain symlinks pointing outside the prefix
* binary_distribution: limit extraction to prefix
Ensure files extracted from spackballs are not links pointing outside of the prefix
* Ensure rpaths are properly set on Windows
* hard error on extraction of absolute links
* refactor for non link-modifying approach
* Restore tarball extraction to original impl
* use custom readlink
* cleanup symlink module
* make lstrip
Add the ability to include any number of (potentially nested) concrete environments, e.g.:
```yaml
spack:
specs: []
concretizer:
unify: true
include_concrete:
- /path/to/environment1
- /path/to/environment2
```
or, from the CLI:
```console
$ spack env create myenv
$ spack -e myenv add python
$ spack -e myenv concretize
$ spack env create --include-concrete myenv included_env
```
The contents of included concrete environments' spack.lock files are
included in the environment's lock file at creation time. Any changes
to included concrete environments are only reflected after the environment
is re-concretized from the re-concretized included environments.
- [x] Concretize included envs
- [x] Save concrete specs in memory by hash
- [x] Add included envs to combined env's lock file
- [x] Add test
- [x] Update documentation
Co-authored-by: Kayla Butler <<butler59@llnl.gov>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.co
m>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
