Commit Graph

7649 Commits

Author SHA1 Message Date
Massimiliano Culpo
55196252dd
Improve reporting when bootstrapping from source
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:17 +01:00
Massimiliano Culpo
d3a7a73a00
Improve error messages for statically checked specs
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:16 +01:00
Massimiliano Culpo
21afe2af1f
spec: implemented direct satisfy semantic
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:16 +01:00
Massimiliano Culpo
646c2f42c4
Fixup binary cache reuse
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:15 +01:00
Massimiliano Culpo
1ab3e8c776
Write adaptors for CompilerSpec and Compiler
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:15 +01:00
Massimiliano Culpo
49978d5b6c
Fix reading Cray manifest files
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:14 +01:00
Massimiliano Culpo
a1866d7a4b
(WIP) Fix LMod module generation
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:14 +01:00
Massimiliano Culpo
6674ce6dc4
(WIP) Remove deprecated argument for Spec.format
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:14 +01:00
Massimiliano Culpo
f729353ac3
fixup: spec copies compiler annotation
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:13 +01:00
Massimiliano Culpo
73e0cf07cb
Restore bootstrapping from binaries
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:13 +01:00
Massimiliano Culpo
8842df3f94
Restore bootstrapping from sources
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:12 +01:00
Massimiliano Culpo
8d3132b26b
spec: change semantic of __getitem__
Now __getitem__ can pick items in the transitive link/run graph,
or from direct build dependencies.

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:12 +01:00
Massimiliano Culpo
e342de41b2
spec: bump specfile format to v5
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:11 +01:00
Massimiliano Culpo
0415390270
Overhaul of the spack.compilers package
Now the package contains modules that help using, or
detecting, compiler packages.

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:11 +01:00
Massimiliano Culpo
5b7caba4a6
Remove spack.compilers Python modules
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:10 +01:00
Massimiliano Culpo
f59c120e0a
(WIP) Install mechanism 2025-02-28 23:34:10 +01:00
Massimiliano Culpo
a0cae04302
(WIP) Recover bootstrapping from binaries on linux 2025-02-28 23:34:10 +01:00
Massimiliano Culpo
496ae0bb31
unit-tests: fix concretization and spack compiler tests 2025-02-28 23:34:09 +01:00
Massimiliano Culpo
4c06f83c60
solver: first working implementation of compiler as nodes
This commit changes the model to treat compilers as nodes, and
drops the concept of a "compiler" as a bundle of a C, C++, and
Fortran compiler.

Implementation does not rely on `Compiler` or `CompilerSpec`.
2025-02-28 23:34:09 +01:00
Massimiliano Culpo
0b11775529
Deprecate packages:all:compiler and update default configs
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:08 +01:00
Massimiliano Culpo
a10f3295bc
directives: remove workaround for the c, cxx and fortran language
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-28 23:34:07 +01:00
Massimiliano Culpo
285926cb69
Overhaul the spack compiler command
This reverts commit 2c47dddbc1.

Now, `spack compiler` writes by default in packages.yaml. Entries
in old `compilers.yaml` are converted to external specs as a way to
support legacy configuration.

Since this operation is expensive, an environment variable can be
used to enforce the deprecation of `compiler.yaml`.

The --mixed-toolchain option has been deprecated, since it stops to
make sense once compiler are treated as nodes.
2025-02-28 23:34:07 +01:00
Massimiliano Culpo
da02a4a606
Allow reading old JSON files 2025-02-28 23:34:06 +01:00
Massimiliano Culpo
37dd777a51
parse_with_version_concrete: remove compiler= switch 2025-02-28 23:34:06 +01:00
Massimiliano Culpo
7dc824d1ff
Make CompilerSpec raise on __init__ 2025-02-28 23:34:06 +01:00
Massimiliano Culpo
78744b11ae
parser: parse compilers as direct build deps 2025-02-28 23:34:02 +01:00
Harmen Stoppels
8e7489bc17
Revert "Honor cmake_prefix_paths property if available (#42569)" (#49237)
This reverts commit fe171a560b.
2025-02-28 23:33:02 +01:00
John W. Parent
d234df62d7
Solver: Cache Concretization Results (#48198)
Concretizer caching for reusing solver results
2025-02-28 12:42:00 -06:00
John W. Parent
5bd184aaaf
Windows Rpath: Allow package test rpaths (#47072)
On Windows, libraries search their directory for dependencies, and
we help libraries in Spack-built packages locate their dependencies
by symlinking them into the dependent's directory (we refer to this
as simulated RPATHing).

We extend the convenience functionality here to support base library
directories outside of the package prefix: this is primarily for
running tests in the build directory (which is not located inside
of the final install prefix chosen by spack).
2025-02-27 19:16:00 -08:00
Kyle Knoepfel
fe171a560b
Honor cmake_prefix_paths property if available (#42569)
* Honor package-specified cmake_prefix_paths at runtime

* Add paths in the correct order and prune duplicates

* Normalize paths for windows' sake
2025-02-27 11:11:22 -07:00
Harmen Stoppels
7363047b82
schema: additionalKeysAreSpecs (#49221)
Currently we validate all keys as specs, but it's meant to validate only additional keys in all cases.
2025-02-27 12:17:25 +01:00
Harmen Stoppels
5da4f18188
schema/modules.py: remove lmod props from tcl schema (#49220) 2025-02-27 10:48:22 +01:00
Massimiliano Culpo
3caa3132aa
python: allow it as a build-tool again (#49201)
Python was removed from being a build tool in #46980, due to issues
when reusing specs. This PR adds a new rule to match the interpreter
among different Python packages, in clingo.

It also adds a bunch of new "build-tools", so that specs like:
```
py-matplotlib backend=tkagg
```
can be concretized in one go.

Modifications:
- [x] Make `py-matplotlib backend=tkagg` concretizable
- [x] Add unit-tests to ensure situations like in #46980 do not happen

---------

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-26 15:04:31 -08:00
Massimiliano Culpo
dbd531112c
Assign priorities to configuration scopes (take 2) (#49187)
Currently, the custom config scopes are pushed at the top when constructing
configuration, and are demoted whenever a context manager activating an
environment is used - see #48414 for details. Workflows that rely on the order
in the [docs](https://spack.readthedocs.io/en/latest/configuration.html#custom-scopes)
are thus fragile, and may break

This PR allows to assign priorities to scopes, and ensures that scopes of lower priorities
are always "below" scopes of higher priorities. When scopes have the same priority,
what matters is the insertion order.

Modifications:
- [x] Add a mapping that iterates over keys according to priorities set when
      adding the key/value pair
- [x] Use that mapping to allow assigning priorities to configuration scopes
- [x] Assign different priorities for different kind of scopes, to fix a bug, and
      add a regression test
- [x] Simplify `Configuration` constructor
- [x] Remove `Configuration.pop_scope`

---------

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-26 10:52:19 -08:00
psakievich
ae5e121502
Preserve --lines (#49194)
This does not propagate in parsing. Open to other ideas.
2025-02-26 17:48:01 +00:00
Massimiliano Culpo
9e508b0321
Revert "Assign priorities to configuration scopes (#48420)" (#49185)
All the build jobs in pipelines are apparently relying on the bug that was fixed.

The issue was not caught in the PR because generation jobs were fine, and
there was nothing to rebuild.

Reverting to fix pipelines in a new PR.

This reverts commit 3ad99d75f9.
2025-02-25 02:33:41 -08:00
dependabot[bot]
1cc63e2b7c
build(deps): bump sphinx from 8.2.0 to 8.2.1 in /lib/spack/docs (#49180)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 8.2.0 to 8.2.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/v8.2.1/CHANGES.rst)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v8.2.0...v8.2.1)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-25 02:33:03 -07:00
Harmen Stoppels
4e311a22d0
spec.py: remove VariantMap.concrete (#49170)
VariantMap.concrete is unused, and would be incorrect if it were used
due to conditional variants.

Just let the Spec dictate what is concrete and what is not.
2025-02-25 10:18:06 +01:00
Massimiliano Culpo
3ad99d75f9
Assign priorities to configuration scopes (#48420)
Currently, environments can end up with higher priority than `-C` custom
config scopes and `-c` command line arguments sometimes. This shouldn't
happen -- those explicit CLI scopes should override active environments.

Up to now configuration behaved like a stack, where scopes could be only be
pushed at the top. This PR allows to assign priorities to scopes, and ensures
that scopes of lower priorities are always "below" scopes of higher priorities.

When scopes have the same priority, what matters is the insertion order.

Modifications:
- [x] Add a mapping that iterates over keys according to priorities set when
      adding the key/value pair
- [x] Use that mapping to allow assigning priorities to configuration scopes
- [x] Assign different priorities for different kind of scopes, to fix a bug, and
      add a regression test
- [x] Simplify `Configuration` constructor
- [x] Remove `Configuration.pop_scope`
- [x] Remove `unify:false` from custom `-C` scope in pipelines

On the last modification: on `develop`, pipelines are relying on the environment
being able to override `-C` scopes, which is a bug. After this fix, we need to be
explicit about the unification strategy in each stack, and remove the blanket
`unify:false` from the highest priority scope

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-25 00:58:16 -08:00
Harmen Stoppels
e783a2851d
Revert "Repo.packages_with_tags: do not construct a set of all packages (#49141)" (#49172)
This reverts commit 0da5bafaf2.
2025-02-24 16:46:41 +01:00
Harmen Stoppels
4e7a5e9362
spack verify libraries: verify dependencies of installed packages can be resolved (#49124)
Currently, we have `config:shared_linking:missing_library_policy` to error
or warn when shared libraries cannot be resolved upon install.

The new `spack verify libraries` command allows users to run this post
install hook at any point in time to check whether their current
installations can resolve shared libs in rpaths.
2025-02-24 11:28:06 +01:00
Harmen Stoppels
974abc8067
Add typehints for directory_layout / Spec.prefix (#48652) 2025-02-24 09:47:07 +00:00
Massimiliano Culpo
2f9ad5f34d
spec.py: fix virtual reconstruction for old specs (#49103)
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2025-02-24 01:23:37 -07:00
Harmen Stoppels
6cd74efa90
Spec.ensure_external_path_if_external, Spec.inject_patches_variant -> spack.solver.asp (#48988)
y
2025-02-24 08:36:23 +01:00
ByteHamster
b304a2d854
Fix installing rust@nightly (#49098)
Installing `rust@nightly` fails because the package file declares a conflict of rust versions older than `:1.64` with `gcc>=13`. However, because `nightly` is alphanumerically smaller than any actual version number, `nightly` is incorrectly detected to have a conflict with `gcc>=13` as well. Marking `nightly` as an infinity version instead solves this.
2025-02-21 09:53:46 -08:00
Ryan Krattiger
1fa1864b37
Reproducer should deduce artifact root from concrete environment (#45281)
* Reproducer should decude artifact root from concrete environment

* Add documentation on the layout of the artifacts directory

* Use dag hash in the container name

* Add reproducer options to improve local testing

* --use-local-head allows running reproducer with
  the current Spack HEAD commit rather than computing
  a commit for the reproducer

* Add test to verify commits and recreating reproduction environment

* Add test for non-merge commit case

* ci reproduce-build: Drop overwrite option
in favor of throwing an error if the working dir is non-empty
2025-02-21 10:46:43 -06:00
Harmen Stoppels
0da5bafaf2
Repo.packages_with_tags: do not construct a set of all packages (#49141) 2025-02-21 16:23:42 +01:00
Harmen Stoppels
b8ec69112f
Extracted changes from 45189 (#49137)
Co-Authored-By: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-21 10:58:00 +01:00
Massimiliano Culpo
a3645fd372
Make BaseConfiguration pickleable (#47545)
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-02-21 09:54:22 +01:00
dependabot[bot]
9bcd86071f
build(deps): bump sphinx from 8.1.3 to 8.2.0 in /lib/spack/docs (#49118)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 8.1.3 to 8.2.0.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/master/CHANGES.rst)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v8.1.3...v8.2.0)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-02-20 16:42:00 -08:00