Commit Graph

7340 Commits

Author SHA1 Message Date
Alex Hedges
34b2f28a5e
Fix malformed RST link in documentation (#47309) 2024-10-30 09:40:35 +01:00
Massimiliano Culpo
b8e3246e89
llnl.util.lang: add classes to help with deprecations (#47279)
* Add a descriptor to have a class level constant

This descriptor helps intercept places where we set a value on instances.
It does not really behave like "const" in C-like languages, but is the
simplest implementation that might still be useful.

* Add a descriptor to deprecate properties/attributes of an object

This descriptor is used as a base class. Derived classes may implement a
factory to return an adaptor to the attribute being deprecated. The
descriptor can either warn, or raise an error, when usage of the deprecated
attribute is intercepted.

---------

Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
2024-10-29 19:06:26 +01:00
Harmen Stoppels
ea1aa0714b
bootstrap: do not consider source when metadata file missing (#47278) 2024-10-29 10:57:31 +01:00
Harmen Stoppels
7af1a3d240
std_meson_args: deprecate (#47259) 2024-10-29 07:54:28 +01:00
Harmen Stoppels
962115b386
builder.py: builder_cls should be associated to spack.pkg module (#47269) 2024-10-29 07:53:06 +01:00
Harmen Stoppels
f81ca0cd89
directives_meta.py: use startswith to test module part of spack.pkg (#47270) 2024-10-29 07:51:36 +01:00
Greg Becker
e81ce18cad
cmd/solve: use interface from cmd/spec (#47182)
Currently, `spack solve` has different spec selection semantics than `spack spec`.
`spack solve` currently does not allow specifying a single spec when an environment is active.

This PR modifies `spack solve` to inherit the interface from `spack spec`, and to use
the same spec selection logic. This will allow for better use of `spack solve --show opt`
for debugging.

---------

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-28 19:28:03 -07:00
Harmen Stoppels
2ec4281c4f
Remove a few redundant imports (#47250)
* remove self-imports

* remove unused imports
2024-10-27 15:40:05 -06:00
Greg Becker
47e70c5c3a
explicit splice: do not fail for bad config replacement if target not matched (#46925)
Originally, concretization failed if the splice config points to an invalid replacement.

This PR defers the check until we know the splice is needed, so that irrelevant splices
with bad config cannot stop concretization.

While I was at it, I improved an error message from an assert to a ValueError.
2024-10-27 11:35:10 -07:00
Todd Gamblin
c8bebff7f5
Add -t short option for spack --backtrace (#47227)
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-26 09:16:31 +02:00
John W. Parent
7b27aed4c8
Normalize Spack Win entry points (#38648)
* Normalize Spack Win entrypoints

Currently Spack has multiple entrypoints on Windows that in addition to
differing from *nix implementations, differ from shell to shell on
Windows. This is a bit confusing for new users and in general
unnecessary.
This PR adds a normal setup script for the batch shell while preserving
the previous "click from file explorer for spack shell" behavior.
Additionally adds a shell title to both powershell and cmd letting users
know this is a Spack shell

* remove doskeys
2024-10-25 15:23:29 -04:00
Dom Heinzeller
ad0b256407
Intel/Oneapi compilers: suppress warnings when using Cray wrappers (#47046)
#44588 we added logic to suppress deprecation warnings for the
Intel classic compilers. This depended on matching against 

* The compiler names (looking for icc, icpc, ifort)
* The compiler version

When using an Intel compiler with fortran wrappers, the first check
always fails. To support using the fortran wrappers (in combination
with the classic Intel compilers), we remove the first check and
suppress if just the version matches. This works because:

* The newer compilers like icx can handle (ignore) the flags that
  suppress deprecation warnings
* The Cray wrappers pass the underlying compiler version (e.g. they
  report what icc would report)
2024-10-25 12:17:49 -07:00
Harmen Stoppels
7d86670826 ensure write_fd.close() isn't called when sys.std* cannot be redirected 2024-10-25 10:16:44 -07:00
Harmen Stoppels
ae306b73c3 Avoid a socket to communicate effectively a bit 2024-10-25 10:16:44 -07:00
Harmen Stoppels
b63cbe4e6e Replace MultiProcessFd with Connection objects
Connection objects are Python version, platform and multiprocessing
start method independent, so better to use those than a mix of plain
file descriptors and inadequate guesses in the child process whether it
was forked or not.

This also allows us to delete the now redundant MultiProcessFd class,
hopefully making things a bit easier to follow.
2024-10-25 10:16:44 -07:00
Harmen Stoppels
e86a3b68f7
file_cache.py: allow read transaction on uninitialized cache (#47212)
This allows the following

```python
cache.init_entry("my/cache")
with cache.read_transaction("my/cache") as f:
    data = f.read() if f is not None else None
```

mirroring `write_transaction`, which returns a tuple `(old, new)` where
`old` is `None` if the cache file did not exist yet.

The alternative that requires less defensive programming on the call
site would be to create the "old" file upon first read, but I did not
want to think about how to safely atomically create the file, and it's
not unthinkable that an empty file is an invalid format (for instance
the call site may expect a json file, which requires at least {} bytes).
2024-10-25 17:10:14 +02:00
Jordan Galby
f13d998d21
Add spack short version in config variables (#47016) 2024-10-25 07:34:59 +02:00
Tamara Dahlgren
1b0631b69e
Env help: expand and refine subcommand help and descriptions (#47089)
This PR is in response to a question in the `environments` slack channel (https://spackpm.slack.com/archives/CMHK7MF51/p1729200068557219) about inadequate CLI help/documentation for one specific subcommand.

This PR uses the approach I took for the descriptions and help for `spack test` subcommands.  Namely, I use the first line of the relevant docstring as the description, which is shown per subcommand in `spack env -h`, and the entire docstring as the help.  I then added, where it seemed appropriate, help.  I also tweaked argument docstrings to tighten them up, make consistent with similar arguments elsewhere in the command, and elaborate when it seemed important.  (The only subcommand I didn't touch is `loads`.)

For example, before:
```
$ spack env update -h
usage: spack env update [-hy] env

positional arguments:
  env               name or directory of the environment to activate

optional arguments:
  -h, --help        show this help message and exit
  -y, --yes-to-all  assume "yes" is the answer to every confirmation request
```

After the changes in this PR:
```
$ spack env update -h
usage: spack env update [-hy] env

update the environment manifest to the latest schema format

    update the environment to the latest schema format, which may not be
    readable by older versions of spack

    a backup copy of the manifest is retained in case there is a need to revert
    this operation
    

positional arguments:
  env               name or directory of the environment

optional arguments:
  -h, --help        show this help message and exit
  -y, --yes-to-all  assume "yes" is the answer to every confirmation request
```

---------

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-24 13:55:00 -07:00
Harmen Stoppels
5ac2b8a178
compilers.yaml: require list of strings for modules (#47197) 2024-10-24 13:28:38 -06:00
Kyle Knoepfel
e0eea48ccf
Restore bold uncolored font face (#47108)
Commit aa0825d642 accidentally added a semicolon
to the ANSI escape sequence even if the color code was `None` or unknown, breaking the
bold, uncolored font-face.  This PR restores the old behavior.

---------

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2024-10-24 09:11:43 +00:00
Harmen Stoppels
d8c8074762
bootstrap: add clingo 3.13 binaries and more (#47126) 2024-10-24 08:55:14 +02:00
Massimiliano Culpo
f6ad1e23f8
Improve Database.query* methods (#47116)
* Add type hints to all query* methods
* Inline docstrings
* Change defaults from `any` to `None` so they can be type hinted in old Python
* Pre-filter on given hashes instead of iterating over all db specs
* Fix a bug where the `--origin` option of uninstall had no effect
* Fix a bug where query args were not applied when searching by concrete spec

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-24 08:13:07 +02:00
Harmen Stoppels
79ad6f6b48
env: continue to mark non-roots as implicitly installed on partial env installs (#47183)
Fixes a change in behavior/bug in
70412612c7, where partial environment
installs would mark the selected spec as explicitly installed, even if
it was not a root of the environment.

The desired behavior is that roots by definition are the to be
explicitly installed specs. The specs on the `spack -e ... install x`
command line are just filters for partial installs, so leave them
implicitly installed if they aren't roots.
2024-10-23 21:17:40 +00:00
Scott Wittenburg
1472dcace4
ci: Remove deprecated logic from the ci module (#47062)
ci: Remove deprecated logic from the ci module

Remove the following from the ci module, schema, and tests:

- deprecated ci stack and handling of old ci config
- deprecated mirror handling logic
- support for artifacts buildcache
- support for temporary storage url
2024-10-23 12:50:55 -06:00
Todd Gamblin
f33912d707
mypy: work around typing issues with functools.partial (#47160) 2024-10-23 06:33:09 -06:00
dependabot[bot]
e785d3716e
build(deps): bump sphinx from 7.4.7 to 8.1.3 in /lib/spack/docs (#47159)
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 7.4.7 to 8.1.3.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/v8.1.3/CHANGES.rst)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v7.4.7...v8.1.3)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-23 06:18:20 -06:00
Massimiliano Culpo
ed15b73c3b
Remove spurious warning, introduced in #46992 (#47152)
fixes #47135

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-23 07:33:09 +02:00
dependabot[bot]
50aa5a7b24
build(deps): bump black from 24.8.0 to 24.10.0 in /lib/spack/docs (#47118)
Bumps [black](https://github.com/psf/black) from 24.8.0 to 24.10.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/24.8.0...24.10.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-22 18:58:31 +02:00
Harmen Stoppels
ef9bb7ebe5
spack arch: add --family --generic flags (#47078)
This allows users to do:

```
spack install ... target=$(spack arch --target --family)
spack install ... arch=$(spack arch --family)

spack install ... target=$(spack arch --target --generic)
spack install ... arch=$(spack arch --generic)
```

Deprecate `--generic-target` in favor of `--generic --target`
2024-10-22 14:13:11 +00:00
Harmen Stoppels
cbad3d464a
buildcache: recognize . and .. as paths instead of names (#47105) 2024-10-22 13:05:06 +02:00
Peter Scheibel
275d1d88f4
avoid double closing of fd in sub-processes (#47035)
Both `multiprocessing.connection.Connection.__del__` and `io.IOBase.__del__` called `os.close` on the same file descriptor. As of Python 3.13, this is an explicit warning. Ensure we close once by usef `os.fdopen(..., closefd=False)`
2024-10-21 18:44:28 +00:00
Tom Scogland
a07d42d35b
Devtools darwin (#46910)
* stacks: add a stack for devtools on darwin

After getting this whole mess building on darwin, let's keep it that
way, and maybe make it so we have some non-ML darwin binaries in spack
as well.

* reuse: false for devtools

* dtc: fix darwin dylib name and id

On mac the convention is `lib<name>.<num>.dylib`, while the makefile
creates a num suffixed one by default. The id in the file is also a
local name rather than rewritten to the full path, this fixes both
problems.

* node-js: make whereis more deterministic

* relocation(darwin): catch Mach-O load failure

The MachO library can throw an exception rather than return no headers,
this happened in an elf file in the test data of go-bootstrap.  Trying
catching the exception and moving on for now.  May also need to look
into why we're trying to rewrite an elf file.

* qemu: add darwin flags to clear out warnings

There's a build failure for qemu in CI, but it's invisible because of
the immense mass of warning output.  Explicitly specify the target macos
version and remove the extraneous unknown-warning-option flag.

* dtc: libyaml is also a link dependency

libyaml is required at runtime to run the dtc binary, lack of it caused
the ci for qemu to fail when the library wasn't found.
2024-10-21 17:32:14 +00:00
Harmen Stoppels
19ad29a690
bootstrap: handle a new edge case of binary python packages with missing python-venv (#47094)
relevant for clingo installed without gcc-runtime and python-venv, which
is done for good reasons.
2024-10-21 10:46:13 -06:00
Massimiliano Culpo
4187c57250
Fix broken spack find -u (#47102)
fixes #47101

The bug was introduced in #33495, where `spack find was not updated,
and wasn't caught by unit tests.

Now a Database can accept a custom predicate to select the installation
records. A unit test is added to prevent regressions. The weird convention
of having `any` as a default value has been replaced by the more commonly
used `None`.

Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-21 18:03:57 +02:00
Jordan Galby
e5a602c1bb
Modules suffixes config are now spec format strings (#38411) 2024-10-21 09:08:59 +02:00
Tamara Dahlgren
260b36e272
Docs: clarify include path options (#47083) 2024-10-21 07:26:18 +02:00
Harmen Stoppels
9b8c06a049
spack external find: show backtrace on error when --backtrace (#47082) 2024-10-19 15:45:59 +02:00
dependabot[bot]
011ff48f82
build(deps): bump python-levenshtein in /lib/spack/docs (#46494)
Bumps [python-levenshtein](https://github.com/rapidfuzz/python-Levenshtein) from 0.25.1 to 0.26.0.
- [Release notes](https://github.com/rapidfuzz/python-Levenshtein/releases)
- [Changelog](https://github.com/rapidfuzz/python-Levenshtein/blob/main/HISTORY.md)
- [Commits](https://github.com/rapidfuzz/python-Levenshtein/compare/v0.25.1...v0.26.0)

---
updated-dependencies:
- dependency-name: python-levenshtein
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-19 12:12:13 +02:00
Wouter Deconinck
31bdcd7dc6
rtd: bump sphinx-rtd-theme to 3.0.1 (#47002) 2024-10-19 03:29:36 +02:00
John W. Parent
31cfcafeba
Build logic fix: reorder definition of package module variables (#46992)
#44327 made sure to always run `set_package_py_globals` on all
packages before running `setup_dependent_package` for any package,
so that packages implementing the latter could depend on variables
like `spack_cc` being defined.

This ran into an undocumented dependency: `std_cmake_args` is set in
`set_package_py_globals` and makes use of `cmake_prefix_paths` (if it
is defined in the package); `py-torch`es implementation of
`cmake_prefix_paths` depends on a variable set by
`setup_dependent_package` (`python_platlib`).

This generally restores #44327, and corrects the resulting issue by
moving assignment of `std_cmake_args` to after both actions have been
run.
2024-10-18 13:36:16 -07:00
Sean Koyama
c006cb573a
implement prefix property for OneAPI compiler (#47066) 2024-10-18 11:50:12 -04:00
Harmen Stoppels
c6bfe7c6bd
fix use of traceback.format_exception (#47080)
Co-authored-by: Peter Scheibel <scheibel1@llnl.gov>
2024-10-18 14:54:21 +00:00
kwryankrattiger
34c89c0f7b
CI RESTful Configuration (#41622)
* CI: Add dynamic mapping section

* Doc: Add documentation for dynamic mapping section

* Add missing schema property

* Fixes from review

* query build fix up
* add warning output for dynamic mapping request errors

* Cleanup ci schema

* Add more protections for disabling/mitigating bad endpoints for dynamic
mapping

* Remove references to "gantry" in the docs

* Fixup rtd header

* Add unit testing for dynamic-mapping section

* Add arch to dynamic-mapping query string

* Tests and cleanup schema
2024-10-16 14:06:09 -06:00
Massimiliano Culpo
cbdc07248f
unit-tests: install.py (#47007)
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-16 15:02:52 +02:00
Harmen Stoppels
8c70912b11
Update release documentation (#46991) 2024-10-16 09:11:53 +02:00
Massimiliano Culpo
64f90c38be
unit-tests: oci/integration_test.py (#47006)
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-16 09:09:52 +02:00
psakievich
0477875667
remove concrete spec constraint from spack develop (#46911)
Remove the constraint for concrete specs and simply take the
max(version) if a version is not given. This should default to the
highest infinity version which is also the logical best guess for
doing development.

* Remove concrete verision constriant for develop, set docs

* Add unit-test

* Update lib/spack/docs/environments.rst

Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>

* Update lib/spack/spack/cmd/develop.py

Co-authored-by: Greg Becker <becker33@llnl.gov>

* Consolidate env collection in cmd

* Style

---------

Co-authored-by: kwryankrattiger <80296582+kwryankrattiger@users.noreply.github.com>
Co-authored-by: Greg Becker <becker33@llnl.gov>
2024-10-15 17:46:27 +00:00
Tobias Ribizel
834ed2f117
env depfile: generate Makefile with absolute script path (#46966)
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2024-10-15 13:52:31 +00:00
Massimiliano Culpo
2f711bda5f
Improve behavior of spack deprecate (#46917)
Signed-off-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2024-10-15 09:04:12 +02:00
kwryankrattiger
c8f61c8662
Don't require OIDC initialization for noop (#46921)
ref. https://github.com/spack/spack-infrastructure/pull/957
2024-10-14 23:39:55 -05:00