Commit Graph

7521 Commits

Author SHA1 Message Date
Harmen Stoppels
ecd14f0ad9
archive.py: fix include_parent_directories=True with path_to_name (#48570) 2025-01-20 10:21:32 +01:00
Massimiliano Culpo
783eccfbd5
jsonschema: use draft7 validator and simplify schemas (#48621) 2025-01-20 09:51:29 +01:00
Harmen Stoppels
7e41288ca6
spack_yaml.py / spec.py: use dict instead of OrderedDict (#48616)
* for config: let `syaml_dict` inherit from `dict` instead of `OrderedDict`. `syaml_dict` now only exists as a mutable wrapper for yaml related metadata.
* for spec serialization / hashing: use `dict` directly

This is possible since we only support cpython 3.6+ in which dicts are ordered.

This improves performance of hash computation a bit. For a larger spec I'm getting 9.22ms instead of 11.9ms, so 22.5% reduction in runtime.
2025-01-20 17:33:44 +09:00
Tara Drwenski
3bb375a47f
Change llvm-amdgpu to a build dependency of rocm (#48612) 2025-01-20 17:28:15 +09:00
Harmen Stoppels
962498095d
compat with mypy 1.14 (#48626)
* OSError.errno apparently is int | None, but already part of __str__ anyway so drop

* fix disambiguate_spec_from_hashes

* List[Spec].sort() complains, ignore it

* fix typing of KnownCompiler, and use it in asp.py
2025-01-18 21:16:05 +01:00
John Gouwar
9f7cff1780
Stable splice-topo order for resolving splicing (#48605)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-01-17 16:27:16 +01:00
Harmen Stoppels
bb43fa5444
spec.py: make hashing of extra_attributes order independent (#48615) 2025-01-17 13:50:36 +01:00
Massimiliano Culpo
59a71959e7
Fix wrong hash in spliced specs, and improve unit-tests (#48574)
Modifications:
* Fix a severe bug where a spliced spec has the same hash as the original build spec
* Removed CacheManager in favor of install_mockery. The latter sets up a temporary store, and removes it at the end 
   of the test.
* Deleted a couple of helper functions e.g. _has_dependencies
* Checked that SolverException is raised, rather than Exception (more specific)
* Extended, and renamed the splicing_setup fixture (now called install_specs)
* Added more specific assertions in each test

One test is currently flaky, due to some instability when sorting multiple specs. It's currently marked xfail
2025-01-15 19:43:55 +01:00
Harmen Stoppels
00e804a94b
package.py: expose types of common globals unconditionally (#48588) 2025-01-15 17:33:38 +01:00
Massimiliano Culpo
6fac041d40
Add type-hints to which and which_string (#48554) 2025-01-15 15:24:18 +01:00
Jordan Galby
c504304d39
Add version attributes up_to_1, up_to_2, up_to_3 (#38410)
Making them also available in spec format string
2025-01-15 13:07:17 +01:00
Harmen Stoppels
976f1c2198
flag_mixing.py: add missing import (#48579) 2025-01-15 10:48:34 +01:00
Greg Becker
e7c591a8b8
Deprecate Spec.concretize/Spec.concretized in favor of spack.concretize.concretize_one (#47971)
The methods spack.spec.Spec.concretize and spack.spec.Spec.concretized
are deprecated in favor of spack.concretize.concretize_one.

This will resolve a circular dependency between the spack.spec and
spack.concretize in the next Spack release.
2025-01-15 10:13:19 +01:00
jclause
335fca7049
Improve support using external modules with zsh (#48115)
* Improve support using external modules with zsh

Spack integrates with external modules by launching a python subprocess
to scrape the path from the module command. In zsh, subshells do not
inheret functions defined in the parent shell (only environment
variables). This breaks the module function in module_cmd.py.

As a workaround, source the module commands using $MODULESHOME prior to
running the module command.

* Fix formatting

* Fix flake error

* Add guard around sourcing module file

* Add improved unit testing to module src command

* Correct style

* Another attempt at style

* formatting again

* formatting again

---------

Co-authored-by: psakievich <psakiev@sandia.gov>
2025-01-14 21:29:15 -07:00
Greg Becker
2447d16e55
bugfix: associate nodes directly with specs for compiler flag reordering (#48496) 2025-01-14 10:15:57 -08:00
Harmen Stoppels
d72b371c8a
relocate_text: fix return value (#48568) 2025-01-14 13:29:06 +01:00
Todd Gamblin
d89ae7bcde
Database: make constructor safe for read-only filesystems (#48454)
`spack find` and other commands that read the DB will fail if the databse is in a
read-only location, because `FailureTracker` and `Database` can try to create their
parent directories when they are constructed, even if the DB root is read-only.

Instead, we should only write to the filesystem when needed.

- [x] don't create parent directories in constructors
- [x] have write operations create parents if needed
- [x] in tests, `chmod` databases to be read-only unless absolutely needed.

---------

Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2025-01-14 00:10:51 -08:00
Harmen Stoppels
53d1665a8b
traverse: use deque and popleft (#48353)
Avoids a quadratic time operation. In practice it is not really felt, cause it has a tiny constant.
But maybe in very large DAGs it is.
2025-01-14 00:09:57 -08:00
Cody Balos
66e8523e14
add long_spec property to get fully enumerated spec string (#48389) 2025-01-13 16:43:39 -08:00
Tamara Dahlgren
f556ba46d9
bugfix/r: git property needs to be classproperty (#48228) 2025-01-13 12:14:26 -06:00
Harmen Stoppels
064e70990d
binary_distribution.py: stop relocation old /bin/bash sbang shebang (#48502) 2025-01-13 09:30:18 +01:00
Massimiliano Culpo
8ab6f33eb6
Deprecate make, gmake and ninja globals in package API (#48450)
Instead, depends_on("gmake", type="build") should be used, which makes the global available through `setup_dependent_package`.
2025-01-11 22:43:22 +01:00
Massimiliano Culpo
dd69b646ad
Add subscript notation to packages (#48467)
This PR allows using the subscript notation directly on packages. The
intent is to reduce the boilerplate needed to retrieve package
properties from nodes other than root.
2025-01-10 19:00:51 +01:00
Harmen Stoppels
d6d8800466
executable.py: fix overlapping overload set (#48503) 2025-01-10 16:34:04 +00:00
Harmen Stoppels
d46ac9b1e4
spec.py: fix return type of concretized() (#48504) 2025-01-10 13:31:41 +00:00
Harmen Stoppels
7edb525599
binary_distribution: improve deps_to_relocate (#48484) 2025-01-10 11:32:11 +01:00
Harmen Stoppels
93cd216603
binary_distribution: stop relocating tarballs with relative rpaths (#48488) 2025-01-10 11:30:18 +01:00
Harmen Stoppels
c1d385ada2
traverse: use overload for return type when depth=True vs depth=False (#48482) 2025-01-10 09:53:28 +01:00
Massimiliano Culpo
5085f635dd
Add type hints to spack.util.executable.Executable (#48468)
* Add type-hints to `spack.util.executable.Executable`
* Add type-hint to input
* Use overload, and remove assertions at calling sites
* Bump mypy to v1.11.2 (working locally), Python to 3.13
2025-01-09 14:16:24 -08:00
dependabot[bot]
e76677cbd5
build(deps): bump pygments from 2.18.0 to 2.19.1 in /lib/spack/docs (#48431)
Bumps [pygments](https://github.com/pygments/pygments) from 2.18.0 to 2.19.1.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.18.0...2.19.1)

---
updated-dependencies:
- dependency-name: pygments
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-01-09 11:31:42 +01:00
Harmen Stoppels
c424611010
directory_layout.py: remove dependency on global spack.store.STORE (#48460) 2025-01-08 17:44:32 +01:00
Harmen Stoppels
fa9ef0ac89
config.py: deprecate config:install_path_scheme (#48463) 2025-01-08 16:02:11 +01:00
Harmen Stoppels
55d9fe20e5
warnings.showwarning: set earlier (#48462)
Co-authored-by: haampie <haampie@users.noreply.github.com>
2025-01-08 07:27:22 -07:00
Harmen Stoppels
434a8d54d4
directory_layout: types and remove kwargs (#48459) 2025-01-08 14:09:36 +01:00
Adrien Bernede
7328c64fc7
CachedCMakePackage: set CMAKE_*_RPATH with implicit and extra rpaths (#48067)
This is useful when CachedCMakePackage is used without the Spack compiler wrapper.
2025-01-08 04:18:44 -07:00
Massimiliano Culpo
4be7b98fd2
Perl build system: add a dependency on gmake (#48437) 2025-01-08 09:10:22 +01:00
Harmen Stoppels
ca8ea63796
compiler.py: deal with temporarily broken compiler (#48418)
When a compiler is temporarily broken, we cache `c_compiler_output:`
`null`. This commit ensures we retry to obtain the `c_compiler_output`.
2025-01-07 09:56:05 +01:00
Lehman Garrison
aa0ab3b38b
py-py-spy: add 0.4.0, and make rust & CargoPackage respect -j (#47883)
* py-py-spy: add 0.4.0

* py-py-spy: port from Package to CargoPackage

* CargoPackage: respect make_jobs

* rust: respect make_jobs during build and install

* spack style

* CargoBuilder: fix make_jobs syntax

* CargoBuilder: don't write to $HOME, use stage dir
2025-01-06 12:14:19 -07:00
Massimiliano Culpo
85c125a0f5
Remove spack.config.add_default_platform_scope (#48425)
This function is used only in tests, and pushes a scope to the
current global CONFIG, when on Windows, regardless of what that
CONFIG is.
2025-01-06 20:06:02 +01:00
Harmen Stoppels
4b4be2e2c2
cmd/__init__.py: pass tests in case n=1 (#48417) 2025-01-06 15:30:29 +01:00
Harmen Stoppels
79027884c7
Add spack API warning with file/line (#48416) 2025-01-06 03:43:04 -07:00
Adam J. Stewart
0a03a92e78
Avoid trailing slashes in strings (#48359) 2025-01-03 23:00:18 +01:00
Harmen Stoppels
0a070512e6
tests: remove file requirement (#48362) 2025-01-03 15:49:06 +01:00
Wouter Deconinck
e374f8fb66
build_environment: only traverse unique edges (#48301)
Wrap EnvironmentVisitor in CoverEdgesVisitor so only unique edges are enqueued.

Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
2025-01-03 12:17:26 +01:00
Todd Gamblin
f635337af8
Remove years from license headers (#48352) 2025-01-02 15:40:28 +01:00
Alec Scott
022ae8503a oxide-rs: new-package 2025-01-02 01:17:37 -08:00
Massimiliano Culpo
71512f5682
mpich: gather in a single place env modifications needed by mpich derivatives (#48276)
* mpich: gather in a single place env modifications needed by mpich derivatives

MPICH, and its derivatives, share a lot of copy/paste code to setup the
environment during the different stages of the package life-cycle.

This commit gathers the common modifications in a single place (a mixin class),
living in the Mpich package, and makes derivatives import, and reuse, it.

* Fix docs for Python < 3.13
2024-12-30 15:42:16 -07:00
dependabot[bot]
15124d7c73
build(deps): bump urllib3 from 2.2.3 to 2.3.0 in /lib/spack/docs (#48278)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.3 to 2.3.0.
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.2.3...2.3.0)

---
updated-dependencies:
- dependency-name: urllib3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-26 09:03:38 -07:00
psakievich
78bd905fe0
Ensure command_line scope is always last (#48255) 2024-12-25 14:32:31 -07:00
Harmen Stoppels
e9cdcc4af0
build caches: collect files to relocate while tarballing w/o file (#48212)
A few changes to tarball creation (for build caches):
- do not run file to distinguish binary from text
- file is slow, even when running it in a batched fashion -- it usually reads all bytes and has slow logic to categorize specific types
- we don't need a highly detailed file categorization; a crude categorization of elf, mach-o, text suffices.
detecting elf and mach-o is straightforward and cheap
- detecting utf-8 (and with that ascii) is highly accurate: false positive rate decays exponentially as file size increases. Further it's not only the most common encoding, but the most common file type in package prefixes.
iso-8859-1 is cheaply (but heuristically) detected too, and sufficiently accurate after binaries and utf-8 files are classified earlier
- remove file as a dependency of Spack in general, which makes Spack itself easier to install
- detect file type and need to relocate as part of creating the tarball, which is more cache friendly and thus faster
2024-12-24 10:53:13 -07:00