Commit Graph

7567 Commits

Author SHA1 Message Date
Harmen Stoppels
1c93fef160
spec.py: ensure == is false if true modulo precomputed dag hash (#48889) 2025-02-06 09:55:27 +01:00
Harmen Stoppels
e0e6f29584
relocate.py: don't warn about symlinks (#48904)
`relocate_links` warns when the target is absolute and not matched by
any prefix from the prefix to prefix map.

This can lead to false positives, cause the prefix to prefix map does
not contain trivial/identity entries whenever a package is installed to
its original location.

Since relocate_links is the odd one out there (we don't warn about
similar issues with rpaths, etc), just remove the warning.
2025-02-05 12:33:44 -07:00
Harmen Stoppels
4487598d60
spec.py: ensure spec.extra_attributes is {} if is null in json (#48896) 2025-02-05 17:55:53 +01:00
Harmen Stoppels
fbb688af07
spec.py: fix hash change due to None vs {} (#48854)
* Fix hash change due to None vs {}

* Enforce null for empty list of external_modules
2025-02-05 09:48:00 +01:00
psakievich
b7f556e4b4
Remove variable from cmake.py (#48824)
* Remove variable from cmake.py

#48775 left a dangling variable that was not caught in CI but by the eyes of @haampie. Restructure variable to local method.

* [@spackbot] updating style on behalf of psakievich

* Update cmake.py

* Update lib/spack/spack/build_systems/cmake.py

* Update lib/spack/spack/build_systems/cmake.py

---------

Co-authored-by: psakievich <psakievich@users.noreply.github.com>
2025-02-04 15:20:15 +01:00
Todd Gamblin
71d1901831
update pyproject.toml for ruff format (#48823)
Add ruff configuration to `pyproject.toml`.

This allows `ruff format` in the Spack repository to format all the files we care about, 
with our line length of 99, the exceptions we already put in place, and excluding things
we don't auto-format, like vendored dependencies.

Right now it'll reformat 175 or so files, but only slightly, in places where `ruff` differs from
`black`. For the most part I like the ruff format decisions better than `black`, but none of
the changes seem too severe.

This does not change `spack style` -- I figure that can come later but this at least will
let people start playing with `ruff`.

---------

Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2025-02-03 20:56:05 -08:00
Harmen Stoppels
a8f5289801
cdash: avoid build_opener (#48846) 2025-02-03 18:41:40 +01:00
Harmen Stoppels
ac635aa777
packge_base.py: remove _patches_by_hash (#48768) 2025-02-03 16:17:42 +01:00
pauleonix
8579efcadf
cuda: add v12.8 (#48708) 2025-02-03 13:49:30 +01:00
Harmen Stoppels
a6c9b55fad
Set version to v1.0.0.dev0 (#48791) 2025-02-03 01:42:32 -08:00
Todd Gamblin
608ed967e1
style: fix not in and is not (#48831)
These are some changes that `ruff check --fix` would make that the current
`spack style` also agrees with.  Make the changes now so that the `ruff`
change is less disruptive.

Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
2025-02-03 00:49:38 -08:00
Todd Gamblin
742eaa32b7
spack_yaml: use unambiguous variable name (#48832) 2025-02-03 09:07:39 +01:00
Till Ehrengruber
392a58e9be
oci/opener.py: respect system proxy settings (#48783) 2025-02-01 09:26:55 +01:00
Greg Becker
bb35a98079
env create: create copies of relative include files in envs created from manifest (#48689)
Currently, environments created from manifest files with relative includes result in broken
references to config files.

This PR modifies `spack env create` to create local copies in the new environment of any local
config files from relative paths in the environment manifest passed as an init file.

This PR does not change the behavior if the include is an absolute path or if the include is from
a relative path outside the environment directory, but it does warn about missing relative includes if
they are inside the environment directory.

Includes regression test and short blurb in docs.
2025-02-01 01:41:18 +00:00
psakievich
759518182c
Bug Fix: Better incremental check for CMake (#48775)
* Bug Fix: Better incremental check for CMake

* Fix syntax error

* Ensure match of config artifact with generator
2025-01-31 08:48:41 -06:00
Harmen Stoppels
6b13017ded
Remove unused values (#48795)
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2025-01-31 08:21:44 +01:00
Harmen Stoppels
2c51b5853f
spack.package: re-export EnvironmentModifications / Prefix (#48792) 2025-01-31 08:20:15 +01:00
Harmen Stoppels
d0cbd056a8
spack.package: wrap llnl.util.tty (#48793)
avoid import of llnl.util.tty in packages
2025-01-31 08:17:29 +01:00
Harmen Stoppels
b02dcf697d
Move from python2 compliant IOError and EnvironmentError to python3-only OSError (#48764)
* IOError -> OSError

* also do EnvironmentError
2025-01-30 09:32:57 -08:00
Tamara Dahlgren
2da8a1d1e3
Docs/bugfix: correct return for Adding flags to configure (#48434) 2025-01-29 13:04:22 -08:00
Massimiliano Culpo
3d82e5c573
Remove pipelines and images based on ppc64le (#48767) 2025-01-29 16:36:25 +01:00
Harmen Stoppels
92260b179d
package api: drop wildcard re-export (#48760)
* package api: drop wildcard re-export

To ensure package repos are forward/backward compatibility with Spack,
we should explicitly export all symbols we want to expose in the public
package API, and drop `from spack.something import *` because
removal/addition to the public API will go unnoticed.

Also `llnl.util.filesystem` has some methods that shouldn't be exposed
in the package API, so better to enumerate a subset explicitly.

* remove flatten_dependencies / install_dependency_symlinks
2025-01-29 15:00:39 +01:00
Massimiliano Culpo
196c912b8a
Deprecate frontend/backend os/target (#47756) 2025-01-29 13:22:51 +01:00
Massimiliano Culpo
40a1da4a73
spec.py: fix ArchSpec.intersects (#48741)
fixes a bug where `x86_64:` and `ppc64le:` intersected, and x86_64: and :haswell did not.
2025-01-28 16:46:09 +01:00
Harmen Stoppels
c054cb818d
import os.path -> os (#48709) 2025-01-28 09:45:43 +01:00
Dom Heinzeller
22ba366e85
Fix creating a bootstrap mirrors (#48252)
Regressed in #47126 

Spack was not interpreting mirrors using relative path with respect to the
metadata directory.

---------

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2025-01-27 22:37:29 +00:00
Massimiliano Culpo
13558269b5
Fix a few minor issues in tests (#48735)
* test_no_matching_compiler_specs: does not need mock_low_high_config,
  since mutable_config is already used at class level
* bindist.py: setup a configuration that doesn't super-impose builtin.mock
  over builtin
* builder.py: use a mock configuration for the tests
2025-01-27 23:36:27 +01:00
Joe
c764400338
Allowing environment variables to be set in a spack.yaml (#47587)
This adds a new configuration section called `env_vars:` that can be set in an environment.

It looks very similar to the existing `environment:` section that can be added to `modules.yaml`,
but it is global for an entire spack environment. It's called `env_vars:` to deconflate it with spack
environments (the term was too overloaded).

The syntax looks like this:

```yaml
spack:
  specs:
    - cmake%gcc
  env_vars:
    set:
      ENVAR_SET_IN_ENV_LOAD: "True"
```

Any of our standard environment modifications can be added to the `env_vars` section, e.g.
`prepend_path:`, `unset:`, `append_path:`, etc.  Operations in `env_vars:` are performed
on `spack env activate` and undone on `spack env deactivate`.
2025-01-27 12:20:22 -08:00
Alec Scott
ebc9f03dda
Go build system: reduce resulting installation sizes (#47943)
* Reduce the size of outputted go built binaries
* Remove unused import from go package
* go: remove comment from setup dependents build env
* Add back missing imports after rebase
2025-01-27 11:32:56 -08:00
Alec Scott
8ac0bd2825
GoPackage: respect -j concurrency (#48421) 2025-01-27 12:58:29 -05:00
Tara Drwenski
cc9e0137df
Initialize deque with path string (#48734) 2025-01-27 17:42:29 +01:00
Harmen Stoppels
c6202842ed
Improve error handling in urlopen / socket read (#48707)
* Backward compat with Python 3.9 for socket.timeout
* Forward compat with Python [unknown] as HTTPResponse.geturl is deprecated
* Catch timeout etc from .read()
* Some minor simplifications: json.load(...) takes file object in binary mode.
* Fix CDash code which does error handling wrong: non-2XX responses raise.
2025-01-27 16:59:05 +01:00
Greg Becker
0ec00a9c9a
rewiring.py: eliminate code duplication from bindist (#48723) 2025-01-27 10:09:04 +01:00
psakievich
5e3020ad02
filesystem.py: fix recursive_mtime_greater_than (#48718) 2025-01-26 08:29:29 +01:00
Harmen Stoppels
25a3e8ba59
Remove unused Tokenizer.full_match (#48650) 2025-01-24 15:53:42 +01:00
Juan Miguel Carceller
f95e27a159
SIPPackage: add gmake as a build dependency to be able to build py-pyqt5 (#48660) 2025-01-23 15:51:59 +01:00
Massimiliano Culpo
2db654bf5a
Define the DB index file name in a single place (#48681) 2025-01-23 12:17:34 +01:00
Harmen Stoppels
9992b563db
Improve spack.build_systems typing (#48590) 2025-01-23 12:15:43 +01:00
Harmen Stoppels
daba1a805e
build_environment.py: clear CONFIG_SITE for configure scripts (#48690)
Should be sufficient to set CONFIG_SITE to /dev/null before running configure to 
prevent any config site file to be loaded, which currently causes non-determinism in builds.

Setting the variable CONFIG_SITE prevents configure files to check $prefix/share/config.site, 
$prefix/etc/config.site, $ac_default_prefix/share/config.site, $ac_default_prefix/etc/config.site 
so we don't have to patch a block of code.
2025-01-23 11:45:52 +01:00
Tara Drwenski
518572e710
Use gcc toolchain when using hip and gcc host compiler (#48632) 2025-01-22 23:37:24 -08:00
jnhealy2
aba0a740c2
Address quoting issue impacting dev_paths containing @ symbols (#48555)
* Address quoting issue that casuses dev_paths containing @ symbols to parse as versions

* Fix additional location were dev_path was imporperly constructed

The dev_path must be quoted to avoid parsing issues when
paths contain '@' symbols. Also add tests to catch
regression of this behavior

* Format to fix line length

* fix failing tests

* Fix whitespace error

* Add binary_compatibility fixture to test

* Change string formatting to avoid multiline f string

* Update lib/spack/spack/test/concretization/core.py

* Add tmate debug session

* Revert "Add tmate debug session"

This reverts commit 24e2f77e3c.

* Move test and refactor to use env methods

* Update lib/spack/spack/test/cmd/develop.py

---------

Co-authored-by: psakievich <psakiev@sandia.gov>
2025-01-22 18:03:13 +01:00
Harmen Stoppels
0fe8e763c3
bootstrap: do not show disabled sources as errors if enabled sources fail (#48676) 2025-01-22 17:02:23 +01:00
Harmen Stoppels
0e2d261b7e
bootstrap/status.py: remove make (#48677) 2025-01-22 17:00:25 +01:00
Harmen Stoppels
87a83db623
autotools.py: set lt_cv_apple_cc_single_mod=yes (#48671)
Since macOS 15 `ld -single_module` warns with a deprecation message,
which makes configure scripts believe the flag is unsupported. That
in turn triggers a code path where `archive_cmds` is set to

```
$CC -r -keep_private_externs -nostdlib ... -dynamiclib
```

instead of just

```
$CC -dynamiclib ...
```

This code path was meant to trigger only on ancient macOS <= 14.4 where
libtool had to add `-single_module`, which is the default since macos
14.4, and is now apparently deprecated because the flag is a no-op for
more than 15 years.

The wrong `archive_cmds` causes actual problems combined with a bug in
OpenMPI's compiler wrapper (`CC=mpicc`), which appends `-rpath` flags,
which cause an error when combined with the `-r` flag added by the
autotools.

Spack's compiler wrapper doesn't do this, but it's likely there are
other compiler wrappers out there that are not aware that `-r` and
`-rpath` cannot be combined.

The fix is to change defaults: `lt_cv_apple_cc_single_mod=yes`.
2025-01-22 11:29:50 +01:00
Harmen Stoppels
70df460fa7
PackageBase.detect_dev_src_change: speed up (#48618) 2025-01-21 16:48:37 +01:00
Harmen Stoppels
31a1b2fd6c
relocate.py, binary_distribution.py: cleanup (#48651) 2025-01-21 15:45:08 +01:00
Harmen Stoppels
ecd14f0ad9
archive.py: fix include_parent_directories=True with path_to_name (#48570) 2025-01-20 10:21:32 +01:00
Massimiliano Culpo
783eccfbd5
jsonschema: use draft7 validator and simplify schemas (#48621) 2025-01-20 09:51:29 +01:00
Harmen Stoppels
7e41288ca6
spack_yaml.py / spec.py: use dict instead of OrderedDict (#48616)
* for config: let `syaml_dict` inherit from `dict` instead of `OrderedDict`. `syaml_dict` now only exists as a mutable wrapper for yaml related metadata.
* for spec serialization / hashing: use `dict` directly

This is possible since we only support cpython 3.6+ in which dicts are ordered.

This improves performance of hash computation a bit. For a larger spec I'm getting 9.22ms instead of 11.9ms, so 22.5% reduction in runtime.
2025-01-20 17:33:44 +09:00
Tara Drwenski
3bb375a47f
Change llvm-amdgpu to a build dependency of rocm (#48612) 2025-01-20 17:28:15 +09:00