Compare commits

..

1 Commits

Author SHA1 Message Date
Todd Gamblin
a438076473 Add a new disinherit directive for extending packages
When you inherit from a package in Spack, you inherit all the metadata from its
directives, including `version`, `provides`, `depends_on`, `conflicts`, etc.

In some cases, you may not want this metadata. This PR adds a new `disinherit` directive
that allows you to throw out things inherited from the base class.  For example:

```python
from spack.pkg.builtin.mpich import Mpich

class MyMpich(Mpich):
    disinherit("versions")    # don't inherit any versions from builtin Mpich
    version("5.0", "08721a102fefcea2ae4add8c9cc548df77e9224f5385ad0872a9150fdd26a415")
    version("6.0", "9cc39dd33dd4227bb82301d285437588d705290846d22ab6b8791c7e631ce385")
```

Without the `disinherit("versions")` directive, this package would have versions `5.0`,
`6.0`, *and* anything inherited from `Mpich`. With it, this package has only versions
`5.0` and `6.0`.

You can `disinherit` any of: `conflicts`, `dependencies`, `extendees`, `patches`,
`provided`, `resources`, `variants`, or `versions`.

- [x] add new `disinherit directive`
- [x] Two packages were modifying their `versions` dictionary in their constructors to
      achieve this, but this causes `spack url stats` to fail with a concurrent
      modification exception as it iterates over all packages. Fixed these to use
      `disinherit` instead.
- [x] Update documentation
- [x] Add tests
2023-01-19 17:06:36 -06:00
7955 changed files with 17423 additions and 24247 deletions

View File

@@ -1,5 +1,3 @@
# .git-blame-ignore-revs
# Formatted entire codebase with black 23
603569e321013a1a63a637813c94c2834d0a0023
# Formatted entire codebase with black 22
# Formatted entire codebase with black
f52f6e99dbf1131886a80112b8c79dfc414afb7c

1
.gitattributes vendored
View File

@@ -1,4 +1,3 @@
*.py diff=python
*.lp linguist-language=Prolog
lib/spack/external/* linguist-vendored
*.bat text eol=crlf

View File

@@ -89,7 +89,7 @@ jobs:
uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # @v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@f03ac48505955848960e80bbb68046aa35c7b9e7 # @v1
uses: docker/setup-buildx-action@8c0edbc76e98fa90f69d9a2c020dcb50019dc325 # @v1
- name: Log in to GitHub Container Registry
uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a # @v1
@@ -106,7 +106,7 @@ jobs:
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build & Deploy ${{ matrix.dockerfile[0] }}
uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671 # @v2
uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3 # @v2
with:
context: dockerfiles/${{ matrix.dockerfile[0] }}
platforms: ${{ matrix.dockerfile[1] }}

View File

@@ -44,7 +44,7 @@ jobs:
cache: 'pip'
- name: Install Python packages
run: |
python3 -m pip install --upgrade pip six setuptools types-six black==23.1.0 mypy isort clingo flake8
python3 -m pip install --upgrade pip six setuptools types-six black mypy isort clingo flake8
- name: Setup git configuration
run: |
# Need this for the git tests to succeed.

View File

@@ -1,28 +1,3 @@
# v0.19.1 (2023-02-07)
### Spack Bugfixes
* `buildcache create`: make "file exists" less verbose (#35019)
* `spack mirror create`: don't change paths to urls (#34992)
* Improve error message for requirements (#33988)
* uninstall: fix accidental cubic complexity (#34005)
* scons: fix signature for `install_args` (#34481)
* Fix `combine_phase_logs` text encoding issues (#34657)
* Use a module-like object to propagate changes in the MRO, when setting build env (#34059)
* PackageBase should not define builder legacy attributes (#33942)
* Forward lookup of the "run_tests" attribute (#34531)
* Bugfix for timers (#33917, #33900)
* Fix path handling in prefix inspections (#35318)
* Fix libtool filter for Fujitsu compilers (#34916)
* Bug fix for duplicate rpath errors on macOS when creating build caches (#34375)
* FileCache: delete the new cache file on exception (#34623)
* Propagate exceptions from Spack python console (#34547)
* Tests: Fix a bug/typo in a `config_values.py` fixture (#33886)
* Various CI fixes (#33953, #34560, #34560, #34828)
* Docs: remove monitors and analyzers, typos (#34358, #33926)
* bump release version for tutorial command (#33859)
# v0.19.0 (2022-11-11)
`v0.19.0` is a major feature release.

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2013-2023 LLNS, LLC and other Spack Project Developers.
Copyright (c) 2013-2022 LLNS, LLC and other Spack Project Developers.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,6 +1,6 @@
#!/bin/sh
#
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# sbang project developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,7 +1,7 @@
#!/bin/sh
# -*- python -*-
#
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,6 +1,6 @@
#!/bin/sh
#
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -72,7 +72,6 @@ config:
root: $TMP_DIR/install
misc_cache: $$user_cache_path/cache
source_cache: $$user_cache_path/source
environments_root: $TMP_DIR/envs
EOF
cat >"$SPACK_USER_CONFIG_PATH/bootstrap.yaml" <<EOF
bootstrap:

View File

@@ -1,4 +1,4 @@
:: Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
:: Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
:: Spack Project Developers. See the top-level COPYRIGHT file for details.
::
:: SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -83,16 +83,6 @@ if defined _sp_flags (
exit /B 0
)
)
if not defined _sp_subcommand (
if not defined _sp_args (
if not defined _sp_flags (
python "%spack%" --help
exit /B 0
)
)
)
:: pass parsed variables outside of local scope. Need to do
:: this because delayedexpansion can only be set by setlocal
echo %_sp_flags%>flags
@@ -102,24 +92,24 @@ endlocal
set /p _sp_subcommand=<subcmd
set /p _sp_flags=<flags
set /p _sp_args=<args
if "%_sp_subcommand%"=="ECHO is off." (set "_sp_subcommand=")
if "%_sp_subcommand%"=="ECHO is on." (set "_sp_subcommand=")
if "%_sp_flags%"=="ECHO is off." (set "_sp_flags=")
if "%_sp_flags%"=="ECHO is on." (set "_sp_flags=")
if "%_sp_args%"=="ECHO is off." (set "_sp_args=")
if "%_sp_args%"=="ECHO is on." (set "_sp_args=")
set str_subcommand=%_sp_subcommand:"='%
set str_flags=%_sp_flags:"='%
set str_args=%_sp_args:"='%
if "%str_subcommand%"=="ECHO is off." (set "_sp_subcommand=")
if "%str_flags%"=="ECHO is off." (set "_sp_flags=")
if "%str_args%"=="ECHO is off." (set "_sp_args=")
del subcmd
del flags
del args
:: Filter out some commands. For any others, just run the command.
if %_sp_subcommand% == "cd" (
if "%_sp_subcommand%" == "cd" (
goto :case_cd
) else if %_sp_subcommand% == "env" (
) else if "%_sp_subcommand%" == "env" (
goto :case_env
) else if %_sp_subcommand% == "load" (
) else if "%_sp_subcommand%" == "load" (
goto :case_load
) else if %_sp_subcommand% == "unload" (
) else if "%_sp_subcommand%" == "unload" (
goto :case_load
) else (
goto :default_case
@@ -153,21 +143,19 @@ goto :end_switch
:: If no args or args contain --bat or -h/--help: just execute.
if NOT defined _sp_args (
goto :default_case
)
set args_no_quote=%_sp_args:"=%
if NOT "%args_no_quote%"=="%args_no_quote:--help=%" (
)else if NOT "%_sp_args%"=="%_sp_args:--help=%" (
goto :default_case
) else if NOT "%args_no_quote%"=="%args_no_quote: -h=%" (
) else if NOT "%_sp_args%"=="%_sp_args: -h=%" (
goto :default_case
) else if NOT "%args_no_quote%"=="%args_no_quote:--bat=%" (
) else if NOT "%_sp_args%"=="%_sp_args:--bat=%" (
goto :default_case
) else if NOT "%args_no_quote%"=="%args_no_quote:deactivate=%" (
) else if NOT "%_sp_args%"=="%_sp_args:deactivate=%" (
for /f "tokens=* USEBACKQ" %%I in (
`call python %spack% %_sp_flags% env deactivate --bat %args_no_quote:deactivate=%`
`call python "%spack%" %_sp_flags% env deactivate --bat %_sp_args:deactivate=%`
) do %%I
) else if NOT "%args_no_quote%"=="%args_no_quote:activate=%" (
) else if NOT "%_sp_args%"=="%_sp_args:activate=%" (
for /f "tokens=* USEBACKQ" %%I in (
`python %spack% %_sp_flags% env activate --bat %args_no_quote:activate=%`
`call python "%spack%" %_sp_flags% env activate --bat %_sp_args:activate=%`
) do %%I
) else (
goto :default_case
@@ -226,10 +214,10 @@ for %%Z in ("%_pa_new_path%") do if EXIST %%~sZ\NUL (
exit /b 0
:: set module system roots
:_sp_multi_pathadd
:_sp_multi_pathadd
for %%I in (%~2) do (
for %%Z in (%_sp_compatible_sys_types%) do (
:pathadd "%~1" "%%I\%%Z"
)
)
exit /B %ERRORLEVEL%
exit /B %ERRORLEVEL%

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -81,10 +81,6 @@ config:
source_cache: $spack/var/spack/cache
## Directory where spack managed environments are created and stored
# environments_root: $spack/var/spack/environments
# Cache directory for miscellaneous files, like the package index.
# This can be purged with `spack clean --misc-cache`
misc_cache: $user_cache_path/cache
@@ -185,7 +181,7 @@ config:
# when Spack needs to manage its own package metadata and all operations are
# expected to complete within the default time limit. The timeout should
# therefore generally be left untouched.
db_lock_timeout: 60
db_lock_timeout: 3
# How long to wait when attempting to modify a package (e.g. to install it).

View File

@@ -5,4 +5,3 @@ llnl*.rst
_build
.spack-env
spack.lock
_spack_root

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -366,7 +366,7 @@ If the ``pyproject.toml`` lists ``mesonpy`` as the ``build-backend``,
it uses the meson build system. Meson uses the default
``pyproject.toml`` keys to list dependencies.
See https://meson-python.readthedocs.io/en/latest/tutorials/introduction.html
See https://meson-python.readthedocs.io/en/latest/usage/start.html
for more information.
"""

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -58,7 +58,9 @@ Testing
``WafPackage`` also provides ``test`` and ``installtest`` methods,
which are run after the ``build`` and ``install`` phases, respectively.
By default, these phases do nothing, but you can override them to
run package-specific unit tests.
run package-specific unit tests. For example, the
`py-py2cairo <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/py-py2cairo/package.py>`_
package uses:
.. code-block:: python

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -89,7 +89,6 @@
# Enable todo items
todo_include_todos = True
#
# Disable duplicate cross-reference warnings.
#
@@ -164,7 +163,7 @@ def setup(sphinx):
# General information about the project.
project = "Spack"
copyright = "2013-2023, Lawrence Livermore National Laboratory."
copyright = "2013-2021, Lawrence Livermore National Laboratory."
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
@@ -354,7 +353,9 @@ class SpackStyle(DefaultStyle):
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [("index", "Spack.tex", "Spack Documentation", "Todd Gamblin", "manual")]
latex_documents = [
("index", "Spack.tex", "Spack Documentation", "Todd Gamblin", "manual"),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
@@ -401,7 +402,7 @@ class SpackStyle(DefaultStyle):
"Spack",
"One line description of project.",
"Miscellaneous",
)
),
]
# Documents to append as an appendix to all manuals.
@@ -417,4 +418,6 @@ class SpackStyle(DefaultStyle):
# -- Extension configuration -------------------------------------------------
# sphinx.ext.intersphinx
intersphinx_mapping = {"python": ("https://docs.python.org/3", None)}
intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
}

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -222,7 +222,7 @@ and location. (See the *Configuration settings* section of ``man
ccache`` to learn more about the default settings and how to change
them). Please note that we currently disable ccache's ``hash_dir``
feature to avoid an issue with the stage directory (see
https://github.com/spack/spack/pull/3761#issuecomment-294352232).
https://github.com/LLNL/spack/pull/3761#issuecomment-294352232).
-----------------------
``shared_linking:type``

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -118,7 +118,7 @@ make another change, test that change, etc. We use `pytest
<http://pytest.org/>`_ as our tests framework, and these types of
arguments are just passed to the ``pytest`` command underneath. See `the
pytest docs
<https://doc.pytest.org/en/latest/how-to/usage.html#specifying-which-tests-to-run>`_
<http://doc.pytest.org/en/latest/usage.html#specifying-tests-selecting-tests>`_
for more details on test selection syntax.
``spack unit-test`` has a few special options that can help you
@@ -147,7 +147,7 @@ you want to know about. For example, to see just the tests in
You can also combine any of these options with a ``pytest`` keyword
search. See the `pytest usage docs
<https://doc.pytest.org/en/latest/how-to/usage.html#specifying-which-tests-to-run>`_
<https://docs.pytest.org/en/stable/usage.html#specifying-tests-selecting-tests>`_:
for more details on test selection syntax. For example, to see the names of all tests that have "spec"
or "concretize" somewhere in their names:

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -58,9 +58,9 @@ Using Environments
Here we follow a typical use case of creating, concretizing,
installing and loading an environment.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Creating a managed Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Creating a named Environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An environment is created by:
@@ -72,8 +72,7 @@ Spack then creates the directory ``var/spack/environments/myenv``.
.. note::
All managed environments by default are stored in the ``var/spack/environments`` folder.
This location can be changed by setting the ``environments_root`` variable in ``config.yaml``.
All named environments are stored in the ``var/spack/environments`` folder.
In the ``var/spack/environments/myenv`` directory, Spack creates the
file ``spack.yaml`` and the hidden directory ``.spack-env``.
@@ -1040,7 +1039,7 @@ gets installed and is available for use in the ``env`` target.
$(SPACK) -e . concretize -f
env.mk: spack.lock
$(SPACK) -e . env depfile -o $@ --make-prefix spack
$(SPACK) -e . env depfile -o $@ --make-target-prefix spack
env: spack/env
$(info Environment installed!)
@@ -1063,9 +1062,9 @@ the include is conditional.
.. note::
When including generated ``Makefile``\s, it is important to use
the ``--make-prefix`` flag and use the non-phony target
``<prefix>/env`` as prerequisite, instead of the phony target
``<prefix>/all``.
the ``--make-target-prefix`` flag and use the non-phony target
``<target-prefix>/env`` as prerequisite, instead of the phony target
``<target-prefix>/all``.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Building a subset of the environment
@@ -1090,52 +1089,4 @@ output (``spack install --verbose``) while its dependencies are installed silent
$ make -j16 install-deps/python-3.11.0-<hash> SPACK_INSTALL_FLAGS=--show-log-on-error
# Install the root spec with verbose output.
$ make -j16 install/python-3.11.0-<hash> SPACK_INSTALL_FLAGS=--verbose
^^^^^^^^^^^^^^^^^^^^^^^^^
Adding post-install hooks
^^^^^^^^^^^^^^^^^^^^^^^^^
Another advanced use-case of generated ``Makefile``\s is running a post-install
command for each package. These "hooks" could be anything from printing a
post-install message, running tests, or pushing just-built binaries to a buildcache.
This can be accomplished through the generated ``[<prefix>/]SPACK_PACKAGE_IDS``
variable. Assuming we have an active and concrete environment, we generate the
associated ``Makefile`` with a prefix ``example``:
.. code:: console
$ spack env depfile -o env.mk --make-prefix example
And we now include it in a different ``Makefile``, in which we create a target
``example/push/%`` with ``%`` referring to a package identifier. This target
depends on the particular package installation. In this target we automatically
have the target-specific ``HASH`` and ``SPEC`` variables at our disposal. They
are respectively the spec hash (excluding leading ``/``), and a human-readable spec.
Finally, we have an entrypoint target ``push`` that will update the buildcache
index once every package is pushed. Note how this target uses the generated
``example/SPACK_PACKAGE_IDS`` variable to define its prerequisites.
.. code:: Makefile
SPACK ?= spack
BUILDCACHE_DIR = $(CURDIR)/tarballs
.PHONY: all
all: push
include env.mk
example/push/%: example/install/%
@mkdir -p $(dir $@)
$(info About to push $(SPEC) to a buildcache)
$(SPACK) -e . buildcache create --allow-root --only=package --directory $(BUILDCACHE_DIR) /$(HASH)
@touch $@
push: $(addprefix example/push/,$(example/SPACK_PACKAGE_IDS))
$(info Updating the buildcache index)
$(SPACK) -e . buildcache update-index --directory $(BUILDCACHE_DIR)
$(info Done!)
@touch $@
$ make -j16 install/python-3.11.0-<hash> SPACK_INSTALL_FLAGS=--verbose

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -116,7 +116,7 @@ creates a simple python file:
# FIXME: Add a list of GitHub accounts to
# notify when the package is updated.
# maintainers("github_user1", "github_user2")
# maintainers = ["github_user1", "github_user2"]
version("0.8.13", sha256="591a9b4ec81c1f2042a97aa60564e0cb79d041c52faa7416acb38bc95bd2c76d")

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -21,7 +21,7 @@ be present on the machine where Spack is run:
:header-rows: 1
These requirements can be easily installed on most modern Linux systems;
on macOS, the Command Line Tools package is required, and a full XCode suite
on macOS, the Command Line Tools package is required, and a full XCode suite
may be necessary for some packages such as Qt and apple-gl. Spack is designed
to run on HPC platforms like Cray. Not all packages should be expected
to work on all platforms.
@@ -1506,7 +1506,7 @@ Spack On Windows
Windows support for Spack is currently under development. While this work is still in an early stage,
it is currently possible to set up Spack and perform a few operations on Windows. This section will guide
you through the steps needed to install Spack and start running it on a fresh Windows machine.
you through the steps needed to install Spack and start running it on a fresh Windows machine.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Step 1: Install prerequisites
@@ -1516,7 +1516,7 @@ To use Spack on Windows, you will need the following packages:
Required:
* Microsoft Visual Studio
* Python
* Python
* Git
Optional:
@@ -1547,8 +1547,8 @@ Intel Fortran
"""""""""""""
For Fortran-based packages on Windows, we strongly recommend Intel's oneAPI Fortran compilers.
The suite is free to download from Intel's website, located at
https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/fortran-compiler.html.
The suite is free to download from Intel's website, located at
https://software.intel.com/content/www/us/en/develop/tools/oneapi/components/fortran-compiler.html#gs.70t5tw.
The executable of choice for Spack will be Intel's Beta Compiler, ifx, which supports the classic
compiler's (ifort's) frontend and runtime libraries by using LLVM.
@@ -1700,15 +1700,29 @@ If in the previous step, you did not have CMake or Ninja installed, running the
Windows Compatible Packages
"""""""""""""""""""""""""""
Not all spack packages currently have Windows support. Some are inherently incompatible with the
platform, and others simply have yet to be ported. To view the current set of packages with Windows
support, the list command should be used via `spack list -t windows`. If there's a package you'd like
to install on Windows but is not in that list, feel free to reach out to request the port or contribute
the port yourself.
Many Spack packages are not currently compatible with Windows, due to Unix
dependencies or incompatible build tools like autoconf. Here are several
packages known to work on Windows:
* abseil-cpp
* bzip2
* clingo
* cpuinfo
* cmake
* hdf5
* glm
* nasm
* netlib-lapack (requires Intel Fortran)
* ninja
* openssl
* perl
* python
* ruby
* wrf
* zlib
.. note::
This is by no means a comprehensive list, some packages may have ports that were not tagged
while others may just work out of the box on Windows and have not been tagged as such.
This is by no means a comprehensive list
^^^^^^^^^^^^^^
For developers
@@ -1720,4 +1734,3 @@ Instructions for creating the installer are at
https://github.com/spack/spack/blob/develop/lib/spack/spack/cmd/installer/README.md
Alternatively a pre-built copy of the Windows installer is available as an artifact of Spack's Windows CI
available at each run of the CI on develop or any PR.

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -234,7 +234,7 @@ generates a boilerplate template for your package, and opens up the new
.. code-block:: python
:linenos:
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -268,7 +268,7 @@ generates a boilerplate template for your package, and opens up the new
# FIXME: Add a list of GitHub accounts to
# notify when the package is updated.
# maintainers("github_user1", "github_user2")
# maintainers = ["github_user1", "github_user2"]
version("6.2.1", sha256="eae9326beb4158c386e39a356818031bd28f3124cf915f8c5b1dc4c7a36b4d7c")
@@ -319,8 +319,14 @@ The rest of the tasks you need to do are as follows:
#. Add a comma-separated list of maintainers.
Add a list of Github accounts of people who want to be notified
any time the package is modified. See :ref:`package_maintainers`.
The ``maintainers`` field is a list of GitHub accounts of people
who want to be notified any time the package is modified. When a
pull request is submitted that updates the package, these people
will be requested to review the PR. This is useful for developers
who maintain a Spack package for their own software, as well as
users who rely on a piece of software and want to ensure that the
package doesn't break. It also gives users a list of people to
contact for help when someone reports a build error with the package.
#. Add ``depends_on()`` calls for the package's dependencies.
@@ -491,31 +497,6 @@ some examples:
In general, you won't have to remember this naming convention because
:ref:`cmd-spack-create` and :ref:`cmd-spack-edit` handle the details for you.
.. _package_maintainers:
-----------
Maintainers
-----------
Each package in Spack may have one or more maintainers, i.e. one or more
GitHub accounts of people who want to be notified any time the package is
modified.
When a pull request is submitted that updates the package, these people will
be requested to review the PR. This is useful for developers who maintain a
Spack package for their own software, as well as users who rely on a piece of
software and want to ensure that the package doesn't break. It also gives users
a list of people to contact for help when someone reports a build error with
the package.
To add maintainers to a package, simply declare them with the ``maintainers`` directive:
.. code-block:: python
maintainers("user1", "user2")
The list of maintainers is additive, and includes all the accounts eventually declared in base classes.
-----------------
Trusted Downloads
-----------------
@@ -1392,7 +1373,7 @@ Go
^^
Go isn't a VCS, it is a programming language with a builtin command,
`go get <https://pkg.go.dev/cmd/go#hdr-Add_dependencies_to_current_module_and_install_them>`_,
`go get <https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies>`_,
that fetches packages and their dependencies automatically.
The destination directory will be the standard stage source path.
@@ -2117,7 +2098,7 @@ dynamic loader where to find its dependencies at runtime. You may be
familiar with `LD_LIBRARY_PATH
<http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html>`_
on Linux or `DYLD_LIBRARY_PATH
<https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/dyld.3.html>`_
<https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html>`_
on Mac OS X. RPATH is similar to these paths, in that it tells
the loader where to find libraries. Unlike them, it is embedded in
the binary and not set in each user's environment.
@@ -3623,6 +3604,70 @@ In the example above ``Cp2k`` inherits all the conflicts and variants that ``Cud
.. _install-environment:
--------------------------------
Package Inheritance
--------------------------------
Spack packages are Python classes, and you can use inheritance with them just as you can
with any Python class. This is common when you have your own package :ref:`repository
<repositories>` with packages that extend Spack's ``builtin`` packages.
You can extend a ``builtin`` package like this:
.. code-block:: python
from spack.pkg.builtin.mpich import Mpich
class MyPackage(Mpich):
version("1.0", "0209444070d9c8af9b62c94095a217e3bc6843692d1e3fdc1ff5371e03aac47c")
version("2.0", "5dda192154047d6296ba14a4ab2d869c6926fd7f44dce8ce94f63aae2e359c5b")
Every repository registered with Spack ends up in a submodule of ``spack.pkg`` with a
name corresponding to its :ref:`namespace <namespaces>`. So, if you have a different
repository with namespace ``myrepo`` you want to import packages from , you might write:
.. code-block:: python
from spack.pkg.myrepo.my_package import MyPackage
class NewPackage(MyPackage):
version("3.0", "08721a102fefcea2ae4add8c9cc548df77e9224f5385ad0872a9150fdd26a415")
version("4.0", "9cc39dd33dd4227bb82301d285437588d705290846d22ab6b8791c7e631ce385")
^^^^^^^^^^^^^^^^^^^^^^^^
``disinherit``
^^^^^^^^^^^^^^^^^^^^^^^^
When you inherit from a package in Spack, you inherit all the metadata from its
directives, including ``version``, ``provides``, ``depends_on``, ``conflicts``, etc. For
example, ``NewPackage`` above will have four versions: ``1.0`` and ``2.0`` inherited
from ``MyPackage``, as well as, ``3.0``, and ``4.0`` defined in ``NewPackage``.
If you do not want your package to define all the same things as its base class, you can
use the ``disinherit`` directive to start fresh in your subclass:
.. code-block:: python
from spack.pkg.myrepo.my_package import MyPackage
class NewerPackage(MyPackage):
disinherit("versions") # don't inherit any versions from MyPackage
version("5.0", "08721a102fefcea2ae4add8c9cc548df77e9224f5385ad0872a9150fdd26a415")
version("6.0", "9cc39dd33dd4227bb82301d285437588d705290846d22ab6b8791c7e631ce385")
Now, ``NewerPackage`` will have **only** versions ``5.0`` and ``6.0``, and will not
inherit ``1.0`` or ``2.0`` from ``MyPackage``. You can ``disinherit`` many different
properties from base packages. The full list of options is:
* ``conflicts``
* ``dependencies``
* ``extendees``
* ``patches``
* ``provided``
* ``resources``
* ``variants``
* ``versions``
-----------------------
The build environment
-----------------------

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
.. Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
.. Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
Spack Project Developers. See the top-level COPYRIGHT file for details.
SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -91,6 +91,8 @@ packages and use the first valid file:
to eventually support URLs in ``repos.yaml``, so that you can easily
point to remote package repositories, but that is not yet implemented.
.. _namespaces:
---------------------
Namespaces
---------------------
@@ -426,36 +428,3 @@ By path:
$ spack repo list
==> 1 package repository.
builtin ~/spack/var/spack/repos/builtin
--------------------------------
Repo namespaces and Python
--------------------------------
You may have noticed that namespace notation for repositories is similar
to the notation for namespaces in Python. As it turns out, you *can*
treat Spack repositories like Python packages; this is how they are
implemented.
You could, for example, extend a ``builtin`` package in your own
repository:
.. code-block:: python
from spack.pkg.builtin.mpich import Mpich
class MyPackage(Mpich):
...
Spack repo namespaces are actually Python namespaces tacked on under
``spack.pkg``. The search semantics of ``repos.yaml`` are actually
implemented using Python's built-in `sys.path
<https://docs.python.org/2/library/sys.html#sys.path>`_ search. The
:py:mod:`spack.repo` module implements a custom `Python importer
<https://docs.python.org/2/library/imp.html>`_.
.. warning::
The mechanism for extending packages is not yet extensively tested,
and extending packages across repositories imposes inter-repo
dependencies, which may be hard to manage. Use this feature at your
own risk, but let us know if you have a use case for it.

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

2
lib/spack/env/cc vendored
View File

@@ -1,7 +1,7 @@
#!/bin/sh -f
# shellcheck disable=SC2034 # evals in this script fool shellcheck
#
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -23,7 +23,7 @@
from llnl.util.lang import dedupe, memoized
from llnl.util.symlink import islink, symlink
from spack.util.executable import Executable, which
from spack.util.executable import CommandNotFoundError, Executable, which
from spack.util.path import path_to_os_path, system_path_filter
is_windows = _platform == "win32"
@@ -84,74 +84,6 @@
"visit_directory_tree",
]
if sys.version_info < (3, 7, 4):
# monkeypatch shutil.copystat to fix PermissionError when copying read-only
# files on Lustre when using Python < 3.7.4
def copystat(src, dst, follow_symlinks=True):
"""Copy file metadata
Copy the permission bits, last access time, last modification time, and
flags from `src` to `dst`. On Linux, copystat() also copies the "extended
attributes" where possible. The file contents, owner, and group are
unaffected. `src` and `dst` are path names given as strings.
If the optional flag `follow_symlinks` is not set, symlinks aren't
followed if and only if both `src` and `dst` are symlinks.
"""
def _nop(args, ns=None, follow_symlinks=None):
pass
# follow symlinks (aka don't not follow symlinks)
follow = follow_symlinks or not (os.path.islink(src) and os.path.islink(dst))
if follow:
# use the real function if it exists
def lookup(name):
return getattr(os, name, _nop)
else:
# use the real function only if it exists
# *and* it supports follow_symlinks
def lookup(name):
fn = getattr(os, name, _nop)
if sys.version_info >= (3, 3):
if fn in os.supports_follow_symlinks: # novermin
return fn
return _nop
st = lookup("stat")(src, follow_symlinks=follow)
mode = stat.S_IMODE(st.st_mode)
lookup("utime")(dst, ns=(st.st_atime_ns, st.st_mtime_ns), follow_symlinks=follow)
# We must copy extended attributes before the file is (potentially)
# chmod()'ed read-only, otherwise setxattr() will error with -EACCES.
shutil._copyxattr(src, dst, follow_symlinks=follow)
try:
lookup("chmod")(dst, mode, follow_symlinks=follow)
except NotImplementedError:
# if we got a NotImplementedError, it's because
# * follow_symlinks=False,
# * lchown() is unavailable, and
# * either
# * fchownat() is unavailable or
# * fchownat() doesn't implement AT_SYMLINK_NOFOLLOW.
# (it returned ENOSUP.)
# therefore we're out of options--we simply cannot chown the
# symlink. give up, suppress the error.
# (which is what shutil always did in this circumstance.)
pass
if hasattr(st, "st_flags"):
try:
lookup("chflags")(dst, st.st_flags, follow_symlinks=follow)
except OSError as why:
for err in "EOPNOTSUPP", "ENOTSUP":
if hasattr(errno, err) and why.errno == getattr(errno, err):
break
else:
raise
shutil.copystat = copystat
def getuid():
if is_windows:
@@ -185,7 +117,13 @@ def path_contains_subdirectory(path, root):
@memoized
def file_command(*args):
"""Creates entry point to `file` system command with provided arguments"""
file_cmd = which("file", required=True)
try:
file_cmd = which("file", required=True)
except CommandNotFoundError as e:
if is_windows:
raise CommandNotFoundError("`file` utility is not available on Windows")
else:
raise e
for arg in args:
file_cmd.add_default_arg(arg)
return file_cmd
@@ -196,11 +134,7 @@ def _get_mime_type():
"""Generate method to call `file` system command to aquire mime type
for a specified path
"""
if is_windows:
# -h option (no-dereference) does not exist in Windows
return file_command("-b", "--mime-type")
else:
return file_command("-b", "-h", "--mime-type")
return file_command("-b", "-h", "--mime-type")
@memoized
@@ -336,6 +270,7 @@ def groupid_to_group(x):
regex = re.escape(regex)
filenames = path_to_os_path(*filenames)
for filename in filenames:
msg = 'FILTER FILE: {0} [replacing "{1}"]'
tty.debug(msg.format(filename, regex))
@@ -1287,6 +1222,7 @@ def traverse_tree(
# target is relative to the link, then that may not resolve properly
# relative to our cwd - see resolve_link_target_relative_to_the_link
if os.path.isdir(source_child) and (follow_links or not os.path.islink(source_child)):
# When follow_nonexisting isn't set, don't descend into dirs
# in source that do not exist in dest
if follow_nonexisting or os.path.exists(dest_child):
@@ -1728,6 +1664,7 @@ def find(root, files, recursive=True):
@system_path_filter
def _find_recursive(root, search_files):
# The variable here is **on purpose** a defaultdict. The idea is that
# we want to poke the filesystem as little as possible, but still maintain
# stability in the order of the answer. Thus we are recording each library
@@ -2698,28 +2635,3 @@ def temporary_dir(
yield tmp_dir
finally:
remove_directory_contents(tmp_dir)
def filesummary(path, print_bytes=16) -> Tuple[int, bytes]:
"""Create a small summary of the given file. Does not error
when file does not exist.
Args:
print_bytes (int): Number of bytes to print from start/end of file
Returns:
Tuple of size and byte string containing first n .. last n bytes.
Size is 0 if file cannot be read."""
try:
n = print_bytes
with open(path, "rb") as f:
size = os.fstat(f.fileno()).st_size
if size <= 2 * n:
short_contents = f.read(2 * n)
else:
short_contents = f.read(n)
f.seek(-n, 2)
short_contents += b"..." + f.read(n)
return size, short_contents
except OSError:
return 0, b""

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -198,7 +198,7 @@ def _memoized_function(*args, **kwargs):
except TypeError as e:
# TypeError is raised when indexing into a dict if the key is unhashable.
raise UnhashableArguments(
"args + kwargs '{}' was not hashable for function '{}'".format(key, func.__name__)
"args + kwargs '{}' was not hashable for function '{}'".format(key, func.__name__),
) from e
return _memoized_function
@@ -237,7 +237,6 @@ def decorator_with_or_without_args(decorator):
@decorator
"""
# See https://stackoverflow.com/questions/653368 for more on this
@functools.wraps(decorator)
def new_dec(*args, **kwargs):
@@ -991,7 +990,8 @@ def enum(**kwargs):
def stable_partition(
input_iterable: Iterable, predicate_fn: Callable[[Any], bool]
input_iterable: Iterable,
predicate_fn: Callable[[Any], bool],
) -> Tuple[List[Any], List[Any]]:
"""Partition the input iterable according to a custom predicate.
@@ -1104,7 +1104,11 @@ def __enter__(self):
def __exit__(self, exc_type, exc_value, tb):
if exc_value is not None:
self._handler._receive_forwarded(self._context, exc_value, traceback.format_tb(tb))
self._handler._receive_forwarded(
self._context,
exc_value,
traceback.format_tb(tb),
)
# Suppress any exception from being re-raised:
# https://docs.python.org/3/reference/datamodel.html#object.__exit__.

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -75,7 +75,7 @@ def __init__(self, ignore=None):
# so that we have a fast lookup and can run mkdir in order.
self.directories = OrderedDict()
# Files to link. Maps dst_rel to (src_root, src_rel)
# Files to link. Maps dst_rel to (src_rel, src_root)
self.files = OrderedDict()
def before_visit_dir(self, root, rel_path, depth):
@@ -430,11 +430,6 @@ class MergeConflictError(Exception):
pass
class ConflictingSpecsError(MergeConflictError):
def __init__(self, spec_1, spec_2):
super(MergeConflictError, self).__init__(spec_1, spec_2)
class SingleMergeConflictError(MergeConflictError):
def __init__(self, path):
super(MergeConflictError, self).__init__("Package merge blocked by file: %s" % path)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -18,7 +18,7 @@ class Barrier:
Python 2 doesn't have multiprocessing barriers so we implement this.
See https://greenteapress.com/semaphores/LittleBookOfSemaphores.pdf, p. 41.
See http://greenteapress.com/semaphores/downey08semaphores.pdf, p. 41.
"""
def __init__(self, n, timeout=None):

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -108,6 +108,7 @@ class SuppressOutput:
"""Class for disabling output in a scope using 'with' keyword"""
def __init__(self, msg_enabled=True, warn_enabled=True, error_enabled=True):
self._msg_enabled_initial = _msg_enabled
self._warn_enabled_initial = _warn_enabled
self._error_enabled_initial = _error_enabled

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -11,7 +11,6 @@
import io
import os
import sys
from typing import IO, Any, List, Optional
from llnl.util.tty import terminal_size
from llnl.util.tty.color import cextra, clen
@@ -98,16 +97,7 @@ def config_uniform_cols(elts, console_width, padding, cols=0):
return config
def colify(
elts: List[Any],
cols: int = 0,
output: Optional[IO] = None,
indent: int = 0,
padding: int = 2,
tty: Optional[bool] = None,
method: str = "variable",
console_cols: Optional[int] = None,
):
def colify(elts, **options):
"""Takes a list of elements as input and finds a good columnization
of them, similar to how gnu ls does. This supports both
uniform-width and variable-width (tighter) columns.
@@ -116,21 +106,31 @@ def colify(
using ``str()``.
Keyword Arguments:
output: A file object to write to. Default is ``sys.stdout``
indent: Optionally indent all columns by some number of spaces
padding: Spaces between columns. Default is 2
width: Width of the output. Default is 80 if tty not detected
cols: Force number of columns. Default is to size to terminal, or
output (typing.IO): A file object to write to. Default is ``sys.stdout``
indent (int): Optionally indent all columns by some number of spaces
padding (int): Spaces between columns. Default is 2
width (int): Width of the output. Default is 80 if tty not detected
cols (int): Force number of columns. Default is to size to terminal, or
single-column if no tty
tty: Whether to attempt to write to a tty. Default is to autodetect a
tty (bool): Whether to attempt to write to a tty. Default is to autodetect a
tty. Set to False to force single-column output
method: Method to use to fit columns. Options are variable or uniform.
method (str): Method to use to fit columns. Options are variable or uniform.
Variable-width columns are tighter, uniform columns are all the same width
and fit less data on the screen
console_cols: number of columns on this console (default: autodetect)
"""
if output is None:
output = sys.stdout
# Get keyword arguments or set defaults
cols = options.pop("cols", 0)
output = options.pop("output", sys.stdout)
indent = options.pop("indent", 0)
padding = options.pop("padding", 2)
tty = options.pop("tty", None)
method = options.pop("method", "variable")
console_cols = options.pop("width", None)
if options:
raise TypeError(
"'%s' is an invalid keyword argument for this function." % next(options.iterkeys())
)
# elts needs to be an array of strings so we can count the elements
elts = [str(elt) for elt in elts]
@@ -153,11 +153,10 @@ def colify(
cols = 1
# Specify the number of character columns to use.
if console_cols is None:
if not console_cols:
console_rows, console_cols = terminal_size()
elif not isinstance(console_cols, int):
elif type(console_cols) != int:
raise ValueError("Number of columns must be an int")
console_cols = max(1, console_cols - indent)
# Choose a method. Variable-width colums vs uniform-width.
@@ -193,13 +192,7 @@ def colify(
return (config.cols, tuple(config.widths))
def colify_table(
table: List[List[Any]],
output: Optional[IO] = None,
indent: int = 0,
padding: int = 2,
console_cols: Optional[int] = None,
):
def colify_table(table, **options):
"""Version of ``colify()`` for data expressed in rows, (list of lists).
Same as regular colify but:
@@ -225,38 +218,20 @@ def transpose():
for row in table:
yield row[i]
colify(
transpose(),
cols=columns, # this is always the number of cols in the table
tty=True, # don't reduce to 1 column for non-tty
output=output,
indent=indent,
padding=padding,
console_cols=console_cols,
)
if "cols" in options:
raise ValueError("Cannot override columsn in colify_table.")
options["cols"] = columns
# don't reduce to 1 column for non-tty
options["tty"] = True
colify(transpose(), **options)
def colified(
elts: List[Any],
cols: int = 0,
output: Optional[IO] = None,
indent: int = 0,
padding: int = 2,
tty: Optional[bool] = None,
method: str = "variable",
console_cols: Optional[int] = None,
):
def colified(elts, **options):
"""Invokes the ``colify()`` function but returns the result as a string
instead of writing it to an output string."""
sio = io.StringIO()
colify(
elts,
cols=cols,
output=sio,
indent=indent,
padding=padding,
tty=tty,
method=method,
console_cols=console_cols,
)
options["output"] = sio
colify(elts, **options)
return sio.getvalue()

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -161,7 +161,10 @@ def _is_background(self):
def _get_canon_echo_flags(self):
"""Get current termios canonical and echo settings."""
cfg = termios.tcgetattr(self.stream)
return (bool(cfg[3] & termios.ICANON), bool(cfg[3] & termios.ECHO))
return (
bool(cfg[3] & termios.ICANON),
bool(cfg[3] & termios.ECHO),
)
def _enable_keyboard_input(self):
"""Disable canonical input and echoing on ``self.stream``."""

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -77,7 +77,10 @@ def __init__(self, pid, controller_fd, timeout=1, sleep_time=1e-1, debug=False):
def get_canon_echo_attrs(self):
"""Get echo and canon attributes of the terminal of controller_fd."""
cfg = termios.tcgetattr(self.controller_fd)
return (bool(cfg[3] & termios.ICANON), bool(cfg[3] & termios.ECHO))
return (
bool(cfg[3] & termios.ICANON),
bool(cfg[3] & termios.ECHO),
)
def horizontal_line(self, name):
"""Labled horizontal line for debugging."""
@@ -89,7 +92,11 @@ def status(self):
if self.debug:
canon, echo = self.get_canon_echo_attrs()
sys.stderr.write(
"canon: %s, echo: %s\n" % ("on" if canon else "off", "on" if echo else "off")
"canon: %s, echo: %s\n"
% (
"on" if canon else "off",
"on" if echo else "off",
)
)
sys.stderr.write("input: %s\n" % self.input_on())
sys.stderr.write("bg: %s\n" % self.background())

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -321,7 +321,8 @@ def _check_patch_urls(pkgs, error_cls):
errors.append(
error_cls(
"patch URL in package {0} must end with {1}".format(
pkg_cls.name, full_index_arg
pkg_cls.name,
full_index_arg,
),
[patch.url],
)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -40,8 +40,6 @@
import spack.relocate as relocate
import spack.repo
import spack.store
import spack.traverse as traverse
import spack.util.crypto
import spack.util.file_cache as file_cache
import spack.util.gpg
import spack.util.spack_json as sjson
@@ -49,7 +47,7 @@
import spack.util.url as url_util
import spack.util.web as web_util
from spack.caches import misc_cache_location
from spack.relocate_text import utf8_paths_to_single_binary_regex
from spack.relocate import utf8_paths_to_single_binary_regex
from spack.spec import Spec
from spack.stage import Stage
from spack.util.executable import which
@@ -211,7 +209,10 @@ def _associate_built_specs_with_mirror(self, cache_key, mirror_url):
break
else:
self._mirrors_for_spec[dag_hash].append(
{"mirror_url": mirror_url, "spec": indexed_spec}
{
"mirror_url": mirror_url,
"spec": indexed_spec,
}
)
finally:
shutil.rmtree(tmpdir)
@@ -293,9 +294,10 @@ def update_spec(self, spec, found_list):
cur_entry["spec"] = new_entry["spec"]
break
else:
current_list.append(
{"mirror_url": new_entry["mirror_url"], "spec": new_entry["spec"]}
)
current_list.append = {
"mirror_url": new_entry["mirror_url"],
"spec": new_entry["spec"],
}
def update(self, with_cooldown=False):
"""Make sure local cache of buildcache index files is up to date.
@@ -362,7 +364,8 @@ def update(self, with_cooldown=False):
# May need to fetch the index and update the local caches
try:
needs_regen = self._fetch_and_cache_index(
cached_mirror_url, cache_entry=cache_entry
cached_mirror_url,
cache_entry=cache_entry,
)
self._last_fetch_times[cached_mirror_url] = (now, True)
all_methods_failed = False
@@ -507,9 +510,9 @@ class NoOverwriteException(spack.error.SpackError):
"""
def __init__(self, file_path):
super(NoOverwriteException, self).__init__(
'"{}" exists in buildcache. Use --force flag to overwrite.'.format(file_path)
)
err_msg = "\n%s\nexists\n" % file_path
err_msg += "Use -f option to overwrite."
super(NoOverwriteException, self).__init__(err_msg)
class NoGpgException(spack.error.SpackError):
@@ -554,12 +557,7 @@ class NoChecksumException(spack.error.SpackError):
Raised if file fails checksum verification.
"""
def __init__(self, path, size, contents, algorithm, expected, computed):
super(NoChecksumException, self).__init__(
f"{algorithm} checksum failed for {path}",
f"Expected {expected} but got {computed}. "
f"File size = {size} bytes. Contents = {contents!r}",
)
pass
class NewLayoutException(spack.error.SpackError):
@@ -1198,42 +1196,40 @@ def _build_tarball(
):
raise NoOverwriteException(url_util.format(remote_specfile_path))
pkg_dir = os.path.basename(spec.prefix.rstrip(os.path.sep))
workdir = os.path.join(tmpdir, pkg_dir)
# TODO: We generally don't want to mutate any files, but when using relative
# mode, Spack unfortunately *does* mutate rpaths and links ahead of time.
# For now, we only make a full copy of the spec prefix when in relative mode.
if relative:
# tarfile is used because it preserves hardlink etc best.
binaries_dir = workdir
temp_tarfile_name = tarball_name(spec, ".tar")
temp_tarfile_path = os.path.join(tarfile_dir, temp_tarfile_name)
with closing(tarfile.open(temp_tarfile_path, "w")) as tar:
tar.add(name="%s" % spec.prefix, arcname=".")
with closing(tarfile.open(temp_tarfile_path, "r")) as tar:
tar.extractall(workdir)
os.remove(temp_tarfile_path)
else:
binaries_dir = spec.prefix
mkdirp(os.path.join(workdir, ".spack"))
# make a copy of the install directory to work with
workdir = os.path.join(tmpdir, os.path.basename(spec.prefix))
# install_tree copies hardlinks
# create a temporary tarfile from prefix and exract it to workdir
# tarfile preserves hardlinks
temp_tarfile_name = tarball_name(spec, ".tar")
temp_tarfile_path = os.path.join(tarfile_dir, temp_tarfile_name)
with closing(tarfile.open(temp_tarfile_path, "w")) as tar:
tar.add(name="%s" % spec.prefix, arcname=".")
with closing(tarfile.open(temp_tarfile_path, "r")) as tar:
tar.extractall(workdir)
os.remove(temp_tarfile_path)
# create info for later relocation and create tar
write_buildinfo_file(spec, workdir, relative)
# optionally make the paths in the binaries relative to each other
# in the spack install tree before creating tarball
try:
if relative:
if relative:
try:
make_package_relative(workdir, spec, allow_root)
elif not allow_root:
ensure_package_relocatable(workdir, binaries_dir)
except Exception as e:
shutil.rmtree(workdir)
shutil.rmtree(tarfile_dir)
shutil.rmtree(tmpdir)
tty.die(e)
except Exception as e:
shutil.rmtree(workdir)
shutil.rmtree(tarfile_dir)
shutil.rmtree(tmpdir)
tty.die(e)
else:
try:
check_package_relocatable(workdir, spec, allow_root)
except Exception as e:
shutil.rmtree(workdir)
shutil.rmtree(tarfile_dir)
shutil.rmtree(tmpdir)
tty.die(e)
# create gzip compressed tarball of the install prefix
# On AMD Ryzen 3700X and an SSD disk, we have the following on compression speed:
@@ -1241,13 +1237,7 @@ def _build_tarball(
# compresslevel=9 python default: llvm takes 12mins, roughly 2.1GB
# So we follow gzip.
with closing(tarfile.open(tarfile_path, "w:gz", compresslevel=6)) as tar:
tar.add(name=binaries_dir, arcname=pkg_dir)
if not relative:
# Add buildinfo file
buildinfo_path = buildinfo_file_name(workdir)
buildinfo_arcname = buildinfo_file_name(pkg_dir)
tar.add(name=buildinfo_path, arcname=buildinfo_arcname)
tar.add(name="%s" % workdir, arcname="%s" % os.path.basename(spec.prefix))
# remove copy of install directory
shutil.rmtree(workdir)
@@ -1308,48 +1298,57 @@ def _build_tarball(
return None
def nodes_to_be_packaged(specs, root=True, dependencies=True):
def nodes_to_be_packaged(specs, include_root=True, include_dependencies=True):
"""Return the list of nodes to be packaged, given a list of specs.
Args:
specs (List[spack.spec.Spec]): list of root specs to be processed
root (bool): include the root of each spec in the nodes
dependencies (bool): include the dependencies of each
include_root (bool): include the root of each spec in the nodes
include_dependencies (bool): include the dependencies of each
spec in the nodes
"""
if not root and not dependencies:
return []
elif dependencies:
nodes = traverse.traverse_nodes(specs, root=root, deptype="all")
else:
nodes = set(specs)
if not include_root and not include_dependencies:
return set()
# Limit to installed non-externals.
packageable = lambda n: not n.external and n.installed
def skip_node(current_node):
if current_node.external or current_node.virtual:
return True
return spack.store.db.query_one(current_node) is None
# Mass install check
with spack.store.db.read_transaction():
return list(filter(packageable, nodes))
expanded_set = set()
for current_spec in specs:
if not include_dependencies:
nodes = [current_spec]
else:
nodes = [
n
for n in current_spec.traverse(
order="post", root=include_root, deptype=("link", "run")
)
]
for node in nodes:
if not skip_node(node):
expanded_set.add(node)
return expanded_set
def push(specs, push_url, include_root: bool = True, include_dependencies: bool = True, **kwargs):
def push(specs, push_url, specs_kwargs=None, **kwargs):
"""Create a binary package for each of the specs passed as input and push them
to a given push URL.
Args:
specs (List[spack.spec.Spec]): installed specs to be packaged
push_url (str): url where to push the binary package
include_root (bool): include the root of each spec in the nodes
include_dependencies (bool): include the dependencies of each
spec in the nodes
specs_kwargs (dict): dictionary with two possible boolean keys, "include_root"
and "include_dependencies", which determine which part of each spec is
packaged and pushed to the mirror
**kwargs: TODO
"""
# Be explicit about the arugment type
if type(include_root) != bool or type(include_dependencies) != bool:
raise ValueError("Expected include_root/include_dependencies to be True/False")
nodes = nodes_to_be_packaged(specs, root=include_root, dependencies=include_dependencies)
specs_kwargs = specs_kwargs or {"include_root": True, "include_dependencies": True}
nodes = nodes_to_be_packaged(specs, **specs_kwargs)
# TODO: This seems to be an easy target for task
# TODO: distribution using a parallel pool
@@ -1566,11 +1565,16 @@ def make_package_relative(workdir, spec, allow_root):
relocate.make_link_relative(cur_path_names, orig_path_names)
def ensure_package_relocatable(workdir, binaries_dir):
"""Check if package binaries are relocatable."""
def check_package_relocatable(workdir, spec, allow_root):
"""
Check if package binaries are relocatable.
Change links to placeholder links.
"""
buildinfo = read_buildinfo_file(workdir)
binaries = [os.path.join(binaries_dir, f) for f in buildinfo["relocate_binaries"]]
relocate.ensure_binaries_are_relocatable(binaries)
cur_path_names = list()
for filename in buildinfo["relocate_binaries"]:
cur_path_names.append(os.path.join(workdir, filename))
allow_root or relocate.ensure_binaries_are_relocatable(cur_path_names)
def dedupe_hardlinks_if_necessary(root, buildinfo):
@@ -1726,16 +1730,16 @@ def is_backup_file(file):
# For all buildcaches
# relocate the install prefixes in text files including dependencies
relocate.relocate_text(text_names, prefix_to_prefix_text)
relocate.unsafe_relocate_text(text_names, prefix_to_prefix_text)
# relocate the install prefixes in binary files including dependencies
relocate.relocate_text_bin(files_to_relocate, prefix_to_prefix_bin)
relocate.unsafe_relocate_text_bin(files_to_relocate, prefix_to_prefix_bin)
# If we are installing back to the same location
# relocate the sbang location if the spack directory changed
else:
if old_spack_prefix != new_spack_prefix:
relocate.relocate_text(text_names, prefix_to_prefix_text)
relocate.unsafe_relocate_text(text_names, prefix_to_prefix_text)
def _extract_inner_tarball(spec, filename, extract_to, unsigned, remote_checksum):
@@ -1773,15 +1777,14 @@ def _extract_inner_tarball(spec, filename, extract_to, unsigned, remote_checksum
raise UnsignedPackageException(
"To install unsigned packages, use the --no-check-signature option."
)
# compute the sha256 checksum of the tarball
# get the sha256 checksum of the tarball
local_checksum = checksum_tarball(tarfile_path)
expected = remote_checksum["hash"]
# if the checksums don't match don't install
if local_checksum != expected:
size, contents = fsys.filesummary(tarfile_path)
raise NoChecksumException(tarfile_path, size, contents, "sha256", expected, local_checksum)
if local_checksum != remote_checksum["hash"]:
raise NoChecksumException(
"Package tarball failed checksum verification.\n" "It cannot be installed."
)
return tarfile_path
@@ -1839,14 +1842,12 @@ def extract_tarball(spec, download_result, allow_root=False, unsigned=False, for
# compute the sha256 checksum of the tarball
local_checksum = checksum_tarball(tarfile_path)
expected = bchecksum["hash"]
# if the checksums don't match don't install
if local_checksum != expected:
size, contents = fsys.filesummary(tarfile_path)
if local_checksum != bchecksum["hash"]:
_delete_staged_downloads(download_result)
raise NoChecksumException(
tarfile_path, size, contents, "sha256", expected, local_checksum
"Package tarball failed checksum verification.\n" "It cannot be installed."
)
new_relative_prefix = str(os.path.relpath(spec.prefix, spack.store.layout.root))
@@ -1937,11 +1938,8 @@ def install_root_node(spec, allow_root, unsigned=False, force=False, sha256=None
tarball_path = download_result["tarball_stage"].save_filename
msg = msg.format(tarball_path, sha256)
if not checker.check(tarball_path):
size, contents = fsys.filesummary(tarball_path)
_delete_staged_downloads(download_result)
raise NoChecksumException(
tarball_path, size, contents, checker.hash_name, sha256, checker.sum
)
raise spack.binary_distribution.NoChecksumException(msg)
tty.debug("Verified SHA256 checksum of the build cache")
# don't print long padded paths while extracting/relocating binaries
@@ -2015,7 +2013,12 @@ def try_direct_fetch(spec, mirrors=None):
fetched_spec = Spec.from_json(specfile_contents)
fetched_spec._mark_concrete()
found_specs.append({"mirror_url": mirror.fetch_url, "spec": fetched_spec})
found_specs.append(
{
"mirror_url": mirror.fetch_url,
"spec": fetched_spec,
}
)
return found_specs
@@ -2317,7 +2320,11 @@ def download_single_spec(concrete_spec, destination, mirror_url=None):
local_tarball_path = os.path.join(destination, tarball_dir_name)
files_to_fetch = [
{"url": [tarball_path_name], "path": local_tarball_path, "required": True},
{
"url": [tarball_path_name],
"path": local_tarball_path,
"required": True,
},
{
"url": [
tarball_name(concrete_spec, ".spec.json.sig"),
@@ -2438,7 +2445,12 @@ def conditional_fetch(self):
response.headers.get("Etag", None) or response.headers.get("etag", None)
)
return FetchIndexResult(etag=etag, hash=computed_hash, data=result, fresh=False)
return FetchIndexResult(
etag=etag,
hash=computed_hash,
data=result,
fresh=False,
)
class EtagIndexFetcher:

View File

@@ -1,11 +1,15 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""Function and classes needed to bootstrap Spack itself."""
from .config import ensure_bootstrap_configuration, is_bootstrapping
from .core import all_core_root_specs, ensure_core_dependencies, ensure_patchelf_in_path_or_raise
from .core import (
all_core_root_specs,
ensure_core_dependencies,
ensure_patchelf_in_path_or_raise,
)
from .environment import BootstrapEnvironment, ensure_environment_dependencies
from .status import status_message

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -59,7 +59,10 @@ def _try_import_from_store(module, query_spec, query_info=None):
# to be picked up and used, possibly depending on something in the store, first
# allows the bootstrap version to work when an incompatible version is in
# sys.path
orders = [module_paths + sys.path, sys.path + module_paths]
orders = [
module_paths + sys.path,
sys.path + module_paths,
]
for path in orders:
sys.path = path
try:

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -53,7 +53,12 @@
import spack.util.url
import spack.version
from ._common import _executables_in_store, _python_import, _root_spec, _try_import_from_store
from ._common import (
_executables_in_store,
_python_import,
_root_spec,
_try_import_from_store,
)
from .config import spack_python_interpreter, spec_for_current_python
#: Name of the file containing metadata about the bootstrapping source

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -171,7 +171,7 @@ def mypy_root_spec():
def black_root_spec():
"""Return the root spec used to bootstrap black"""
return _root_spec("py-black@:23.1.0")
return _root_spec("py-black")
def flake8_root_spec():

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -1016,6 +1016,7 @@ def get_cmake_prefix_path(pkg):
def _setup_pkg_and_run(
serialized_pkg, function, kwargs, child_pipe, input_multiprocess_fd, jsfd1, jsfd2
):
context = kwargs.get("context", "build")
try:

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -110,7 +110,11 @@ class AutotoolsBuilder(BaseBuilder):
phases = ("autoreconf", "configure", "build", "install")
#: Names associated with package methods in the old build-system format
legacy_methods = ("configure_args", "check", "installcheck")
legacy_methods = (
"configure_args",
"check",
"installcheck",
)
#: Names associated with package attributes in the old build-system format
legacy_attributes = (

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -31,6 +31,7 @@ def cmake_cache_option(name, boolean_value, comment=""):
class CachedCMakeBuilder(CMakeBuilder):
#: Phases of a Cached CMake package
#: Note: the initconfig phase is used for developer builds as a final phase to stop on
phases: Tuple[str, ...] = ("initconfig", "cmake", "build", "install")

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -252,7 +252,10 @@ def std_args(pkg, generator=None):
if platform.mac_ver()[0]:
args.extend(
[define("CMAKE_FIND_FRAMEWORK", "LAST"), define("CMAKE_FIND_APPBUNDLE", "LAST")]
[
define("CMAKE_FIND_FRAMEWORK", "LAST"),
define("CMAKE_FIND_APPBUNDLE", "LAST"),
]
)
# Set up CMake rpath

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

View File

@@ -1,4 +1,4 @@
# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
@@ -8,7 +8,7 @@
import spack.directives
import spack.package_base
from ._checks import BaseBuilder, apply_macos_rpath_fixups, execute_install_time_tests
from ._checks import BaseBuilder, apply_macos_rpath_fixups
class Package(spack.package_base.PackageBase):
@@ -38,13 +38,7 @@ class GenericBuilder(BaseBuilder):
legacy_methods: Tuple[str, ...] = ()
#: Names associated with package attributes in the old build-system format
legacy_attributes: Tuple[str, ...] = ("archive_files", "install_time_test_callbacks")
#: Callback names for post-install phase tests
install_time_test_callbacks = []
legacy_attributes: Tuple[str, ...] = ("archive_files",)
# On macOS, force rpaths for shared library IDs and remove duplicate rpaths
spack.builder.run_after("install", when="platform=darwin")(apply_macos_rpath_fixups)
# unconditionally perform any post-install phase tests
spack.builder.run_after("install")(execute_install_time_tests)

Some files were not shown because too many files have changed in this diff Show More