zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update metadata for bootstrapping (#33665)

Browse Source
This commit is contained in:
Massimiliano Culpo
2022-11-03 10:05:03 +01:00
committed by GitHub
parent 707e56dea8
commit 0d82688903
17 changed files with 1128 additions and 999 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
lib/spack/docs/getting_started.rst
View File

@@ -124,61 +124,36 @@ Spack provides two ways of bootstrapping ``clingo``: from pre-built binaries
(default), or from sources. The fastest way to get started is to bootstrap from
pre-built binaries.
.. note::
When bootstrapping from pre-built binaries, Spack currently requires
``patchelf`` on Linux and ``otool`` on macOS. If ``patchelf`` is not in the
``PATH``, Spack will build it from sources, and a C++ compiler is required.
The first time you concretize a spec, Spack will bootstrap in the background:
The first time you concretize a spec, Spack will bootstrap automatically:
.. code-block:: console
$ time spack spec zlib
$ spack spec zlib
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-ba5ijauisd3uuixtmactc36vps7yfsrl.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstrap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-ba5ijauisd3uuixtmactc36vps7yfsrl.spack
==> Installing "clingo-bootstrap@spack%gcc@10.2.1~docs~ipo+python+static_libstdcpp build_type=Release arch=linux-centos7-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.4/build_cache/linux-centos7-x86_64/gcc-10.2.1/patchelf-0.16.1/linux-centos7-x86_64-gcc-10.2.1-patchelf-0.16.1-p72zyan5wrzuabtmzq7isa5mzyh6ahdp.spack
==> Installing "patchelf@0.16.1%gcc@10.2.1 ldflags="-static-libstdc++ -static-libgcc" build_system=autotools arch=linux-centos7-x86_64" from a buildcache
Input spec
--------------------------------
zlib
Concretized
--------------------------------
zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=linux-ubuntu18.04-zen
real 0m20.023s
user 0m18.351s
sys 0m0.784s
After this command you'll see that ``clingo`` has been installed for Spack's own use:
.. code-block:: console
$ spack find -b
==> Showing internal bootstrap store at "/root/.spack/bootstrap/store"
==> 3 installed packages
-- linux-rhel5-x86_64 / gcc@9.3.0 -------------------------------
clingo-bootstrap@spack python@3.6
-- linux-ubuntu18.04-zen / gcc@7.5.0 ----------------------------
patchelf@0.13
Subsequent calls to the concretizer will then be much faster:
.. code-block:: console
$ time spack spec zlib
[ ... ]
real 0m0.490s
user 0m0.431s
sys 0m0.041s
zlib@1.2.13%gcc@9.4.0+optimize+pic+shared build_system=makefile arch=linux-ubuntu20.04-icelake
If for security concerns you cannot bootstrap ``clingo`` from pre-built
binaries, you have to mark this bootstrapping method as untrusted. This makes
Spack fall back to bootstrapping from sources:
binaries, you have to disable fetching the binaries we generated with Github Actions.
.. code-block:: console
$ spack bootstrap disable github-actions-v0.4
==> "github-actions-v0.4" is now disabled and will not be used for bootstrapping
$ spack bootstrap disable github-actions-v0.3
==> "github-actions-v0.2" is now disabled and will not be used for bootstrapping
==> "github-actions-v0.3" is now disabled and will not be used for bootstrapping
You can verify that the new settings are effective with:
@@ -213,9 +188,7 @@ under the ``${HOME}/.spack`` directory. The software installed there can be quer
.. code-block:: console
$ spack find --bootstrap
==> Showing internal bootstrap store at "/home/spack/.spack/bootstrap/store"
==> 3 installed packages
$ spack -b find
-- linux-ubuntu18.04-x86_64 / gcc@10.1.0 ------------------------
clingo-bootstrap@spack python@3.6.9 re2c@1.2.1
@@ -224,7 +197,7 @@ In case it's needed the bootstrap store can also be cleaned with:
.. code-block:: console
$ spack clean -b
==> Removing software in "/home/spack/.spack/bootstrap/store"
==> Removing bootstrapped software and configuration in "/home/spack/.spack/bootstrap"
^^^^^^^^^^^^^^^^^^
Check Installation

10
lib/spack/spack/cmd/bootstrap.py
View File

@@ -30,7 +30,7 @@
# Tarball to be downloaded if binary packages are requested in a local mirror
BINARY_TARBALL = "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.2/bootstrap-buildcache.tar.gz"
BINARY_TARBALL = "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.4/bootstrap-buildcache.tar.gz"
#: Subdirectory where to create the mirror
LOCAL_MIRROR_DIR = "bootstrap_cache"
@@ -50,8 +50,8 @@
},
}
CLINGO_JSON = "$spack/share/spack/bootstrap/github-actions-v0.2/clingo.json"
GNUPG_JSON = "$spack/share/spack/bootstrap/github-actions-v0.2/gnupg.json"
CLINGO_JSON = "$spack/share/spack/bootstrap/github-actions-v0.4/clingo.json"
GNUPG_JSON = "$spack/share/spack/bootstrap/github-actions-v0.4/gnupg.json"
# Metadata for a generated source mirror
SOURCE_METADATA = {
@@ -449,10 +449,10 @@ def write_metadata(subdir, metadata):
def _now(args):
with spack.bootstrap.ensure_bootstrap_configuration():
spack.bootstrap.ensure_clingo_importable_or_raise()
spack.bootstrap.ensure_gpg_in_path_or_raise()
if platform.system().lower() == "linux":
spack.bootstrap.ensure_patchelf_in_path_or_raise()
spack.bootstrap.ensure_clingo_importable_or_raise()
spack.bootstrap.ensure_gpg_in_path_or_raise()
def bootstrap(parser, args):

2
lib/spack/spack/test/cmd/bootstrap.py
View File

@@ -168,7 +168,7 @@ def test_remove_and_add_a_source(mutable_config):
assert not sources
# Add it back and check we restored the initial state
_bootstrap("add", "github-actions", "$spack/share/spack/bootstrap/github-actions-v0.2")
_bootstrap("add", "github-actions", "$spack/share/spack/bootstrap/github-actions-v0.3")
sources = spack.bootstrap.bootstrapping_sources()
assert len(sources) == 1

2
lib/spack/spack/test/data/config/bootstrap.yaml
View File

@@ -1,5 +1,5 @@
bootstrap:
sources:
- name: 'github-actions'
metadata: $spack/share/spack/bootstrap/github-actions-v0.2
metadata: $spack/share/spack/bootstrap/github-actions-v0.3
trusted: {}