binary_distribution: content addressable tarballs (#48713)

binary_distribution: content addressable url buildcache

Change how binary mirrors are laid out, adopting content addressing for every
piece of data spack stores in a binary mirror. Items (e.g. tarballs, specfiles, public
keys, indices, etc) are now discoverable via manifest files which give the size,
checksum, compression type, etc of the the stored item. The information in the
manifest, in turn, is used to find the actual data, which is stored by its content
address in the blobs directory. Additionally, signing is now applied to the manifest
files, rather than to the spec files themselves.
This commit is contained in:
Scott Wittenburg 2025-05-06 12:32:15 -06:00 committed by GitHub
parent 6587b2a231
commit 2c05ce3607
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
61 changed files with 5940 additions and 1333 deletions

View File

@ -45,10 +45,14 @@ provided binary cache, which can be a local directory or a remote URL.
Here is an example where a build cache is created in a local directory named Here is an example where a build cache is created in a local directory named
"spack-cache", to which we push the "ninja" spec: "spack-cache", to which we push the "ninja" spec:
ninja-1.12.1-vmvycib6vmiofkdqgrblo7zsvp7odwut
.. code-block:: console .. code-block:: console
$ spack buildcache push ./spack-cache ninja $ spack buildcache push ./spack-cache ninja
==> Pushing binary packages to file:///home/spackuser/spack/spack-cache/build_cache ==> Selected 30 specs to push to file:///home/spackuser/spack/spack-cache
...
==> [30/30] Pushed ninja@1.12.1/ngldn2k
Note that ``ninja`` must be installed locally for this to work. Note that ``ninja`` must be installed locally for this to work.
@ -98,9 +102,10 @@ Now you can use list:
.. code-block:: console .. code-block:: console
$ spack buildcache list $ spack buildcache list
==> 1 cached build. ==> 24 cached builds.
-- linux-ubuntu20.04-skylake / gcc@9.3.0 ------------------------ -- linux-ubuntu22.04-sapphirerapids / gcc@12.3.0 ----------------
ninja@1.10.2 [ ... ]
ninja@1.12.1
With ``mymirror`` configured and an index available, Spack will automatically With ``mymirror`` configured and an index available, Spack will automatically
use it during concretization and installation. That means that you can expect use it during concretization and installation. That means that you can expect
@ -111,17 +116,17 @@ verify by re-installing ninja:
$ spack uninstall ninja $ spack uninstall ninja
$ spack install ninja $ spack install ninja
==> Installing ninja-1.11.1-yxferyhmrjkosgta5ei6b4lqf6bxbscz [ ... ]
==> Fetching file:///home/spackuser/spack/spack-cache/build_cache/linux-ubuntu20.04-skylake-gcc-9.3.0-ninja-1.10.2-yxferyhmrjkosgta5ei6b4lqf6bxbscz.spec.json.sig ==> Installing ninja-1.12.1-ngldn2kpvb6lqc44oqhhow7fzg7xu7lh [24/24]
gpg: Signature made Do 12 Jan 2023 16:01:04 CET gpg: Signature made Thu 06 Mar 2025 10:03:38 AM MST
gpg: using RSA key 61B82B2B2350E171BD17A1744E3A689061D57BF6 gpg: using RSA key 75BC0528114909C076E2607418010FFAD73C9B07
gpg: Good signature from "example (GPG created for Spack) <example@example.com>" [ultimate] gpg: Good signature from "example (GPG created for Spack) <example@example.com>" [ultimate]
==> Fetching file:///home/spackuser/spack/spack-cache/build_cache/linux-ubuntu20.04-skylake/gcc-9.3.0/ninja-1.10.2/linux-ubuntu20.04-skylake-gcc-9.3.0-ninja-1.10.2-yxferyhmrjkosgta5ei6b4lqf6bxbscz.spack ==> Fetching file:///home/spackuser/spack/spack-cache/blobs/sha256/f0/f08eb62661ad159d2d258890127fc6053f5302a2f490c1c7f7bd677721010ee0
==> Extracting ninja-1.10.2-yxferyhmrjkosgta5ei6b4lqf6bxbscz from binary cache ==> Fetching file:///home/spackuser/spack/spack-cache/blobs/sha256/c7/c79ac6e40dfdd01ac499b020e52e57aa91151febaea3ad183f90c0f78b64a31a
==> ninja: Successfully installed ninja-1.11.1-yxferyhmrjkosgta5ei6b4lqf6bxbscz ==> Extracting ninja-1.12.1-ngldn2kpvb6lqc44oqhhow7fzg7xu7lh from binary cache
Search: 0.00s. Fetch: 0.17s. Install: 0.12s. Total: 0.29s ==> ninja: Successfully installed ninja-1.12.1-ngldn2kpvb6lqc44oqhhow7fzg7xu7lh
[+] /home/harmen/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-9.3.0/ninja-1.11.1-yxferyhmrjkosgta5ei6b4lqf6bxbscz Search: 0.00s. Fetch: 0.11s. Install: 0.11s. Extract: 0.10s. Relocate: 0.00s. Total: 0.22s
[+] /home/spackuser/spack/opt/spack/linux-ubuntu22.04-sapphirerapids/gcc-12.3.0/ninja-1.12.1-ngldn2kpvb6lqc44oqhhow7fzg7xu7lh
It worked! You've just completed a full example of creating a build cache with It worked! You've just completed a full example of creating a build cache with
a spec of interest, adding it as a mirror, updating its index, listing the contents, a spec of interest, adding it as a mirror, updating its index, listing the contents,
@ -344,19 +349,18 @@ which lets you get started quickly. See the following resources for more informa
^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Create tarball of installed Spack package and all dependencies. Create tarball of installed Spack package and all dependencies.
Tarballs are checksummed and signed if gpg2 is available. Tarballs and specfiles are compressed and checksummed, manifests are signed if gpg2 is available.
Places them in a directory ``build_cache`` that can be copied to a mirror. Commands like ``spack buildcache install`` will search Spack mirrors to get the list of build caches.
Commands like ``spack buildcache install`` will search Spack mirrors for build_cache to get the list of build caches.
============== ======================================================================================================================== ============== ========================================================================================================================
Arguments Description Arguments Description
============== ======================================================================================================================== ============== ========================================================================================================================
``<specs>`` list of partial specs or hashes with a leading ``/`` to match from installed packages and used for creating build caches ``<specs>`` list of partial specs or hashes with a leading ``/`` to match from installed packages and used for creating build caches
``-d <path>`` directory in which ``build_cache`` directory is created, defaults to ``.`` ``-d <path>`` directory in which ``v3`` and ``blobs`` directories are created, defaults to ``.``
``-f`` overwrite ``.spack`` file in ``build_cache`` directory if it exists ``-f`` overwrite compressed tarball and spec metadata files if they already exist
``-k <key>`` the key to sign package with. In the case where multiple keys exist, the package will be unsigned unless ``-k`` is used. ``-k <key>`` the key to sign package with. In the case where multiple keys exist, the package will be unsigned unless ``-k`` is used.
``-r`` make paths in binaries relative before creating tarball ``-r`` make paths in binaries relative before creating tarball
``-y`` answer yes to all create unsigned ``build_cache`` questions ``-y`` answer yes to all questions about creating unsigned build caches
============== ======================================================================================================================== ============== ========================================================================================================================
^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^
@ -397,6 +401,165 @@ List public keys available on Spack mirror.
========= ============================================== ========= ==============================================
Arguments Description Arguments Description
========= ============================================== ========= ==============================================
``-i`` trust the keys downloaded with prompt for each ``-it`` trust the keys downloaded with prompt for each
``-y`` answer yes to all trust all keys downloaded ``-y`` answer yes to all trust all keys downloaded
========= ============================================== ========= ==============================================
.. _build_cache_layout:
------------------
Build Cache Layout
------------------
This section describes the structure and content of URL-style build caches, as
distinguished from OCI-style build caches.
The entry point for a binary package is a manifest json file that points to at
least two other files stored as content-addressed blobs. These files include a spec
metadata file, as well as the installation directory of the package stored as
a compressed archive file. Binary package manifest files are named to indicate
the package name and version, as well as the hash of the concrete spec. For
example::
gcc-runtime-12.3.0-qyu2lvgt3nxh7izxycugdbgf5gsdpkjt.spec.manifest.json
would contain the manifest for a binary package of ``gcc-runtime@12.3.0``.
The id of the built package is defined to be the DAG hash of the concrete spec,
and exists in the name of the file as well. The id distinguishes a particular
binary package from all other binary packages with the same package name and
version. Below is an example binary package manifest file. Such a file would
live in the versioned spec manifests directory of a binary mirror, for example
``v3/manifests/spec/``::
{
"version": 3,
"data": [
{
"contentLength": 10731083,
"mediaType": "application/vnd.spack.install.v2.tar+gzip",
"compression": "gzip",
"checksumAlgorithm": "sha256",
"checksum": "0f24aa6b5dd7150067349865217acd3f6a383083f9eca111d2d2fed726c88210"
},
{
"contentLength": 1000,
"mediaType": "application/vnd.spack.spec.v5+json",
"compression": "gzip",
"checksumAlgorithm": "sha256",
"checksum": "fba751c4796536737c9acbb718dad7429be1fa485f5585d450ab8b25d12ae041"
}
]
}
The manifest points to both the compressed tar file as well as the compressed
spec metadata file, and contains the checksum of each. This checksum
is also used as the address of the associated file, and hence, must be
known in order to locate the tarball or spec file within the mirror. Once the
tarball or spec metadata file is downloaded, the checksum should be computed locally
and compared to the checksum in the manifest to ensure the contents have not changed
since the binary package was pushed. Spack stores all data files (including compressed
tar files, spec metadata, indices, public keys, etc) within a ``blobs/<hash-algorithm>/``
directory, using the first two characters of the checksum as a sub-directory
to reduce the number files in a single folder. Here is a depiction of the
organization of binary mirror contents::
mirror_directory/
v3/
layout.json
manifests/
spec/
gcc-runtime/
gcc-runtime-12.3.0-s2nqujezsce4x6uhtvxscu7jhewqzztx.spec.manifest.json
gmake/
gmake-4.4.1-lpr4j77rcgkg5536tmiuzwzlcjsiomph.spec.manifest.json
compiler-wrapper/
compiler-wrapper-1.0-s7ieuyievp57vwhthczhaq2ogowf3ohe.spec.manifest.json
index/
index.manifest.json
key/
75BC0528114909C076E2607418010FFAD73C9B07.key.manifest.json
keys.manifest.json
blobs/
sha256/
0f/
0f24aa6b5dd7150067349865217acd3f6a383083f9eca111d2d2fed726c88210
fb/
fba751c4796536737c9acbb718dad7429be1fa485f5585d450ab8b25d12ae041
2a/
2a21836d206ccf0df780ab0be63fdf76d24501375306a35daa6683c409b7922f
...
Files within the ``manifests`` directory are organized into subdirectories by
the type of entity they represent. Binary package manifests live in the ``spec/``
directory, binary cache index manifests live in the ``index/`` directory, and
manifests for public keys and their indices live in the ``key/`` subdirectory.
Regardless of the type of entity they represent, all manifest files are named
with an extension ``.manifest.json``.
Every manifest contains a ``data`` array, each element of which refers to an
associated file stored a content-addressed blob. Considering the example spec
manifest shown above, the compressed installation archive can be found by
picking out the data blob with the appropriate ``mediaType``, which in this
case would be ``application/vnd.spack.install.v1.tar+gzip``. The associated
file is found by looking in the blobs directory under ``blobs/sha256/fb/`` for
the file named with the complete checksum value.
As mentioned above, every entity in a binary mirror (aka build cache) is stored
as a content-addressed blob pointed to by a manifest. While an example spec
manifest (i.e. a manifest for a binary package) is shown above, here is what
the manifest of a build cache index looks like::
{
"version": 3,
"data": [
{
"contentLength": 6411,
"mediaType": "application/vnd.spack.db.v8+json",
"compression": "none",
"checksumAlgorithm": "sha256",
"checksum": "225a3e9da24d201fdf9d8247d66217f5b3f4d0fc160db1498afd998bfd115234"
}
]
}
Some things to note about this manifest are that it points to a blob that is not
compressed (``compression: "none"``), and that the ``mediaType`` is one we have
not seen yet, ``application/vnd.spack.db.v8+json``. The decision not to compress
build cache indices stems from the fact that spack does not yet sign build cache
index manifests. Once that changes, you may start to see these indices stored as
compressed blobs.
For completeness, here are examples of manifests for the other two types of entities
you might find in a spack build cache. First a public key manifest::
{
"version": 3,
"data": [
{
"contentLength": 2472,
"mediaType": "application/pgp-keys",
"compression": "none",
"checksumAlgorithm": "sha256",
"checksum": "9fc18374aebc84deb2f27898da77d4d4410e5fb44c60c6238cb57fb36147e5c7"
}
]
}
Note the ``mediaType`` of ``application/pgp-keys``. Finally, a public key index manifest::
{
"version": 3,
"data": [
{
"contentLength": 56,
"mediaType": "application/vnd.spack.keyindex.v1+json",
"compression": "none",
"checksumAlgorithm": "sha256",
"checksum": "29b3a0eb6064fd588543bc43ac7d42d708a69058dafe4be0859e3200091a9a1c"
}
]
}
Again note the ``mediaType`` of ``application/vnd.spack.keyindex.v1+json``. Also note
that both the above manifest examples refer to uncompressed blobs, this is for the same
reason spack does not yet compress build cache index blobs.

View File

@ -176,92 +176,72 @@ community without needing deep familiarity with GnuPG or Public Key
Infrastructure. Infrastructure.
.. _build_cache_format: .. _build_cache_signing:
------------------ -------------------
Build Cache Format Build Cache Signing
------------------ -------------------
A binary package consists of a metadata file unambiguously defining the For an in-depth description of the layout of a binary mirror, see
built package (and including other details such as how to relocate it) the :ref:`documentation<build_cache_layout>` covering binary caches. The
and the installation directory of the package stored as a compressed key takeaway from that discussion that applies here is that the entry point
archive file. The metadata files can either be unsigned, in which case to a binary package is it's manifest. The manifest refers unambiguously to the
the contents are simply the json-serialized concrete spec plus metadata, spec metadata and compressed archive, which are stored as content-addressed
or they can be signed, in which case the json-serialized concrete spec blobs.
plus metadata is wrapped in a gpg cleartext signature. Built package
metadata files are named to indicate the operating system and
architecture for which the package was built as well as the compiler
used to build it and the packages name and version. For example::
linux-ubuntu18.04-haswell-gcc-7.5.0-zlib-1.2.12-llv2ysfdxnppzjrt5ldybb5c52qbmoow.spec.json.sig The manifest files can either be signed or unsigned, but are always given
a name ending with ``.spec.manifest.json`` regardless. The difference between
would contain the concrete spec and binary metadata for a binary package signed and unsigned manifests is simply that the signed version is wrapped in
of ``zlib@1.2.12``, built for the ``ubuntu`` operating system and ``haswell`` a gpg cleartext signature, as illustrated below::
architecture. The id of the built package exists in the name of the file
as well (after the package name and version) and in this case begins
with ``llv2ys``. The id distinguishes a particular built package from all
other built packages with the same os/arch, compiler, name, and version.
Below is an example of a signed binary package metadata file. Such a
file would live in the ``build_cache`` directory of a binary mirror::
-----BEGIN PGP SIGNED MESSAGE----- -----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512 Hash: SHA512
{ {
"spec": { "version": 3,
<concrete-spec-contents-omitted> "data": [
}, {
"contentLength": 10731083,
"buildcache_layout_version": 1, "mediaType": "application/vnd.spack.install.v2.tar+gzip",
"binary_cache_checksum": { "compression": "gzip",
"hash_algorithm": "sha256", "checksumAlgorithm": "sha256",
"hash": "4f1e46452c35a5e61bcacca205bae1bfcd60a83a399af201a29c95b7cc3e1423" "checksum": "0f24aa6b5dd7150067349865217acd3f6a383083f9eca111d2d2fed726c88210"
} },
{
"contentLength": 1000,
"mediaType": "application/vnd.spack.spec.v5+json",
"compression": "gzip",
"checksumAlgorithm": "sha256",
"checksum": "fba751c4796536737c9acbb718dad7429be1fa485f5585d450ab8b25d12ae041"
}
]
} }
-----BEGIN PGP SIGNATURE----- -----BEGIN PGP SIGNATURE-----
iQGzBAEBCgAdFiEETZn0sLle8jIrdAPLx/P+voVcifMFAmKAGvwACgkQx/P+voVc
ifNoVgv/VrhA+wurVs5GB9PhmMA1m5U/AfXZb4BElDRwpT8ZcTPIv5X8xtv60eyn iQGzBAEBCgAdFiEEdbwFKBFJCcB24mB0GAEP+tc8mwcFAmf2rr4ACgkQGAEP+tc8
4EOneGVbZoMThVxgev/NKARorGmhFXRqhWf+jknJZ1dicpqn/qpv34rELKUpgXU+ mwfefwv+KJs8MsQ5ovFaBdmyx5H/3k4rO4QHBzuSPOB6UaxErA9IyOB31iP6vNTU
QDQ4d1P64AIdTczXe2GI9ZvhOo6+bPvK7LIsTkBbtWmopkomVxF0LcMuxAVIbA6b HzYpxz6F5dJCJWmmNEMN/0+vjhMHEOkqd7M1l5reVcxduTF2yc4tBZUO2gienEHL
887yBvVO0VGlqRnkDW7nXx49r3AG2+wDcoU1f8ep8QtjOcMNaPTPJ0UnjD0VQGW6 W0e+SnUznl1yc/aVpChUiahO2zToCsI8HZRNT4tu6iCnE/OpghqjsSdBOZHmSNDD
4ZFaGZWzdo45MY6tF3o5mqM7zJkVobpoW3iUz6J5tjz7H/nMlGgMkUwY9Kxp2PVH 5wuuCxfDUyWI6ZlLclaaB7RdbCUUJf/iqi711J+wubvnDFhc6Ynwm1xai5laJ1bD
qoj6Zip3LWplnl2OZyAY+vflPFdFh12Xpk4FG7Sxm/ux0r+l8tCAPvtw+G38a5P7 ev3NrSb2AAroeNFVo4iECA0fZC1OZQYzaRmAEhBXtCideGJ5Zf2Cp9hmCwNK8Hq6
QEk2JBr8qMGKASmnRlJUkm1vwz0a95IF3S9YDfTAA2vz6HH3PtsNLFhtorfx8eBi bNt94JP9LqC3FCCJJOMsPyOOhMSA5MU44zyyzloRwEQpHHLuFzVdbTHA3dmTc18n
Wn5aPJAGEPOawEOvXGGbsH4cDEKPeN0n6cy1k92uPEmBLDVsdnur8q42jk5c2Qyx HxNLkZoEMYRc8zNr40g0yb2lCbc+P11TtL1E+5NlE34MX15mPewRCiIFTMwhCnE3
j3DXty57 gFSKtW1MKustZE35/RUwd2mpJRf+mSRVCl1f1RiFjktLjz7vWQq7imIUSam0fPDr
=3gvm XD4aDogm
=RrFX
-----END PGP SIGNATURE----- -----END PGP SIGNATURE-----
If a user has trusted the public key associated with the private key If a user has trusted the public key associated with the private key
used to sign the above spec file, the signature can be verified with used to sign the above manifest file, the signature can be verified with
gpg, as follows:: gpg, as follows::
$ gpg verify linux-ubuntu18.04-haswell-gcc-7.5.0-zlib-1.2.12-llv2ysfdxnppzjrt5ldybb5c52qbmoow.spec.json.sig $ gpg --verify gcc-runtime-12.3.0-s2nqujezsce4x6uhtvxscu7jhewqzztx.spec.manifest.json
The metadata (regardless whether signed or unsigned) contains the checksum When attempting to install a binary package that has been signed, spack will
of the ``.spack`` file containing the actual installation. The checksum should attempt to verify the signature with one of the trusted keys in its keyring,
be compared to a checksum computed locally on the ``.spack`` file to ensure the and will fail if unable to do so. While not recommended, it is possible to
contents have not changed since the binary spec plus metadata were signed. The force installation of a signed package without verification by providing the
``.spack`` files are actually tarballs containing the compressed archive of the ``--no-check-signature`` argument to ``spack install ...``.
install tree. These files, along with the metadata files, live within the
``build_cache`` directory of the mirror, and together are organized as follows::
build_cache/
# unsigned metadata (for indexing, contains sha256 of .spack file)
<arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json
# clearsigned metadata (same as above, but signed)
<arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spec.json.sig
<arch>/
<compiler>/
<name>-<ver>/
# tar.gz-compressed prefix (may support more compression formats later)
<arch>-<compiler>-<name>-<ver>-24zvipcqgg2wyjpvdq2ajy5jnm564hen.spack
Uncompressing and extracting the ``.spack`` file results in the install tree.
This is in contrast to previous versions of spack, where the ``.spack`` file
contained a (duplicated) metadata file, a signature file and a nested tarball
containing the install tree.
.. _internal_implementation: .. _internal_implementation:
@ -320,10 +300,10 @@ the following way:
Reputational Public Key are imported into a keyring by the ``spack gpg …`` Reputational Public Key are imported into a keyring by the ``spack gpg …``
sub-command. This is initiated by the jobs build script which is created by sub-command. This is initiated by the jobs build script which is created by
the generate job at the beginning of the pipeline. the generate job at the beginning of the pipeline.
4. Assuming the package has dependencies those specs are verified using 4. Assuming the package has dependencies those spec manifests are verified using
the keyring. the keyring.
5. The package is built and the spec.json is generated 5. The package is built and the spec manifest is generated
6. The spec.json is signed by the keyring and uploaded to the mirrors 6. The spec manifest is signed by the keyring and uploaded to the mirrors
build cache. build cache.
**Reputational Key** **Reputational Key**
@ -376,24 +356,24 @@ following way:
4. In addition to the secret, the runner creates a tmpfs memory mounted 4. In addition to the secret, the runner creates a tmpfs memory mounted
directory where the GnuPG keyring will be created to verify, and directory where the GnuPG keyring will be created to verify, and
then resign the package specs. then resign the package specs.
5. The job script syncs all spec.json.sig files from the build cache to 5. The job script syncs all spec manifest files from the build cache to
a working directory in the jobs execution environment. a working directory in the jobs execution environment.
6. The job script then runs the ``sign.sh`` script built into the 6. The job script then runs the ``sign.sh`` script built into the
notary Docker image. notary Docker image.
7. The ``sign.sh`` script imports the public components of the 7. The ``sign.sh`` script imports the public components of the
Reputational and Intermediate CI Keys and uses them to verify good Reputational and Intermediate CI Keys and uses them to verify good
signatures on the spec.json.sig files. If any signed spec does not signatures on the spec.manifest.json files. If any signed manifest
verify the job immediately fails. does not verify, the job immediately fails.
8. Assuming all specs are verified, the ``sign.sh`` script then unpacks 8. Assuming all manifests are verified, the ``sign.sh`` script then unpacks
the spec json data from the signed file in preparation for being the manifest json data from the signed file in preparation for being
re-signed with the Reputational Key. re-signed with the Reputational Key.
9. The private components of the Reputational Key are decrypted to 9. The private components of the Reputational Key are decrypted to
standard out using ``aws-encryption-cli`` directly into a ``gpg standard out using ``aws-encryption-cli`` directly into a ``gpg
import …`` statement which imports the key into the import …`` statement which imports the key into the
keyring mounted in-memory. keyring mounted in-memory.
10. The private key is then used to sign each of the json specs and the 10. The private key is then used to sign each of the manifests and the
keyring is removed from disk. keyring is removed from disk.
11. The re-signed json specs are resynced to the AWS S3 Mirror and the 11. The re-signed manifests are resynced to the AWS S3 Mirror and the
public signing of the packages for the develop or release pipeline public signing of the packages for the develop or release pipeline
that created them is complete. that created them is complete.

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,351 @@
# Copyright Spack Project Developers. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import codecs
import json
import os
import pathlib
import tempfile
from typing import NamedTuple
import llnl.util.tty as tty
import spack.binary_distribution as bindist
import spack.database as spack_db
import spack.error
import spack.mirrors.mirror
import spack.spec
import spack.stage
import spack.util.crypto
import spack.util.parallel
import spack.util.url as url_util
import spack.util.web as web_util
from .enums import InstallRecordStatus
from .url_buildcache import (
BlobRecord,
BuildcacheComponent,
compressed_json_from_dict,
get_url_buildcache_class,
sign_file,
try_verify,
)
def v2_tarball_directory_name(spec):
"""
Return name of the tarball directory according to the convention
<os>-<architecture>/<compiler>/<package>-<version>/
"""
return spec.format_path("{architecture}/{compiler.name}-{compiler.version}/{name}-{version}")
def v2_tarball_name(spec, ext):
"""
Return the name of the tarfile according to the convention
<os>-<architecture>-<package>-<dag_hash><ext>
"""
spec_formatted = spec.format_path(
"{architecture}-{compiler.name}-{compiler.version}-{name}-{version}-{hash}"
)
return f"{spec_formatted}{ext}"
def v2_tarball_path_name(spec, ext):
"""
Return the full path+name for a given spec according to the convention
<tarball_directory_name>/<tarball_name>
"""
return os.path.join(v2_tarball_directory_name(spec), v2_tarball_name(spec, ext))
class MigrateSpecResult(NamedTuple):
success: bool
message: str
class MigrationException(spack.error.SpackError):
"""
Raised when migration fails irrevocably
"""
def __init__(self, msg):
super().__init__(msg)
def _migrate_spec(
s: spack.spec.Spec, mirror_url: str, tmpdir: str, unsigned: bool = False, signing_key: str = ""
) -> MigrateSpecResult:
"""Parallelizable function to migrate a single spec"""
print_spec = f"{s.name}/{s.dag_hash()[:7]}"
# Check if the spec file exists in the new location and exit early if so
v3_cache_class = get_url_buildcache_class(layout_version=3)
v3_cache_entry = v3_cache_class(mirror_url, s, allow_unsigned=unsigned)
exists = v3_cache_entry.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
v3_cache_entry.destroy()
if exists:
msg = f"No need to migrate {print_spec}"
return MigrateSpecResult(True, msg)
# Try to fetch the spec metadata
v2_metadata_urls = [
url_util.join(mirror_url, "build_cache", v2_tarball_name(s, ".spec.json.sig"))
]
if unsigned:
v2_metadata_urls.append(
url_util.join(mirror_url, "build_cache", v2_tarball_name(s, ".spec.json"))
)
spec_contents = None
for meta_url in v2_metadata_urls:
try:
_, _, meta_file = web_util.read_from_url(meta_url)
spec_contents = codecs.getreader("utf-8")(meta_file).read()
v2_spec_url = meta_url
break
except (web_util.SpackWebError, OSError):
pass
else:
msg = f"Unable to read metadata for {print_spec}"
return MigrateSpecResult(False, msg)
spec_dict = {}
if unsigned:
# User asked for unsigned, if we found a signed specfile, just ignore
# the signature
if v2_spec_url.endswith(".sig"):
spec_dict = spack.spec.Spec.extract_json_from_clearsig(spec_contents)
else:
spec_dict = json.loads(spec_contents)
else:
# User asked for signed, we must successfully verify the signature
local_signed_pre_verify = os.path.join(
tmpdir, f"{s.name}_{s.dag_hash()}_verify.spec.json.sig"
)
with open(local_signed_pre_verify, "w", encoding="utf-8") as fd:
fd.write(spec_contents)
if not try_verify(local_signed_pre_verify):
return MigrateSpecResult(False, f"Failed to verify signature of {print_spec}")
with open(local_signed_pre_verify, encoding="utf-8") as fd:
spec_dict = spack.spec.Spec.extract_json_from_clearsig(fd.read())
# Read out and remove the bits needed to rename and position the archive
bcc = spec_dict.pop("binary_cache_checksum", None)
if not bcc:
msg = "Cannot migrate a spec that does not have 'binary_cache_checksum'"
return MigrateSpecResult(False, msg)
algorithm = bcc["hash_algorithm"]
checksum = bcc["hash"]
# TODO: Remove this key once oci buildcache no longer uses it
spec_dict["buildcache_layout_version"] = 2
v2_archive_url = url_util.join(mirror_url, "build_cache", v2_tarball_path_name(s, ".spack"))
# spacks web utilities do not include direct copying of s3 objects, so we
# need to download the archive locally, and then push it back to the target
# location
archive_stage_path = os.path.join(tmpdir, f"archive_stage_{s.name}_{s.dag_hash()}")
archive_stage = spack.stage.Stage(v2_archive_url, path=archive_stage_path)
try:
archive_stage.create()
archive_stage.fetch()
except spack.error.FetchError:
return MigrateSpecResult(False, f"Unable to fetch archive for {print_spec}")
local_tarfile_path = archive_stage.save_filename
# As long as we have to download the tarball anyway, we might as well compute the
# checksum locally and check it against the expected value
local_checksum = spack.util.crypto.checksum(
spack.util.crypto.hash_fun_for_algo(algorithm), local_tarfile_path
)
if local_checksum != checksum:
return MigrateSpecResult(
False, f"Checksum mismatch for {print_spec}: expected {checksum}, got {local_checksum}"
)
spec_dict["archive_size"] = os.stat(local_tarfile_path).st_size
# Compress the spec dict and compute its checksum
metadata_checksum_algo = "sha256"
spec_json_path = os.path.join(tmpdir, f"{s.name}_{s.dag_hash()}.spec.json")
metadata_checksum, metadata_size = compressed_json_from_dict(
spec_json_path, spec_dict, metadata_checksum_algo
)
tarball_blob_record = BlobRecord(
spec_dict["archive_size"], v3_cache_class.TARBALL_MEDIATYPE, "gzip", algorithm, checksum
)
metadata_blob_record = BlobRecord(
metadata_size,
v3_cache_class.SPEC_MEDIATYPE,
"gzip",
metadata_checksum_algo,
metadata_checksum,
)
# Compute the urls to the new blobs
v3_archive_url = v3_cache_class.get_blob_url(mirror_url, tarball_blob_record)
v3_spec_url = v3_cache_class.get_blob_url(mirror_url, metadata_blob_record)
# First push the tarball
tty.debug(f"Pushing {local_tarfile_path} to {v3_archive_url}")
try:
web_util.push_to_url(local_tarfile_path, v3_archive_url, keep_original=True)
except Exception:
return MigrateSpecResult(False, f"Failed to push archive for {print_spec}")
# Then push the spec file
tty.debug(f"Pushing {spec_json_path} to {v3_spec_url}")
try:
web_util.push_to_url(spec_json_path, v3_spec_url, keep_original=True)
except Exception:
return MigrateSpecResult(False, f"Failed to push spec metadata for {print_spec}")
# Generate the manifest and write it to a temporary location
manifest = {
"version": v3_cache_class.get_layout_version(),
"data": [tarball_blob_record.to_dict(), metadata_blob_record.to_dict()],
}
manifest_path = os.path.join(tmpdir, f"{s.dag_hash()}.manifest.json")
with open(manifest_path, "w", encoding="utf-8") as f:
json.dump(manifest, f, indent=0, separators=(",", ":"))
# Note: when using gpg clear sign, we need to avoid long lines (19995
# chars). If lines are longer, they are truncated without error. So,
# here we still add newlines, but no indent, so save on file size and
# line length.
# Possibly sign the manifest
if not unsigned:
manifest_path = sign_file(signing_key, manifest_path)
v3_manifest_url = v3_cache_class.get_manifest_url(s, mirror_url)
# Push the manifest
try:
web_util.push_to_url(manifest_path, v3_manifest_url, keep_original=True)
except Exception:
return MigrateSpecResult(False, f"Failed to push manifest for {print_spec}")
return MigrateSpecResult(True, f"Successfully migrated {print_spec}")
def migrate(
mirror: spack.mirrors.mirror.Mirror, unsigned: bool = False, delete_existing: bool = False
) -> None:
"""Perform migration of the given mirror
If unsigned is True, signatures on signed specs will be ignored, and specs
will not be re-signed before pushing to the new location. Otherwise, spack
will attempt to verify signatures and re-sign specs, and will fail if not
able to do so. If delete_existing is True, spack will delete the original
contents of the mirror once the migration is complete."""
signing_key = ""
if not unsigned:
try:
signing_key = bindist.select_signing_key()
except (bindist.NoKeyException, bindist.PickKeyException):
raise MigrationException(
"Signed migration requires exactly one secret key in keychain"
)
delete_action = "deleting" if delete_existing else "keeping"
sign_action = "an unsigned" if unsigned else "a signed"
mirror_url = mirror.fetch_url
tty.msg(
f"Performing {sign_action} migration of {mirror.push_url} "
f"and {delete_action} existing contents"
)
index_url = url_util.join(mirror_url, "build_cache", spack_db.INDEX_JSON_FILE)
contents = None
try:
_, _, index_file = web_util.read_from_url(index_url)
contents = codecs.getreader("utf-8")(index_file).read()
except (web_util.SpackWebError, OSError):
raise MigrationException("Buildcache migration requires a buildcache index")
with tempfile.TemporaryDirectory(dir=spack.stage.get_stage_root()) as tmpdir:
index_path = os.path.join(tmpdir, "_tmp_index.json")
with open(index_path, "w", encoding="utf-8") as fd:
fd.write(contents)
db = bindist.BuildCacheDatabase(tmpdir)
db._read_from_file(pathlib.Path(index_path))
specs_to_migrate = [
s
for s in db.query_local(installed=InstallRecordStatus.ANY)
if not s.external and db.query_local_by_spec_hash(s.dag_hash()).in_buildcache
]
# Run the tasks in parallel if possible
executor = spack.util.parallel.make_concurrent_executor()
migrate_futures = [
executor.submit(_migrate_spec, spec, mirror_url, tmpdir, unsigned, signing_key)
for spec in specs_to_migrate
]
success_count = 0
tty.msg("Migration summary:")
for spec, migrate_future in zip(specs_to_migrate, migrate_futures):
result = migrate_future.result()
msg = f" {spec.name}/{spec.dag_hash()[:7]}: {result.message}"
if result.success:
success_count += 1
tty.msg(msg)
else:
tty.error(msg)
# The migrated index should have the same specs as the original index,
# modulo any specs that we failed to migrate for whatever reason. So
# to avoid having to re-fetch all the spec files now, just mark them
# appropriately in the existing database and push that.
db.mark(spec, "in_buildcache", result.success)
if success_count > 0:
tty.msg("Updating index and pushing keys")
# If the layout.json doesn't yet exist on this mirror, push it
v3_cache_class = get_url_buildcache_class(layout_version=3)
v3_cache_class.maybe_push_layout_json(mirror_url)
# Push the migrated mirror index
index_tmpdir = os.path.join(tmpdir, "rebuild_index")
os.mkdir(index_tmpdir)
bindist._push_index(db, index_tmpdir, mirror_url)
# Push the public part of the signing key
if not unsigned:
keys_tmpdir = os.path.join(tmpdir, "keys")
os.mkdir(keys_tmpdir)
bindist._url_push_keys(
mirror_url, keys=[signing_key], update_index=True, tmpdir=keys_tmpdir
)
else:
tty.warn("No specs migrated, did you mean to perform an unsigned migration instead?")
# Delete the old layout if the user requested it
if delete_existing:
delete_prefix = url_util.join(mirror_url, "build_cache")
tty.msg(f"Recursively deleting {delete_prefix}")
web_util.remove_url(delete_prefix, recursive=True)
tty.msg("Migration complete")

View File

@ -33,6 +33,7 @@
import spack.paths import spack.paths
import spack.repo import spack.repo
import spack.spec import spack.spec
import spack.stage
import spack.store import spack.store
import spack.util.git import spack.util.git
import spack.util.gpg as gpg_util import spack.util.gpg as gpg_util
@ -245,7 +246,9 @@ def rebuild_filter(s: spack.spec.Spec) -> RebuildDecision:
if not spec_locations: if not spec_locations:
return RebuildDecision(True, "not found anywhere") return RebuildDecision(True, "not found anywhere")
urls = ",".join([loc["mirror_url"] for loc in spec_locations]) urls = ",".join(
[f"{loc.url_and_version.url}@v{loc.url_and_version.version}" for loc in spec_locations]
)
message = f"up-to-date [{urls}]" message = f"up-to-date [{urls}]"
return RebuildDecision(False, message) return RebuildDecision(False, message)
@ -1242,33 +1245,31 @@ def write_broken_spec(url, pkg_name, stack_name, job_url, pipeline_url, spec_dic
"""Given a url to write to and the details of the failed job, write an entry """Given a url to write to and the details of the failed job, write an entry
in the broken specs list. in the broken specs list.
""" """
tmpdir = tempfile.mkdtemp() with tempfile.TemporaryDirectory(dir=spack.stage.get_stage_root()) as tmpdir:
file_path = os.path.join(tmpdir, "broken.txt") file_path = os.path.join(tmpdir, "broken.txt")
broken_spec_details = { broken_spec_details = {
"broken-spec": { "broken-spec": {
"job-name": pkg_name, "job-name": pkg_name,
"job-stack": stack_name, "job-stack": stack_name,
"job-url": job_url, "job-url": job_url,
"pipeline-url": pipeline_url, "pipeline-url": pipeline_url,
"concrete-spec-dict": spec_dict, "concrete-spec-dict": spec_dict,
}
} }
}
try: try:
with open(file_path, "w", encoding="utf-8") as fd: with open(file_path, "w", encoding="utf-8") as fd:
syaml.dump(broken_spec_details, fd) syaml.dump(broken_spec_details, fd)
web_util.push_to_url( web_util.push_to_url(
file_path, url, keep_original=False, extra_args={"ContentType": "text/plain"} file_path, url, keep_original=False, extra_args={"ContentType": "text/plain"}
) )
except Exception as err: except Exception as err:
# If there is an S3 error (e.g., access denied or connection # If there is an S3 error (e.g., access denied or connection
# error), the first non boto-specific class in the exception # error), the first non boto-specific class in the exception
# hierarchy is Exception. Just print a warning and return # hierarchy is Exception. Just print a warning and return
msg = f"Error writing to broken specs list {url}: {err}" msg = f"Error writing to broken specs list {url}: {err}"
tty.warn(msg) tty.warn(msg)
finally:
shutil.rmtree(tmpdir)
def read_broken_spec(broken_spec_url): def read_broken_spec(broken_spec_url):

View File

@ -31,12 +31,12 @@
import spack.spec import spack.spec
import spack.util.compression as compression import spack.util.compression as compression
import spack.util.spack_yaml as syaml import spack.util.spack_yaml as syaml
import spack.util.url as url_util
import spack.util.web as web_util import spack.util.web as web_util
from spack import traverse from spack import traverse
from spack.reporters import CDash, CDashConfiguration from spack.reporters import CDash, CDashConfiguration
from spack.reporters.cdash import SPACK_CDASH_TIMEOUT from spack.reporters.cdash import SPACK_CDASH_TIMEOUT
from spack.reporters.cdash import build_stamp as cdash_build_stamp from spack.reporters.cdash import build_stamp as cdash_build_stamp
from spack.url_buildcache import get_url_buildcache_class
IS_WINDOWS = sys.platform == "win32" IS_WINDOWS = sys.platform == "win32"
SPACK_RESERVED_TAGS = ["public", "protected", "notary"] SPACK_RESERVED_TAGS = ["public", "protected", "notary"]
@ -179,33 +179,13 @@ def write_pipeline_manifest(specs, src_prefix, dest_prefix, output_file):
for release_spec in specs: for release_spec in specs:
release_spec_dag_hash = release_spec.dag_hash() release_spec_dag_hash = release_spec.dag_hash()
# TODO: This assumes signed version of the spec cache_class = get_url_buildcache_class(
buildcache_copies[release_spec_dag_hash] = [ layout_version=bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
{ )
"src": url_util.join( buildcache_copies[release_spec_dag_hash] = {
src_prefix, "src": cache_class.get_manifest_url(release_spec, src_prefix),
bindist.build_cache_relative_path(), "dest": cache_class.get_manifest_url(release_spec, dest_prefix),
bindist.tarball_name(release_spec, ".spec.json.sig"), }
),
"dest": url_util.join(
dest_prefix,
bindist.build_cache_relative_path(),
bindist.tarball_name(release_spec, ".spec.json.sig"),
),
},
{
"src": url_util.join(
src_prefix,
bindist.build_cache_relative_path(),
bindist.tarball_path_name(release_spec, ".spack"),
),
"dest": url_util.join(
dest_prefix,
bindist.build_cache_relative_path(),
bindist.tarball_path_name(release_spec, ".spack"),
),
},
]
target_dir = os.path.dirname(output_file) target_dir = os.path.dirname(output_file)

View File

@ -292,6 +292,9 @@ def main_script_replacements(cmd):
) )
maybe_generate_manifest(pipeline, options, manifest_path) maybe_generate_manifest(pipeline, options, manifest_path)
relative_specs_url = bindist.buildcache_relative_specs_url()
relative_keys_url = bindist.buildcache_relative_keys_url()
if options.pipeline_type == PipelineType.COPY_ONLY: if options.pipeline_type == PipelineType.COPY_ONLY:
stage_names.append("copy") stage_names.append("copy")
sync_job = copy.deepcopy(spack_ci_ir["jobs"]["copy"]["attributes"]) sync_job = copy.deepcopy(spack_ci_ir["jobs"]["copy"]["attributes"])
@ -301,9 +304,12 @@ def main_script_replacements(cmd):
if "variables" not in sync_job: if "variables" not in sync_job:
sync_job["variables"] = {} sync_job["variables"] = {}
sync_job["variables"][ sync_job["variables"].update(
"SPACK_COPY_ONLY_DESTINATION" {
] = options.buildcache_destination.fetch_url "SPACK_COPY_ONLY_DESTINATION": options.buildcache_destination.fetch_url,
"SPACK_BUILDCACHE_RELATIVE_KEYS_URL": relative_keys_url,
}
)
pipeline_mirrors = spack.mirrors.mirror.MirrorCollection(binary=True) pipeline_mirrors = spack.mirrors.mirror.MirrorCollection(binary=True)
if "buildcache-source" not in pipeline_mirrors: if "buildcache-source" not in pipeline_mirrors:
@ -333,9 +339,13 @@ def main_script_replacements(cmd):
signing_job["interruptible"] = True signing_job["interruptible"] = True
if "variables" not in signing_job: if "variables" not in signing_job:
signing_job["variables"] = {} signing_job["variables"] = {}
signing_job["variables"][ signing_job["variables"].update(
"SPACK_BUILDCACHE_DESTINATION" {
] = options.buildcache_destination.push_url "SPACK_BUILDCACHE_DESTINATION": options.buildcache_destination.push_url,
"SPACK_BUILDCACHE_RELATIVE_SPECS_URL": relative_specs_url,
"SPACK_BUILDCACHE_RELATIVE_KEYS_URL": relative_keys_url,
}
)
signing_job["dependencies"] = [] signing_job["dependencies"] = []
output_object["sign-pkgs"] = signing_job output_object["sign-pkgs"] = signing_job

View File

@ -2,6 +2,7 @@
# #
# SPDX-License-Identifier: (Apache-2.0 OR MIT) # SPDX-License-Identifier: (Apache-2.0 OR MIT)
import os import os
import pathlib
import shutil import shutil
import sys import sys
import tempfile import tempfile
@ -28,7 +29,7 @@
# Tarball to be downloaded if binary packages are requested in a local mirror # Tarball to be downloaded if binary packages are requested in a local mirror
BINARY_TARBALL = "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.6/bootstrap-buildcache.tar.gz" BINARY_TARBALL = "https://github.com/spack/spack-bootstrap-mirrors/releases/download/v0.6/bootstrap-buildcache-v3.tar.gz"
#: Subdirectory where to create the mirror #: Subdirectory where to create the mirror
LOCAL_MIRROR_DIR = "bootstrap_cache" LOCAL_MIRROR_DIR = "bootstrap_cache"
@ -410,8 +411,9 @@ def _mirror(args):
stage.create() stage.create()
stage.fetch() stage.fetch()
stage.expand_archive() stage.expand_archive()
build_cache_dir = os.path.join(stage.source_path, "build_cache") stage_dir = pathlib.Path(stage.source_path)
shutil.move(build_cache_dir, mirror_dir) for entry in stage_dir.iterdir():
shutil.move(str(entry), mirror_dir)
llnl.util.tty.set_msg_enabled(True) llnl.util.tty.set_msg_enabled(True)
def write_metadata(subdir, metadata): def write_metadata(subdir, metadata):
@ -436,7 +438,6 @@ def write_metadata(subdir, metadata):
shutil.copy(spack.util.path.canonicalize_path(GNUPG_JSON), abs_directory) shutil.copy(spack.util.path.canonicalize_path(GNUPG_JSON), abs_directory)
shutil.copy(spack.util.path.canonicalize_path(PATCHELF_JSON), abs_directory) shutil.copy(spack.util.path.canonicalize_path(PATCHELF_JSON), abs_directory)
instructions += cmd.format("local-binaries", rel_directory) instructions += cmd.format("local-binaries", rel_directory)
instructions += " % spack buildcache update-index <final-path>/bootstrap_cache\n"
print(instructions) print(instructions)

View File

@ -4,11 +4,9 @@
import argparse import argparse
import glob import glob
import json import json
import os
import shutil
import sys import sys
import tempfile import tempfile
from typing import List, Tuple from typing import List, Optional, Tuple
import llnl.util.tty as tty import llnl.util.tty as tty
from llnl.string import plural from llnl.string import plural
@ -27,14 +25,21 @@
import spack.stage import spack.stage
import spack.store import spack.store
import spack.util.parallel import spack.util.parallel
import spack.util.url as url_util
import spack.util.web as web_util import spack.util.web as web_util
from spack import traverse from spack import traverse
from spack.cmd import display_specs from spack.cmd import display_specs
from spack.cmd.common import arguments from spack.cmd.common import arguments
from spack.spec import Spec, save_dependency_specfiles from spack.spec import Spec, save_dependency_specfiles
from ..buildcache_migrate import migrate
from ..enums import InstallRecordStatus from ..enums import InstallRecordStatus
from ..url_buildcache import (
BuildcacheComponent,
BuildcacheEntryError,
URLBuildcacheEntry,
check_mirror_for_layout,
get_url_buildcache_class,
)
description = "create, download and install binary packages" description = "create, download and install binary packages"
section = "packaging" section = "packaging"
@ -272,6 +277,27 @@ def setup_parser(subparser: argparse.ArgumentParser):
) )
update_index.set_defaults(func=update_index_fn) update_index.set_defaults(func=update_index_fn)
# Migrate a buildcache from layout_version 2 to version 3
migrate = subparsers.add_parser("migrate", help=migrate_fn.__doc__)
migrate.add_argument("mirror", type=arguments.mirror_name, help="name of a configured mirror")
migrate.add_argument(
"-u",
"--unsigned",
default=False,
action="store_true",
help="Ignore signatures and do not resign, default is False",
)
migrate.add_argument(
"-d",
"--delete-existing",
default=False,
action="store_true",
help="Delete the previous layout, the default is to keep it.",
)
arguments.add_common_arguments(migrate, ["yes_to_all"])
# TODO: add -y argument to prompt if user really means to delete existing
migrate.set_defaults(func=migrate_fn)
def _matching_specs(specs: List[Spec]) -> List[Spec]: def _matching_specs(specs: List[Spec]) -> List[Spec]:
"""Disambiguate specs and return a list of matching specs""" """Disambiguate specs and return a list of matching specs"""
@ -397,6 +423,10 @@ def push_fn(args):
(s, PackageNotInstalledError("package not installed")) for s in not_installed (s, PackageNotInstalledError("package not installed")) for s in not_installed
) )
# Warn about possible old binary mirror layout
if not mirror.push_url.startswith("oci://"):
check_mirror_for_layout(mirror)
with bindist.make_uploader( with bindist.make_uploader(
mirror=mirror, mirror=mirror,
force=args.force, force=args.force,
@ -527,8 +557,7 @@ def download_fn(args):
if len(specs) != 1: if len(specs) != 1:
tty.die("a single spec argument is required to download from a buildcache") tty.die("a single spec argument is required to download from a buildcache")
if not bindist.download_single_spec(specs[0], args.path): bindist.download_single_spec(specs[0], args.path)
sys.exit(1)
def save_specfile_fn(args): def save_specfile_fn(args):
@ -553,29 +582,78 @@ def save_specfile_fn(args):
) )
def copy_buildcache_file(src_url, dest_url, local_path=None): def copy_buildcache_entry(cache_entry: URLBuildcacheEntry, destination_url: str):
"""Copy from source url to destination url""" """Download buildcache entry and copy it to the destination_url"""
tmpdir = None try:
spec_dict = cache_entry.fetch_metadata()
cache_entry.fetch_archive()
except bindist.BuildcacheEntryError as e:
tty.warn(f"Failed to retrieve buildcache for copying due to {e}")
cache_entry.destroy()
return
if not local_path: spec_blob_record = cache_entry.get_blob_record(BuildcacheComponent.SPEC)
tmpdir = tempfile.mkdtemp() local_spec_path = cache_entry.get_local_spec_path()
local_path = os.path.join(tmpdir, os.path.basename(src_url)) tarball_blob_record = cache_entry.get_blob_record(BuildcacheComponent.TARBALL)
local_tarball_path = cache_entry.get_local_archive_path()
target_spec = spack.spec.Spec.from_dict(spec_dict)
spec_label = f"{target_spec.name}/{target_spec.dag_hash()[:7]}"
if not tarball_blob_record:
cache_entry.destroy()
raise BuildcacheEntryError(f"No source tarball blob record, failed to sync {spec_label}")
# Try to push the tarball
tarball_dest_url = cache_entry.get_blob_url(destination_url, tarball_blob_record)
try: try:
temp_stage = spack.stage.Stage(src_url, path=os.path.dirname(local_path)) web_util.push_to_url(local_tarball_path, tarball_dest_url, keep_original=True)
try: except Exception as e:
temp_stage.create() tty.warn(f"Failed to push {local_tarball_path} to {tarball_dest_url} due to {e}")
temp_stage.fetch() cache_entry.destroy()
web_util.push_to_url(local_path, dest_url, keep_original=True) return
except spack.error.FetchError as e:
# Expected, since we have to try all the possible extensions if not spec_blob_record:
tty.debug("no such file: {0}".format(src_url)) cache_entry.destroy()
tty.debug(e) raise BuildcacheEntryError(f"No source spec blob record, failed to sync {spec_label}")
finally:
temp_stage.destroy() # Try to push the spec file
finally: spec_dest_url = cache_entry.get_blob_url(destination_url, spec_blob_record)
if tmpdir and os.path.exists(tmpdir):
shutil.rmtree(tmpdir) try:
web_util.push_to_url(local_spec_path, spec_dest_url, keep_original=True)
except Exception as e:
tty.warn(f"Failed to push {local_spec_path} to {spec_dest_url} due to {e}")
cache_entry.destroy()
return
# Stage the manifest locally, since if it's signed, we don't want to try to
# to reproduce that here. Instead just push the locally staged manifest to
# the expected path at the destination url.
manifest_src_url = cache_entry.remote_manifest_url
manifest_dest_url = cache_entry.get_manifest_url(target_spec, destination_url)
manifest_stage = spack.stage.Stage(manifest_src_url)
try:
manifest_stage.create()
manifest_stage.fetch()
except Exception as e:
tty.warn(f"Failed to fetch manifest from {manifest_src_url} due to {e}")
manifest_stage.destroy()
cache_entry.destroy()
return
local_manifest_path = manifest_stage.save_filename
try:
web_util.push_to_url(local_manifest_path, manifest_dest_url, keep_original=True)
except Exception as e:
tty.warn(f"Failed to push manifest to {manifest_dest_url} due to {e}")
manifest_stage.destroy()
cache_entry.destroy()
def sync_fn(args): def sync_fn(args):
@ -615,37 +693,21 @@ def sync_fn(args):
) )
) )
build_cache_dir = bindist.build_cache_relative_path()
buildcache_rel_paths = []
tty.debug("Syncing the following specs:") tty.debug("Syncing the following specs:")
for s in env.all_specs(): specs_to_sync = [s for s in env.all_specs() if not s.external]
for s in specs_to_sync:
tty.debug(" {0}{1}: {2}".format("* " if s in env.roots() else " ", s.name, s.dag_hash())) tty.debug(" {0}{1}: {2}".format("* " if s in env.roots() else " ", s.name, s.dag_hash()))
cache_class = get_url_buildcache_class(
buildcache_rel_paths.extend( layout_version=bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
[
os.path.join(build_cache_dir, bindist.tarball_path_name(s, ".spack")),
os.path.join(build_cache_dir, bindist.tarball_name(s, ".spec.json.sig")),
os.path.join(build_cache_dir, bindist.tarball_name(s, ".spec.json")),
os.path.join(build_cache_dir, bindist.tarball_name(s, ".spec.yaml")),
]
) )
src_cache_entry = cache_class(src_mirror_url, s, allow_unsigned=True)
tmpdir = tempfile.mkdtemp() src_cache_entry.read_manifest()
copy_buildcache_entry(src_cache_entry, dest_mirror_url)
try:
for rel_path in buildcache_rel_paths:
src_url = url_util.join(src_mirror_url, rel_path)
local_path = os.path.join(tmpdir, rel_path)
dest_url = url_util.join(dest_mirror_url, rel_path)
tty.debug("Copying {0} to {1} via {2}".format(src_url, dest_url, local_path))
copy_buildcache_file(src_url, dest_url, local_path=local_path)
finally:
shutil.rmtree(tmpdir)
def manifest_copy(manifest_file_list, dest_mirror=None): def manifest_copy(
manifest_file_list: List[str], dest_mirror: Optional[spack.mirrors.mirror.Mirror] = None
):
"""Read manifest files containing information about specific specs to copy """Read manifest files containing information about specific specs to copy
from source to destination, remove duplicates since any binary packge for from source to destination, remove duplicates since any binary packge for
a given hash should be the same as any other, and copy all files specified a given hash should be the same as any other, and copy all files specified
@ -655,21 +717,24 @@ def manifest_copy(manifest_file_list, dest_mirror=None):
for manifest_path in manifest_file_list: for manifest_path in manifest_file_list:
with open(manifest_path, encoding="utf-8") as fd: with open(manifest_path, encoding="utf-8") as fd:
manifest = json.loads(fd.read()) manifest = json.loads(fd.read())
for spec_hash, copy_list in manifest.items(): for spec_hash, copy_obj in manifest.items():
# Last duplicate hash wins # Last duplicate hash wins
deduped_manifest[spec_hash] = copy_list deduped_manifest[spec_hash] = copy_obj
build_cache_dir = bindist.build_cache_relative_path() for spec_hash, copy_obj in deduped_manifest.items():
for spec_hash, copy_list in deduped_manifest.items(): cache_class = get_url_buildcache_class(
for copy_file in copy_list: layout_version=bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
dest = copy_file["dest"] )
if dest_mirror: src_cache_entry = cache_class(
src_relative_path = os.path.join( cache_class.get_base_url(copy_obj["src"]), allow_unsigned=True
build_cache_dir, copy_file["src"].rsplit(build_cache_dir, 1)[1].lstrip("/") )
) src_cache_entry.read_manifest(manifest_url=copy_obj["src"])
dest = url_util.join(dest_mirror.push_url, src_relative_path) if dest_mirror:
tty.debug("copying {0} to {1}".format(copy_file["src"], dest)) destination_url = dest_mirror.push_url
copy_buildcache_file(copy_file["src"], dest) else:
destination_url = cache_class.get_base_url(copy_obj["dest"])
tty.debug("copying {0} to {1}".format(copy_obj["src"], destination_url))
copy_buildcache_entry(src_cache_entry, destination_url)
def update_index(mirror: spack.mirrors.mirror.Mirror, update_keys=False): def update_index(mirror: spack.mirrors.mirror.Mirror, update_keys=False):
@ -693,13 +758,9 @@ def update_index(mirror: spack.mirrors.mirror.Mirror, update_keys=False):
bindist._url_generate_package_index(url, tmpdir) bindist._url_generate_package_index(url, tmpdir)
if update_keys: if update_keys:
keys_url = url_util.join(
url, bindist.build_cache_relative_path(), bindist.build_cache_keys_relative_path()
)
try: try:
with tempfile.TemporaryDirectory(dir=spack.stage.get_stage_root()) as tmpdir: with tempfile.TemporaryDirectory(dir=spack.stage.get_stage_root()) as tmpdir:
bindist.generate_key_index(keys_url, tmpdir) bindist.generate_key_index(url, tmpdir)
except bindist.CannotListKeys as e: except bindist.CannotListKeys as e:
# Do not error out if listing keys went wrong. This usually means that the _gpg path # Do not error out if listing keys went wrong. This usually means that the _gpg path
# does not exist. TODO: distinguish between this and other errors. # does not exist. TODO: distinguish between this and other errors.
@ -711,5 +772,53 @@ def update_index_fn(args):
return update_index(args.mirror, update_keys=args.keys) return update_index(args.mirror, update_keys=args.keys)
def migrate_fn(args):
"""perform in-place binary mirror migration (2 to 3)
A mirror can contain both layout version 2 and version 3 simultaneously without
interference. This command performs in-place migration of a binary mirror laid
out according to version 2, to a binary mirror laid out according to layout
version 3. Only indexed specs will be migrated, so consider updating the mirror
index before running this command. Re-run the command to migrate any missing
items.
The default mode of operation is to perform a signed migration, that is, spack
will attempt to verify the signatures on specs, and then re-sign them before
migration, using whatever keys are already installed in your key ring. You can
migrate a mirror of unsigned binaries (or convert a mirror of signed binaries
to unsigned) by providing the --unsigned argument.
By default spack will leave the original mirror contents (in the old layout) in
place after migration. You can have spack remove the old contents by providing
the --delete-existing argument. Because migrating a mostly-already-migrated
mirror should be fast, consider a workflow where you perform a default migration,
(i.e. preserve the existing layout rather than deleting it) then evaluate the
state of the migrated mirror by attempting to install from it, and finally
running the migration again with --delete-existing."""
target_mirror = args.mirror
unsigned = args.unsigned
assert isinstance(target_mirror, spack.mirrors.mirror.Mirror)
delete_existing = args.delete_existing
proceed = True
if delete_existing and not args.yes_to_all:
msg = (
"Using --delete-existing will delete the entire contents \n"
" of the old layout within the mirror. Because migrating a mirror \n"
" that has already been migrated should be fast, consider a workflow \n"
" where you perform a default migration (i.e. preserve the existing \n"
" layout rather than deleting it), then evaluate the state of the \n"
" migrated mirror by attempting to install from it, and finally, \n"
" run the migration again with --delete-existing."
)
tty.warn(msg)
proceed = tty.get_yes_or_no("Do you want to proceed?", default=False)
if not proceed:
tty.die("Migration aborted.")
migrate(target_mirror, unsigned=unsigned, delete_existing=delete_existing)
def buildcache(parser, args): def buildcache(parser, args):
return args.func(args) return args.func(args)

View File

@ -423,7 +423,7 @@ def ci_rebuild(args):
# jobs in subsequent stages. # jobs in subsequent stages.
tty.msg("No need to rebuild {0}, found hash match at: ".format(job_spec_pkg_name)) tty.msg("No need to rebuild {0}, found hash match at: ".format(job_spec_pkg_name))
for match in matches: for match in matches:
tty.msg(" {0}".format(match["mirror_url"])) tty.msg(" {0}".format(match.url_and_version.url))
# Now we are done and successful # Now we are done and successful
return 0 return 0

View File

@ -202,3 +202,16 @@ class MirrorError(SpackError):
def __init__(self, msg, long_msg=None): def __init__(self, msg, long_msg=None):
super().__init__(msg, long_msg) super().__init__(msg, long_msg)
class NoChecksumException(SpackError):
"""
Raised if file fails checksum verification.
"""
def __init__(self, path, size, contents, algorithm, expected, computed):
super().__init__(
f"{algorithm} checksum failed for {path}",
f"Expected {expected} but got {computed}. "
f"File size = {size} bytes. Contents = {contents!r}",
)

View File

@ -65,6 +65,7 @@
import spack.util.executable import spack.util.executable
import spack.util.path import spack.util.path
import spack.util.timer as timer import spack.util.timer as timer
from spack.url_buildcache import BuildcacheEntryError
from spack.util.environment import EnvironmentModifications, dump_environment from spack.util.environment import EnvironmentModifications, dump_environment
from spack.util.executable import which from spack.util.executable import which
@ -449,17 +450,17 @@ def _process_binary_cache_tarball(
else ``False`` else ``False``
""" """
with timer.measure("fetch"): with timer.measure("fetch"):
download_result = binary_distribution.download_tarball( tarball_stage = binary_distribution.download_tarball(
pkg.spec.build_spec, unsigned, mirrors_for_spec pkg.spec.build_spec, unsigned, mirrors_for_spec
) )
if download_result is None: if tarball_stage is None:
return False return False
tty.msg(f"Extracting {package_id(pkg.spec)} from binary cache") tty.msg(f"Extracting {package_id(pkg.spec)} from binary cache")
with timer.measure("install"), spack.util.path.filter_padding(): with timer.measure("install"), spack.util.path.filter_padding():
binary_distribution.extract_tarball(pkg.spec, download_result, force=False, timer=timer) binary_distribution.extract_tarball(pkg.spec, tarball_stage, force=False, timer=timer)
if pkg.spec.spliced: # overwrite old metadata with new if pkg.spec.spliced: # overwrite old metadata with new
spack.store.STORE.layout.write_spec( spack.store.STORE.layout.write_spec(
@ -2177,7 +2178,7 @@ def install(self) -> None:
) )
raise raise
except binary_distribution.NoChecksumException as exc: except BuildcacheEntryError as exc:
if task.cache_only: if task.cache_only:
raise raise

View File

@ -19,10 +19,6 @@
"additionalProperties": True, "additionalProperties": True,
"items": spack.schema.spec.properties, "items": spack.schema.spec.properties,
}, },
"binary_cache_checksum": {
"type": "object",
"properties": {"hash_algorithm": {"type": "string"}, "hash": {"type": "string"}},
},
"buildcache_layout_version": {"type": "number"}, "buildcache_layout_version": {"type": "number"},
} }
@ -30,6 +26,6 @@
"$schema": "http://json-schema.org/draft-07/schema#", "$schema": "http://json-schema.org/draft-07/schema#",
"title": "Spack buildcache specfile schema", "title": "Spack buildcache specfile schema",
"type": "object", "type": "object",
"additionalProperties": False, "additionalProperties": True,
"properties": properties, "properties": properties,
} }

View File

@ -0,0 +1,45 @@
# Copyright Spack Project Developers. See COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""Schema for buildcache entry manifest file
.. literalinclude:: _spack_root/lib/spack/spack/schema/url_buildcache_manifest.py
:lines: 11-
"""
from typing import Any, Dict
properties: Dict[str, Any] = {
"version": {"type": "integer"},
"data": {
"type": "array",
"items": {
"type": "object",
"required": [
"contentLength",
"mediaType",
"compression",
"checksumAlgorithm",
"checksum",
],
"properties": {
"contentLength": {"type": "integer"},
"mediaType": {"type": "string"},
"compression": {"type": "string"},
"checksumAlgorithm": {"type": "string"},
"checksum": {"type": "string"},
},
"additionalProperties": True,
},
},
}
#: Full schema with metadata
schema = {
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Buildcache manifest schema",
"type": "object",
"required": ["version", "data"],
"additionalProperties": True,
"properties": properties,
}

View File

@ -17,11 +17,10 @@
import urllib.request import urllib.request
import urllib.response import urllib.response
from pathlib import Path, PurePath from pathlib import Path, PurePath
from typing import Any, Callable, Dict, NamedTuple, Optional
import pytest import pytest
import archspec.cpu
from llnl.util.filesystem import copy_tree, join_path from llnl.util.filesystem import copy_tree, join_path
from llnl.util.symlink import readlink from llnl.util.symlink import readlink
@ -38,16 +37,27 @@
import spack.paths import spack.paths
import spack.repo import spack.repo
import spack.spec import spack.spec
import spack.stage
import spack.store import spack.store
import spack.util.gpg import spack.util.gpg
import spack.util.spack_yaml as syaml import spack.util.spack_yaml as syaml
import spack.util.url as url_util import spack.util.url as url_util
import spack.util.web as web_util import spack.util.web as web_util
from spack.binary_distribution import INDEX_HASH_FILE, CannotListKeys, GenerateIndexError from spack.binary_distribution import CannotListKeys, GenerateIndexError
from spack.database import INDEX_JSON_FILE from spack.database import INDEX_JSON_FILE
from spack.installer import PackageInstaller from spack.installer import PackageInstaller
from spack.paths import test_path from spack.paths import test_path
from spack.spec import Spec from spack.spec import Spec
from spack.url_buildcache import (
INDEX_MANIFEST_FILE,
BuildcacheComponent,
BuildcacheEntryError,
URLBuildcacheEntry,
URLBuildcacheEntryV2,
compression_writer,
get_url_buildcache_class,
get_valid_spec_file,
)
pytestmark = pytest.mark.not_on_windows("does not run on windows") pytestmark = pytest.mark.not_on_windows("does not run on windows")
@ -372,7 +382,7 @@ def test_built_spec_cache(temporary_mirror_dir):
for s in [gspec, cspec]: for s in [gspec, cspec]:
results = bindist.get_mirrors_for_spec(s) results = bindist.get_mirrors_for_spec(s)
assert any([r["spec"] == s for r in results]) assert any([r.spec == s for r in results])
def fake_dag_hash(spec, length=None): def fake_dag_hash(spec, length=None):
@ -435,7 +445,11 @@ def test_generate_index_missing(monkeypatch, tmpdir, mutable_config):
assert "libelf" in cache_list assert "libelf" in cache_list
# Remove dependency from cache # Remove dependency from cache
libelf_files = glob.glob(os.path.join(mirror_dir.join("build_cache").strpath, "*libelf*")) libelf_files = glob.glob(
os.path.join(
mirror_dir.join(bindist.buildcache_relative_specs_path()).strpath, "libelf", "*libelf*"
)
)
os.remove(*libelf_files) os.remove(*libelf_files)
# Update index # Update index
@ -480,8 +494,7 @@ def mock_list_url(url, recursive=False):
assert ( assert (
"Warning: Encountered problem listing packages at " "Warning: Encountered problem listing packages at "
f"{test_url}/{bindist.BUILD_CACHE_RELATIVE_PATH}: Some HTTP error" f"{test_url}: Some HTTP error" in capfd.readouterr().err
in capfd.readouterr().err
) )
@ -538,29 +551,6 @@ def test_update_sbang(tmp_path, temporary_mirror, mock_fetch, install_mockery):
assert f.read() == new_contents assert f.read() == new_contents
@pytest.mark.skipif(
str(archspec.cpu.host().family) != "x86_64",
reason="test data uses gcc 4.5.0 which does not support aarch64",
)
def test_install_legacy_buildcache_layout(mutable_config, compiler_factory, install_mockery):
"""Legacy buildcache layout involved a nested archive structure
where the .spack file contained a repeated spec.json and another
compressed archive file containing the install tree. This test
makes sure we can still read that layout."""
legacy_layout_dir = os.path.join(test_path, "data", "mirrors", "legacy_layout")
mirror_url = f"file://{legacy_layout_dir}"
filename = (
"test-debian6-core2-gcc-4.5.0-archive-files-2.0-"
"l3vdiqvbobmspwyb4q2b62fz6nitd4hk.spec.json"
)
spec_json_path = os.path.join(legacy_layout_dir, "build_cache", filename)
mirror_cmd("add", "--scope", "site", "test-legacy-layout", mirror_url)
output = install_cmd("--no-check-signature", "--cache-only", "-f", spec_json_path, output=str)
mirror_cmd("rm", "--scope=site", "test-legacy-layout")
expect_line = "Extracting archive-files-2.0-l3vdiqvbobmspwyb4q2b62fz6nitd4hk from binary cache"
assert expect_line in output
def test_FetchCacheError_only_accepts_lists_of_errors(): def test_FetchCacheError_only_accepts_lists_of_errors():
with pytest.raises(TypeError, match="list"): with pytest.raises(TypeError, match="list"):
bindist.FetchCacheError("error") bindist.FetchCacheError("error")
@ -600,7 +590,60 @@ def test_text_relocate_if_needed(install_mockery, temporary_store, mock_fetch, t
assert join_path("bin", "secretexe") not in manifest["relocate_textfiles"] assert join_path("bin", "secretexe") not in manifest["relocate_textfiles"]
def test_etag_fetching_304(): def test_compression_writer(tmp_path):
text = "This is some text. We might or might not like to compress it as we write."
checksum_algo = "sha256"
# Write the data using gzip compression
compressed_output_path = str(tmp_path / "compressed_text")
with compression_writer(compressed_output_path, "gzip", checksum_algo) as (
compressor,
checker,
):
compressor.write(text.encode("utf-8"))
compressed_size = checker.length
compressed_checksum = checker.hexdigest()
with open(compressed_output_path, "rb") as f:
binary_content = f.read()
assert bindist.compute_hash(binary_content) == compressed_checksum
assert os.stat(compressed_output_path).st_size == compressed_size
assert binary_content[:2] == b"\x1f\x8b"
decompressed_content = gzip.decompress(binary_content).decode("utf-8")
assert decompressed_content == text
# Write the data without compression
uncompressed_output_path = str(tmp_path / "uncompressed_text")
with compression_writer(uncompressed_output_path, "none", checksum_algo) as (
compressor,
checker,
):
compressor.write(text.encode("utf-8"))
uncompressed_size = checker.length
uncompressed_checksum = checker.hexdigest()
with open(uncompressed_output_path, "r", encoding="utf-8") as f:
content = f.read()
assert bindist.compute_hash(content) == uncompressed_checksum
assert os.stat(uncompressed_output_path).st_size == uncompressed_size
assert content == text
# Make sure we raise if requesting unknown compression type
nocare_output_path = str(tmp_path / "wontwrite")
with pytest.raises(BuildcacheEntryError, match="Unknown compression type"):
with compression_writer(nocare_output_path, "gsip", checksum_algo) as (
compressor,
checker,
):
compressor.write(text)
def test_v2_etag_fetching_304():
# Test conditional fetch with etags. If the remote hasn't modified the file # Test conditional fetch with etags. If the remote hasn't modified the file
# it returns 304, which is an HTTPError in urllib-land. That should be # it returns 304, which is an HTTPError in urllib-land. That should be
# handled as success, since it means the local cache is up-to-date. # handled as success, since it means the local cache is up-to-date.
@ -613,7 +656,7 @@ def response_304(request: urllib.request.Request):
) )
assert False, "Should not fetch {}".format(url) assert False, "Should not fetch {}".format(url)
fetcher = bindist.EtagIndexFetcher( fetcher = bindist.EtagIndexFetcherV2(
url="https://www.example.com", url="https://www.example.com",
etag="112a8bbc1b3f7f185621c1ee335f0502", etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_304, urlopen=response_304,
@ -624,7 +667,7 @@ def response_304(request: urllib.request.Request):
assert result.fresh assert result.fresh
def test_etag_fetching_200(): def test_v2_etag_fetching_200():
# Test conditional fetch with etags. The remote has modified the file. # Test conditional fetch with etags. The remote has modified the file.
def response_200(request: urllib.request.Request): def response_200(request: urllib.request.Request):
url = request.get_full_url() url = request.get_full_url()
@ -638,7 +681,7 @@ def response_200(request: urllib.request.Request):
) )
assert False, "Should not fetch {}".format(url) assert False, "Should not fetch {}".format(url)
fetcher = bindist.EtagIndexFetcher( fetcher = bindist.EtagIndexFetcherV2(
url="https://www.example.com", url="https://www.example.com",
etag="112a8bbc1b3f7f185621c1ee335f0502", etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_200, urlopen=response_200,
@ -652,7 +695,7 @@ def response_200(request: urllib.request.Request):
assert result.hash == bindist.compute_hash("Result") assert result.hash == bindist.compute_hash("Result")
def test_etag_fetching_404(): def test_v2_etag_fetching_404():
# Test conditional fetch with etags. The remote has modified the file. # Test conditional fetch with etags. The remote has modified the file.
def response_404(request: urllib.request.Request): def response_404(request: urllib.request.Request):
raise urllib.error.HTTPError( raise urllib.error.HTTPError(
@ -663,7 +706,7 @@ def response_404(request: urllib.request.Request):
fp=None, fp=None,
) )
fetcher = bindist.EtagIndexFetcher( fetcher = bindist.EtagIndexFetcherV2(
url="https://www.example.com", url="https://www.example.com",
etag="112a8bbc1b3f7f185621c1ee335f0502", etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_404, urlopen=response_404,
@ -673,13 +716,13 @@ def response_404(request: urllib.request.Request):
fetcher.conditional_fetch() fetcher.conditional_fetch()
def test_default_index_fetch_200(): def test_v2_default_index_fetch_200():
index_json = '{"Hello": "World"}' index_json = '{"Hello": "World"}'
index_json_hash = bindist.compute_hash(index_json) index_json_hash = bindist.compute_hash(index_json)
def urlopen(request: urllib.request.Request): def urlopen(request: urllib.request.Request):
url = request.get_full_url() url = request.get_full_url()
if url.endswith(INDEX_HASH_FILE): if url.endswith("index.json.hash"):
return urllib.response.addinfourl( # type: ignore[arg-type] return urllib.response.addinfourl( # type: ignore[arg-type]
io.BytesIO(index_json_hash.encode()), io.BytesIO(index_json_hash.encode()),
headers={}, # type: ignore[arg-type] headers={}, # type: ignore[arg-type]
@ -697,7 +740,7 @@ def urlopen(request: urllib.request.Request):
assert False, "Unexpected request {}".format(url) assert False, "Unexpected request {}".format(url)
fetcher = bindist.DefaultIndexFetcher( fetcher = bindist.DefaultIndexFetcherV2(
url="https://www.example.com", local_hash="outdated", urlopen=urlopen url="https://www.example.com", local_hash="outdated", urlopen=urlopen
) )
@ -710,7 +753,7 @@ def urlopen(request: urllib.request.Request):
assert result.hash == index_json_hash assert result.hash == index_json_hash
def test_default_index_dont_fetch_index_json_hash_if_no_local_hash(): def test_v2_default_index_dont_fetch_index_json_hash_if_no_local_hash():
# When we don't have local hash, we should not be fetching the # When we don't have local hash, we should not be fetching the
# remote index.json.hash file, but only index.json. # remote index.json.hash file, but only index.json.
index_json = '{"Hello": "World"}' index_json = '{"Hello": "World"}'
@ -728,7 +771,7 @@ def urlopen(request: urllib.request.Request):
assert False, "Unexpected request {}".format(url) assert False, "Unexpected request {}".format(url)
fetcher = bindist.DefaultIndexFetcher( fetcher = bindist.DefaultIndexFetcherV2(
url="https://www.example.com", local_hash=None, urlopen=urlopen url="https://www.example.com", local_hash=None, urlopen=urlopen
) )
@ -741,13 +784,13 @@ def urlopen(request: urllib.request.Request):
assert not result.fresh assert not result.fresh
def test_default_index_not_modified(): def test_v2_default_index_not_modified():
index_json = '{"Hello": "World"}' index_json = '{"Hello": "World"}'
index_json_hash = bindist.compute_hash(index_json) index_json_hash = bindist.compute_hash(index_json)
def urlopen(request: urllib.request.Request): def urlopen(request: urllib.request.Request):
url = request.get_full_url() url = request.get_full_url()
if url.endswith(INDEX_HASH_FILE): if url.endswith("index.json.hash"):
return urllib.response.addinfourl( return urllib.response.addinfourl(
io.BytesIO(index_json_hash.encode()), io.BytesIO(index_json_hash.encode()),
headers={}, # type: ignore[arg-type] headers={}, # type: ignore[arg-type]
@ -758,7 +801,7 @@ def urlopen(request: urllib.request.Request):
# No request to index.json should be made. # No request to index.json should be made.
assert False, "Unexpected request {}".format(url) assert False, "Unexpected request {}".format(url)
fetcher = bindist.DefaultIndexFetcher( fetcher = bindist.DefaultIndexFetcherV2(
url="https://www.example.com", local_hash=index_json_hash, urlopen=urlopen url="https://www.example.com", local_hash=index_json_hash, urlopen=urlopen
) )
@ -766,7 +809,7 @@ def urlopen(request: urllib.request.Request):
@pytest.mark.parametrize("index_json", [b"\xa9", b"!#%^"]) @pytest.mark.parametrize("index_json", [b"\xa9", b"!#%^"])
def test_default_index_invalid_hash_file(index_json): def test_v2_default_index_invalid_hash_file(index_json):
# Test invalid unicode / invalid hash type # Test invalid unicode / invalid hash type
index_json_hash = bindist.compute_hash(index_json) index_json_hash = bindist.compute_hash(index_json)
@ -778,21 +821,21 @@ def urlopen(request: urllib.request.Request):
code=200, code=200,
) )
fetcher = bindist.DefaultIndexFetcher( fetcher = bindist.DefaultIndexFetcherV2(
url="https://www.example.com", local_hash=index_json_hash, urlopen=urlopen url="https://www.example.com", local_hash=index_json_hash, urlopen=urlopen
) )
assert fetcher.get_remote_hash() is None assert fetcher.get_remote_hash() is None
def test_default_index_json_404(): def test_v2_default_index_json_404():
# Test invalid unicode / invalid hash type # Test invalid unicode / invalid hash type
index_json = '{"Hello": "World"}' index_json = '{"Hello": "World"}'
index_json_hash = bindist.compute_hash(index_json) index_json_hash = bindist.compute_hash(index_json)
def urlopen(request: urllib.request.Request): def urlopen(request: urllib.request.Request):
url = request.get_full_url() url = request.get_full_url()
if url.endswith(INDEX_HASH_FILE): if url.endswith("index.json.hash"):
return urllib.response.addinfourl( return urllib.response.addinfourl(
io.BytesIO(index_json_hash.encode()), io.BytesIO(index_json_hash.encode()),
headers={}, # type: ignore[arg-type] headers={}, # type: ignore[arg-type]
@ -811,7 +854,7 @@ def urlopen(request: urllib.request.Request):
assert False, "Unexpected fetch {}".format(url) assert False, "Unexpected fetch {}".format(url)
fetcher = bindist.DefaultIndexFetcher( fetcher = bindist.DefaultIndexFetcherV2(
url="https://www.example.com", local_hash="invalid", urlopen=urlopen url="https://www.example.com", local_hash="invalid", urlopen=urlopen
) )
@ -1097,9 +1140,7 @@ def test_get_valid_spec_file(tmp_path, layout, expect_success):
json.dump(spec_dict, f) json.dump(spec_dict, f)
try: try:
spec_dict_disk, layout_disk = bindist._get_valid_spec_file( spec_dict_disk, layout_disk = get_valid_spec_file(str(path), max_supported_layout=1)
str(path), max_supported_layout=1
)
assert expect_success assert expect_success
assert spec_dict_disk == spec_dict assert spec_dict_disk == spec_dict
assert layout_disk == effective_layout assert layout_disk == effective_layout
@ -1109,51 +1150,66 @@ def test_get_valid_spec_file(tmp_path, layout, expect_success):
def test_get_valid_spec_file_doesnt_exist(tmp_path): def test_get_valid_spec_file_doesnt_exist(tmp_path):
with pytest.raises(bindist.InvalidMetadataFile, match="No such file"): with pytest.raises(bindist.InvalidMetadataFile, match="No such file"):
bindist._get_valid_spec_file(str(tmp_path / "no-such-file"), max_supported_layout=1) get_valid_spec_file(str(tmp_path / "no-such-file"), max_supported_layout=1)
def test_get_valid_spec_file_gzipped(tmp_path):
# Create a gzipped file, contents don't matter
path = tmp_path / "spec.json.gz"
with gzip.open(path, "wb") as f:
f.write(b"hello")
with pytest.raises(
bindist.InvalidMetadataFile, match="Compressed spec files are not supported"
):
bindist._get_valid_spec_file(str(path), max_supported_layout=1)
@pytest.mark.parametrize("filename", ["spec.json", "spec.json.sig"]) @pytest.mark.parametrize("filename", ["spec.json", "spec.json.sig"])
def test_get_valid_spec_file_no_json(tmp_path, filename): def test_get_valid_spec_file_no_json(tmp_path, filename):
tmp_path.joinpath(filename).write_text("not json") tmp_path.joinpath(filename).write_text("not json")
with pytest.raises(bindist.InvalidMetadataFile): with pytest.raises(bindist.InvalidMetadataFile):
bindist._get_valid_spec_file(str(tmp_path / filename), max_supported_layout=1) get_valid_spec_file(str(tmp_path / filename), max_supported_layout=1)
def test_download_tarball_with_unsupported_layout_fails( @pytest.mark.usefixtures("install_mockery", "mock_packages", "mock_fetch", "temporary_mirror")
tmp_path, mock_packages, mutable_config, capsys def test_url_buildcache_entry_v3(monkeypatch, tmpdir):
): """Make sure URLBuildcacheEntry behaves as expected"""
layout_version = bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION + 1
spec = spack.concretize.concretize_one("pkg-c")
spec_dict = spec.to_dict()
spec_dict["buildcache_layout_version"] = layout_version
# Setup a basic local build cache structure # Create a temp mirror directory for buildcache usage
path = ( mirror_dir = tmpdir.join("mirror_dir")
tmp_path / bindist.build_cache_relative_path() / bindist.tarball_name(spec, ".spec.json") mirror_url = url_util.path_to_file_url(mirror_dir.strpath)
)
path.parent.mkdir(parents=True)
with open(path, "w", encoding="utf-8") as f:
json.dump(spec_dict, f)
# Configure as a mirror. s = Spec("libdwarf").concretized()
mirror_cmd("add", "test-mirror", str(tmp_path))
# Shouldn't be able "download" this. # Install libdwarf
assert bindist.download_tarball(spec, unsigned=True) is None install_cmd("--fake", s.name)
# And there should be a warning about an unsupported layout version. # Push libdwarf to buildcache
assert f"Layout version {layout_version} is too new" in capsys.readouterr().err buildcache_cmd("push", "-u", mirror_dir.strpath, s.name)
cache_class = get_url_buildcache_class(bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION)
build_cache = cache_class(mirror_url, s, allow_unsigned=True)
manifest = build_cache.read_manifest()
spec_dict = build_cache.fetch_metadata()
local_tarball_path = build_cache.fetch_archive()
assert "spec" in spec_dict
for blob_record in manifest.data:
blob_path = build_cache.get_staged_blob_path(blob_record)
assert os.path.exists(blob_path)
actual_blob_size = os.stat(blob_path).st_size
assert blob_record.content_length == actual_blob_size
build_cache.destroy()
assert not os.path.exists(local_tarball_path)
def test_relative_path_components():
blobs_v3 = URLBuildcacheEntry.get_relative_path_components(BuildcacheComponent.BLOB)
assert len(blobs_v3) == 1
assert "blobs" in blobs_v3
blobs_v2 = URLBuildcacheEntryV2.get_relative_path_components(BuildcacheComponent.BLOB)
assert len(blobs_v2) == 1
assert "build_cache" in blobs_v2
v2_spec_url = "file:///home/me/mymirror/build_cache/linux-ubuntu22.04-sapphirerapids-gcc-12.3.0-gmake-4.4.1-5pddli3htvfe6svs7nbrqmwi5735agi3.spec.json.sig"
assert URLBuildcacheEntryV2.get_base_url(v2_spec_url) == "file:///home/me/mymirror"
v3_manifest_url = "file:///home/me/mymirror/v3/manifests/gmake-4.4.1-5pddli3htvfe6svs7nbrqmwi5735agi3.spec.manifest.json"
assert URLBuildcacheEntry.get_base_url(v3_manifest_url) == "file:///home/me/mymirror"
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -1170,3 +1226,244 @@ def test_download_tarball_with_unsupported_layout_fails(
def test_default_tag(spec: str): def test_default_tag(spec: str):
"""Make sure that computed image tags are valid.""" """Make sure that computed image tags are valid."""
assert re.fullmatch(spack.oci.image.tag, bindist._oci_default_tag(spack.spec.Spec(spec))) assert re.fullmatch(spack.oci.image.tag, bindist._oci_default_tag(spack.spec.Spec(spec)))
class IndexInformation(NamedTuple):
manifest_contents: Dict[str, Any]
index_contents: str
index_hash: str
manifest_path: str
index_path: str
manifest_etag: str
fetched_blob: Callable[[], bool]
@pytest.fixture
def mock_index(tmp_path, monkeypatch) -> IndexInformation:
mirror_root = tmp_path / "mymirror"
index_json = '{"Hello": "World"}'
index_json_hash = bindist.compute_hash(index_json)
fetched = False
cache_class = get_url_buildcache_class(
layout_version=bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
)
index_blob_path = os.path.join(
str(mirror_root),
*cache_class.get_relative_path_components(BuildcacheComponent.BLOB),
"sha256",
index_json_hash[:2],
index_json_hash,
)
os.makedirs(os.path.dirname(index_blob_path))
with open(index_blob_path, "w", encoding="utf-8") as fd:
fd.write(index_json)
index_blob_record = bindist.BlobRecord(
os.stat(index_blob_path).st_size,
cache_class.BUILDCACHE_INDEX_MEDIATYPE,
"none",
"sha256",
index_json_hash,
)
index_manifest = {
"version": cache_class.get_layout_version(),
"data": [index_blob_record.to_dict()],
}
manifest_json_path = cache_class.get_index_url(str(mirror_root))
os.makedirs(os.path.dirname(manifest_json_path))
with open(manifest_json_path, "w", encoding="utf-8") as f:
json.dump(index_manifest, f)
def fetch_patch(stage, mirror_only: bool = False, err_msg: Optional[str] = None):
nonlocal fetched
fetched = True
@property # type: ignore
def save_filename_patch(stage):
return str(index_blob_path)
monkeypatch.setattr(spack.stage.Stage, "fetch", fetch_patch)
monkeypatch.setattr(spack.stage.Stage, "save_filename", save_filename_patch)
def get_did_fetch():
# nonlocal fetched
return fetched
return IndexInformation(
index_manifest,
index_json,
index_json_hash,
manifest_json_path,
index_blob_path,
"59bcc3ad6775562f845953cf01624225",
get_did_fetch,
)
def test_etag_fetching_304():
# Test conditional fetch with etags. If the remote hasn't modified the file
# it returns 304, which is an HTTPError in urllib-land. That should be
# handled as success, since it means the local cache is up-to-date.
def response_304(request: urllib.request.Request):
url = request.get_full_url()
if url.endswith(INDEX_MANIFEST_FILE):
assert request.get_header("If-none-match") == '"112a8bbc1b3f7f185621c1ee335f0502"'
raise urllib.error.HTTPError(
url, 304, "Not Modified", hdrs={}, fp=None # type: ignore[arg-type]
)
assert False, "Unexpected request {}".format(url)
fetcher = bindist.EtagIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_304,
)
result = fetcher.conditional_fetch()
assert isinstance(result, bindist.FetchIndexResult)
assert result.fresh
def test_etag_fetching_200(mock_index):
# Test conditional fetch with etags. The remote has modified the file.
def response_200(request: urllib.request.Request):
url = request.get_full_url()
if url.endswith(INDEX_MANIFEST_FILE):
assert request.get_header("If-none-match") == '"112a8bbc1b3f7f185621c1ee335f0502"'
return urllib.response.addinfourl(
io.BytesIO(json.dumps(mock_index.manifest_contents).encode()),
headers={"Etag": f'"{mock_index.manifest_etag}"'}, # type: ignore[arg-type]
url=url,
code=200,
)
assert False, "Unexpected request {}".format(url)
fetcher = bindist.EtagIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_200,
)
result = fetcher.conditional_fetch()
assert isinstance(result, bindist.FetchIndexResult)
assert not result.fresh
assert mock_index.fetched_blob()
assert result.etag == mock_index.manifest_etag
assert result.data == mock_index.index_contents
assert result.hash == mock_index.index_hash
def test_etag_fetching_404():
# Test conditional fetch with etags. The remote has modified the file.
def response_404(request: urllib.request.Request):
raise urllib.error.HTTPError(
request.get_full_url(),
404,
"Not found",
hdrs={"Etag": '"59bcc3ad6775562f845953cf01624225"'}, # type: ignore[arg-type]
fp=None,
)
fetcher = bindist.EtagIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
etag="112a8bbc1b3f7f185621c1ee335f0502",
urlopen=response_404,
)
with pytest.raises(bindist.FetchIndexError):
fetcher.conditional_fetch()
def test_default_index_fetch_200(mock_index):
# We fetch the manifest and then the index blob if the hash is outdated
def urlopen(request: urllib.request.Request):
url = request.get_full_url()
if url.endswith(INDEX_MANIFEST_FILE):
return urllib.response.addinfourl( # type: ignore[arg-type]
io.BytesIO(json.dumps(mock_index.manifest_contents).encode()),
headers={"Etag": f'"{mock_index.manifest_etag}"'}, # type: ignore[arg-type]
url=url,
code=200,
)
assert False, "Unexpected request {}".format(url)
fetcher = bindist.DefaultIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
local_hash="outdated",
urlopen=urlopen,
)
result = fetcher.conditional_fetch()
assert isinstance(result, bindist.FetchIndexResult)
assert not result.fresh
assert mock_index.fetched_blob()
assert result.etag == mock_index.manifest_etag
assert result.data == mock_index.index_contents
assert result.hash == mock_index.index_hash
def test_default_index_404():
# We get a fetch error if the index can't be fetched
def urlopen(request: urllib.request.Request):
raise urllib.error.HTTPError(
request.get_full_url(),
404,
"Not found",
hdrs={"Etag": '"59bcc3ad6775562f845953cf01624225"'}, # type: ignore[arg-type]
fp=None,
)
fetcher = bindist.DefaultIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
local_hash=None,
urlopen=urlopen,
)
with pytest.raises(bindist.FetchIndexError):
fetcher.conditional_fetch()
def test_default_index_not_modified(mock_index):
# We don't fetch the index blob if hash didn't change
def urlopen(request: urllib.request.Request):
url = request.get_full_url()
if url.endswith(INDEX_MANIFEST_FILE):
return urllib.response.addinfourl(
io.BytesIO(json.dumps(mock_index.manifest_contents).encode()),
headers={}, # type: ignore[arg-type]
url=url,
code=200,
)
# No other request should be made.
assert False, "Unexpected request {}".format(url)
fetcher = bindist.DefaultIndexFetcher(
bindist.MirrorURLAndVersion(
"https://www.example.com", bindist.CURRENT_BUILD_CACHE_LAYOUT_VERSION
),
local_hash=mock_index.index_hash,
urlopen=urlopen,
)
assert fetcher.conditional_fetch().fresh
assert not mock_index.fetched_blob()

View File

@ -2,7 +2,7 @@
# #
# SPDX-License-Identifier: (Apache-2.0 OR MIT) # SPDX-License-Identifier: (Apache-2.0 OR MIT)
import os import shutil
import pytest import pytest
@ -37,12 +37,7 @@ def test_build_tarball_overwrite(install_mockery, mock_fetch, monkeypatch, tmp_p
assert not skipped assert not skipped
# Remove the tarball, which should cause push to push. # Remove the tarball, which should cause push to push.
os.remove( shutil.rmtree(tmp_path / bd.buildcache_relative_blobs_path())
tmp_path
/ bd.BUILD_CACHE_RELATIVE_PATH
/ bd.tarball_directory_name(spec)
/ bd.tarball_name(spec, ".spack")
)
with bd.make_uploader(mirror) as uploader: with bd.make_uploader(mirror) as uploader:
skipped = uploader.push_or_raise(specs) skipped = uploader.push_or_raise(specs)

View File

@ -5,12 +5,16 @@
import errno import errno
import json import json
import os import os
import pathlib
import shutil import shutil
from typing import List from typing import List
import pytest import pytest
from llnl.util.filesystem import copy_tree, find
import spack.binary_distribution import spack.binary_distribution
import spack.buildcache_migrate as migrate
import spack.cmd.buildcache import spack.cmd.buildcache
import spack.concretize import spack.concretize
import spack.environment as ev import spack.environment as ev
@ -18,8 +22,16 @@
import spack.main import spack.main
import spack.mirrors.mirror import spack.mirrors.mirror
import spack.spec import spack.spec
import spack.util.url import spack.util.url as url_util
from spack.installer import PackageInstaller from spack.installer import PackageInstaller
from spack.paths import test_path
from spack.url_buildcache import (
BuildcacheComponent,
URLBuildcacheEntry,
URLBuildcacheEntryV2,
check_mirror_for_layout,
get_url_buildcache_class,
)
buildcache = spack.main.SpackCommand("buildcache") buildcache = spack.main.SpackCommand("buildcache")
install = spack.main.SpackCommand("install") install = spack.main.SpackCommand("install")
@ -74,20 +86,6 @@ def test_buildcache_list_allarch(database, mock_get_specs_multiarch, capsys):
assert output.count("mpileaks") == 2 assert output.count("mpileaks") == 2
def tests_buildcache_create(install_mockery, mock_fetch, monkeypatch, tmpdir):
""" "Ensure that buildcache create creates output files"""
pkg = "trivial-install-test-package"
install(pkg)
buildcache("push", "--unsigned", str(tmpdir), pkg)
spec = spack.concretize.concretize_one(pkg)
tarball_path = spack.binary_distribution.tarball_path_name(spec, ".spack")
tarball = spack.binary_distribution.tarball_name(spec, ".spec.json")
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball_path))
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball))
def tests_buildcache_create_env( def tests_buildcache_create_env(
install_mockery, mock_fetch, monkeypatch, tmpdir, mutable_mock_env_path install_mockery, mock_fetch, monkeypatch, tmpdir, mutable_mock_env_path
): ):
@ -102,10 +100,15 @@ def tests_buildcache_create_env(
buildcache("push", "--unsigned", str(tmpdir)) buildcache("push", "--unsigned", str(tmpdir))
spec = spack.concretize.concretize_one(pkg) spec = spack.concretize.concretize_one(pkg)
tarball_path = spack.binary_distribution.tarball_path_name(spec, ".spack")
tarball = spack.binary_distribution.tarball_name(spec, ".spec.json") mirror_url = f"file://{tmpdir.strpath}"
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball_path))
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball)) cache_class = get_url_buildcache_class(
layout_version=spack.binary_distribution.CURRENT_BUILD_CACHE_LAYOUT_VERSION
)
cache_entry = cache_class(mirror_url, spec, allow_unsigned=True)
assert cache_entry.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
cache_entry.destroy()
def test_buildcache_create_fails_on_noargs(tmpdir): def test_buildcache_create_fails_on_noargs(tmpdir):
@ -159,12 +162,14 @@ def test_update_key_index(
# it causes the index to get update. # it causes the index to get update.
buildcache("update-index", "--keys", mirror_dir.strpath) buildcache("update-index", "--keys", mirror_dir.strpath)
key_dir_list = os.listdir(os.path.join(mirror_dir.strpath, "build_cache", "_pgp")) key_dir_list = os.listdir(
os.path.join(mirror_dir.strpath, spack.binary_distribution.buildcache_relative_keys_path())
)
uninstall("-y", s.name) uninstall("-y", s.name)
mirror("rm", "test-mirror") mirror("rm", "test-mirror")
assert "index.json" in key_dir_list assert "keys.manifest.json" in key_dir_list
def test_buildcache_autopush(tmp_path, install_mockery, mock_fetch): def test_buildcache_autopush(tmp_path, install_mockery, mock_fetch):
@ -180,10 +185,14 @@ def test_buildcache_autopush(tmp_path, install_mockery, mock_fetch):
# Install and generate build cache index # Install and generate build cache index
PackageInstaller([s.package], fake=True, explicit=True).install() PackageInstaller([s.package], fake=True, explicit=True).install()
metadata_file = spack.binary_distribution.tarball_name(s, ".spec.json") assert s.name is not None
manifest_file = URLBuildcacheEntry.get_manifest_filename(s)
specs_dirs = os.path.join(
*URLBuildcacheEntry.get_relative_path_components(BuildcacheComponent.SPEC), s.name
)
assert not (mirror_dir / "build_cache" / metadata_file).exists() assert not (mirror_dir / specs_dirs / manifest_file).exists()
assert (mirror_autopush_dir / "build_cache" / metadata_file).exists() assert (mirror_autopush_dir / specs_dirs / manifest_file).exists()
def test_buildcache_sync( def test_buildcache_sync(
@ -205,7 +214,11 @@ def test_buildcache_sync(
out_env_pkg = "libdwarf" out_env_pkg = "libdwarf"
def verify_mirror_contents(): def verify_mirror_contents():
dest_list = os.listdir(os.path.join(dest_mirror_dir, "build_cache")) dest_list = os.listdir(
os.path.join(
dest_mirror_dir, spack.binary_distribution.buildcache_relative_specs_path()
)
)
found_pkg = False found_pkg = False
@ -252,33 +265,15 @@ def verify_mirror_contents():
verify_mirror_contents() verify_mirror_contents()
shutil.rmtree(dest_mirror_dir) shutil.rmtree(dest_mirror_dir)
cache_class = get_url_buildcache_class(
layout_version=spack.binary_distribution.CURRENT_BUILD_CACHE_LAYOUT_VERSION
)
def manifest_insert(manifest, spec, dest_url): def manifest_insert(manifest, spec, dest_url):
manifest[spec.dag_hash()] = [ manifest[spec.dag_hash()] = {
{ "src": cache_class.get_manifest_url(spec, src_mirror_url),
"src": spack.util.url.join( "dest": cache_class.get_manifest_url(spec, dest_url),
src_mirror_url, }
spack.binary_distribution.build_cache_relative_path(),
spack.binary_distribution.tarball_name(spec, ".spec.json"),
),
"dest": spack.util.url.join(
dest_url,
spack.binary_distribution.build_cache_relative_path(),
spack.binary_distribution.tarball_name(spec, ".spec.json"),
),
},
{
"src": spack.util.url.join(
src_mirror_url,
spack.binary_distribution.build_cache_relative_path(),
spack.binary_distribution.tarball_path_name(spec, ".spack"),
),
"dest": spack.util.url.join(
dest_url,
spack.binary_distribution.build_cache_relative_path(),
spack.binary_distribution.tarball_path_name(spec, ".spack"),
),
},
]
manifest_file = os.path.join(tmpdir.strpath, "manifest_dest.json") manifest_file = os.path.join(tmpdir.strpath, "manifest_dest.json")
with open(manifest_file, "w", encoding="utf-8") as fd: with open(manifest_file, "w", encoding="utf-8") as fd:
@ -298,9 +293,7 @@ def manifest_insert(manifest, spec, dest_url):
with open(manifest_file, "w", encoding="utf-8") as fd: with open(manifest_file, "w", encoding="utf-8") as fd:
manifest = {} manifest = {}
for spec in test_env.specs_by_hash.values(): for spec in test_env.specs_by_hash.values():
manifest_insert( manifest_insert(manifest, spec, url_util.join(dest_mirror_url, "invalid_path"))
manifest, spec, spack.util.url.join(dest_mirror_url, "invalid_path")
)
json.dump(manifest, fd) json.dump(manifest, fd)
# Trigger the warning # Trigger the warning
@ -327,11 +320,37 @@ def test_buildcache_create_install(
buildcache("push", "--unsigned", str(tmpdir), pkg) buildcache("push", "--unsigned", str(tmpdir), pkg)
mirror_url = f"file://{tmpdir.strpath}"
spec = spack.concretize.concretize_one(pkg) spec = spack.concretize.concretize_one(pkg)
tarball_path = spack.binary_distribution.tarball_path_name(spec, ".spack") cache_class = get_url_buildcache_class(
tarball = spack.binary_distribution.tarball_name(spec, ".spec.json") layout_version=spack.binary_distribution.CURRENT_BUILD_CACHE_LAYOUT_VERSION
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball_path)) )
assert os.path.exists(os.path.join(str(tmpdir), "build_cache", tarball)) cache_entry = cache_class(mirror_url, spec, allow_unsigned=True)
assert spec.name is not None
manifest_path = os.path.join(
str(tmpdir),
*cache_class.get_relative_path_components(BuildcacheComponent.SPEC),
spec.name,
cache_class.get_manifest_filename(spec),
)
assert os.path.exists(manifest_path)
cache_entry.read_manifest()
spec_blob_record = cache_entry.get_blob_record(BuildcacheComponent.SPEC)
tarball_blob_record = cache_entry.get_blob_record(BuildcacheComponent.TARBALL)
spec_blob_path = os.path.join(
tmpdir.strpath, *cache_class.get_blob_path_components(spec_blob_record)
)
assert os.path.exists(spec_blob_path)
tarball_blob_path = os.path.join(
tmpdir.strpath, *cache_class.get_blob_path_components(tarball_blob_record)
)
assert os.path.exists(tarball_blob_path)
cache_entry.destroy()
@pytest.mark.parametrize( @pytest.mark.parametrize(
@ -503,3 +522,230 @@ def test_push_without_build_deps(tmp_path, temporary_store, mock_packages, mutab
"push", "--update-index", "--without-build-dependencies", "my-mirror", f"/{s.dag_hash()}" "push", "--update-index", "--without-build-dependencies", "my-mirror", f"/{s.dag_hash()}"
) )
assert spack.binary_distribution.update_cache_and_get_specs() == [s] assert spack.binary_distribution.update_cache_and_get_specs() == [s]
@pytest.fixture(scope="function")
def v2_buildcache_layout(tmp_path):
def _layout(signedness: str = "signed"):
source_path = str(pathlib.Path(test_path) / "data" / "mirrors" / "v2_layout" / signedness)
test_mirror_path = tmp_path / "mirror"
copy_tree(source_path, test_mirror_path)
return test_mirror_path
return _layout
def test_check_mirror_for_layout(v2_buildcache_layout, mutable_config, capsys):
"""Check printed warning in the presence of v2 layout binary mirrors"""
test_mirror_path = v2_buildcache_layout("unsigned")
check_mirror_for_layout(spack.mirrors.mirror.Mirror.from_local_path(str(test_mirror_path)))
err = str(capsys.readouterr()[1])
assert all([word in err for word in ["Warning", "missing", "layout"]])
def test_url_buildcache_entry_v2_exists(
capsys, v2_buildcache_layout, mock_packages, mutable_config
):
"""Test existence check for v2 buildcache entries"""
test_mirror_path = v2_buildcache_layout("unsigned")
mirror_url = f"file://{test_mirror_path}"
mirror("add", "v2mirror", mirror_url)
with capsys.disabled():
output = buildcache("list", "-a", "-l")
assert "Fetching an index from a v2 binary mirror layout" in output
assert "is deprecated" in output
v2_cache_class = URLBuildcacheEntryV2
# If you don't give it a spec, it returns False
build_cache = v2_cache_class(mirror_url)
assert not build_cache.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
spec = spack.concretize.concretize_one("libdwarf")
# In v2 we have to ask for both, because we need to have the spec to have the tarball
build_cache = v2_cache_class(mirror_url, spec, allow_unsigned=True)
assert not build_cache.exists([BuildcacheComponent.TARBALL])
assert not build_cache.exists([BuildcacheComponent.SPEC])
# But if we do ask for both, they should be there in this case
assert build_cache.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
spec_path = build_cache._get_spec_url(spec, mirror_url, ext=".spec.json")[7:]
tarball_path = build_cache._get_tarball_url(spec, mirror_url)[7:]
os.remove(tarball_path)
build_cache = v2_cache_class(mirror_url, spec, allow_unsigned=True)
assert not build_cache.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
os.remove(spec_path)
build_cache = v2_cache_class(mirror_url, spec, allow_unsigned=True)
assert not build_cache.exists([BuildcacheComponent.SPEC, BuildcacheComponent.TARBALL])
@pytest.mark.parametrize("signing", ["unsigned", "signed"])
def test_install_v2_layout(
signing,
capsys,
v2_buildcache_layout,
mock_packages,
mutable_config,
mutable_mock_env_path,
install_mockery,
mock_gnupghome,
monkeypatch,
):
"""Ensure we can still install from signed and unsigned v2 buildcache"""
test_mirror_path = v2_buildcache_layout(signing)
mirror("add", "my-mirror", str(test_mirror_path))
# Trust original signing key (no-op if this is the unsigned pass)
buildcache("keys", "--install", "--trust")
with capsys.disabled():
output = install("--fake", "--no-check-signature", "libdwarf")
assert "Extracting libelf" in output
assert "libelf: Successfully installed" in output
assert "Extracting libdwarf" in output
assert "libdwarf: Successfully installed" in output
assert "Installing a spec from a v2 binary mirror layout" in output
assert "is deprecated" in output
def test_basic_migrate_unsigned(capsys, v2_buildcache_layout, mutable_config):
"""Make sure first unsigned migration results in usable buildcache,
leaving the previous layout in place. Also test that a subsequent one
doesn't need to migrate anything, and that using --delete-existing
removes the previous layout"""
test_mirror_path = v2_buildcache_layout("unsigned")
mirror("add", "my-mirror", str(test_mirror_path))
with capsys.disabled():
output = buildcache("migrate", "--unsigned", "my-mirror")
# The output indicates both specs were migrated
assert output.count("Successfully migrated") == 6
build_cache_path = str(test_mirror_path / "build_cache")
# Without "--delete-existing" and "--yes-to-all", migration leaves the
# previous layout in place
assert os.path.exists(build_cache_path)
assert os.path.isdir(build_cache_path)
# Now list the specs available under the new layout
with capsys.disabled():
output = buildcache("list", "--allarch")
assert "libdwarf" in output and "libelf" in output
with capsys.disabled():
output = buildcache(
"migrate", "--unsigned", "--delete-existing", "--yes-to-all", "my-mirror"
)
# A second migration of the same mirror indicates neither spec
# needs to be migrated
assert output.count("No need to migrate") == 6
# When we provide "--delete-existing" and "--yes-to-all", migration
# removes the old layout
assert not os.path.exists(build_cache_path)
def test_basic_migrate_signed(
capsys, v2_buildcache_layout, monkeypatch, mock_gnupghome, mutable_config
):
"""Test a signed migration requires a signing key, requires the public
key originally used to sign the pkgs, fails and prints reasonable messages
if those requirements are unmet, and eventually succeeds when they are met."""
test_mirror_path = v2_buildcache_layout("signed")
mirror("add", "my-mirror", str(test_mirror_path))
with pytest.raises(migrate.MigrationException) as error:
buildcache("migrate", "my-mirror")
# Without a signing key spack fails and explains why
assert error.value.message == "Signed migration requires exactly one secret key in keychain"
# Create a signing key and trust the key used to sign the pkgs originally
gpg("create", "New Test Signing Key", "noone@nowhere.org")
with capsys.disabled():
output = buildcache("migrate", "my-mirror")
# Without trusting the original signing key, spack fails with an explanation
assert "Failed to verify signature of libelf" in output
assert "Failed to verify signature of libdwarf" in output
assert "did you mean to perform an unsigned migration" in output
# Trust original signing key (since it's in the original layout location,
# this is where the monkeypatched attribute is used)
with capsys.disabled():
output = buildcache("keys", "--install", "--trust")
with capsys.disabled():
output = buildcache("migrate", "my-mirror")
# Once we have the proper keys, migration should succeed
assert "Successfully migrated libelf" in output
assert "Successfully migrated libelf" in output
# Now list the specs available under the new layout
with capsys.disabled():
output = buildcache("list", "--allarch")
assert "libdwarf" in output and "libelf" in output
def test_unsigned_migrate_of_signed_mirror(capsys, v2_buildcache_layout, mutable_config):
"""Test spack can do an unsigned migration of a signed buildcache by
ignoring signatures and skipping re-signing."""
test_mirror_path = v2_buildcache_layout("signed")
mirror("add", "my-mirror", str(test_mirror_path))
with capsys.disabled():
output = buildcache(
"migrate", "--unsigned", "--delete-existing", "--yes-to-all", "my-mirror"
)
# Now list the specs available under the new layout
with capsys.disabled():
output = buildcache("list", "--allarch")
assert "libdwarf" in output and "libelf" in output
# We should find two spec manifest files, one for each spec
file_list = find(test_mirror_path, "*.spec.manifest.json")
assert len(file_list) == 6
assert any(["libdwarf" in file for file in file_list])
assert any(["libelf" in file for file in file_list])
# The two spec manifest files should be unsigned
for file_path in file_list:
with open(file_path, "r", encoding="utf-8") as fd:
assert json.load(fd)
def test_migrate_requires_index(capsys, v2_buildcache_layout, mutable_config):
"""Test spack fails with a reasonable error message when mirror does
not have an index"""
test_mirror_path = v2_buildcache_layout("unsigned")
v2_index_path = test_mirror_path / "build_cache" / "index.json"
v2_index_hash_path = test_mirror_path / "build_cache" / "index.json.hash"
os.remove(str(v2_index_path))
os.remove(str(v2_index_hash_path))
mirror("add", "my-mirror", str(test_mirror_path))
with pytest.raises(migrate.MigrationException) as error:
buildcache("migrate", "--unsigned", "my-mirror")
# If the buildcache has no index, spack fails and explains why
assert error.value.message == "Buildcache migration requires a buildcache index"

View File

@ -31,11 +31,8 @@
from spack.ci.common import PipelineDag, PipelineOptions, SpackCIConfig from spack.ci.common import PipelineDag, PipelineOptions, SpackCIConfig
from spack.ci.generator_registry import generator from spack.ci.generator_registry import generator
from spack.cmd.ci import FAILED_CREATE_BUILDCACHE_CODE from spack.cmd.ci import FAILED_CREATE_BUILDCACHE_CODE
from spack.database import INDEX_JSON_FILE
from spack.error import SpackError from spack.error import SpackError
from spack.schema.buildcache_spec import schema as specfile_schema
from spack.schema.database_index import schema as db_idx_schema from spack.schema.database_index import schema as db_idx_schema
from spack.spec import Spec
from spack.test.conftest import MockHTTPResponse from spack.test.conftest import MockHTTPResponse
config_cmd = spack.main.SpackCommand("config") config_cmd = spack.main.SpackCommand("config")
@ -718,7 +715,7 @@ def test_ci_nothing_to_rebuild(
) )
install_cmd("archive-files") install_cmd("archive-files")
buildcache_cmd("push", "-f", "-u", mirror_url, "archive-files") buildcache_cmd("push", "-f", "-u", "--update-index", mirror_url, "archive-files")
with working_dir(tmp_path): with working_dir(tmp_path):
env_cmd("create", "test", "./spack.yaml") env_cmd("create", "test", "./spack.yaml")
@ -855,18 +852,18 @@ def test_push_to_build_cache(
# Test generating buildcache index while we have bin mirror # Test generating buildcache index while we have bin mirror
buildcache_cmd("update-index", mirror_url) buildcache_cmd("update-index", mirror_url)
with open(mirror_dir / "build_cache" / INDEX_JSON_FILE, encoding="utf-8") as idx_fd:
index_object = json.load(idx_fd) # Validate resulting buildcache (database) index
jsonschema.validate(index_object, db_idx_schema) layout_version = spack.binary_distribution.CURRENT_BUILD_CACHE_LAYOUT_VERSION
url_and_version = spack.binary_distribution.MirrorURLAndVersion(
mirror_url, layout_version
)
index_fetcher = spack.binary_distribution.DefaultIndexFetcher(url_and_version, None)
result = index_fetcher.conditional_fetch()
jsonschema.validate(json.loads(result.data), db_idx_schema)
# Now that index is regenerated, validate "buildcache list" output # Now that index is regenerated, validate "buildcache list" output
assert "patchelf" in buildcache_cmd("list", output=str) assert "patchelf" in buildcache_cmd("list", output=str)
# Also test buildcache_spec schema
for file_name in os.listdir(mirror_dir / "build_cache"):
if file_name.endswith(".spec.json.sig"):
with open(mirror_dir / "build_cache" / file_name, encoding="utf-8") as f:
spec_dict = Spec.extract_json_from_clearsig(f.read())
jsonschema.validate(spec_dict, specfile_schema)
logs_dir = scratch / "logs_dir" logs_dir = scratch / "logs_dir"
logs_dir.mkdir() logs_dir.mkdir()
@ -1032,7 +1029,7 @@ def test_ci_generate_override_runner_attrs(
def test_ci_rebuild_index( def test_ci_rebuild_index(
tmp_path: pathlib.Path, working_env, mutable_mock_env_path, install_mockery, mock_fetch tmp_path: pathlib.Path, working_env, mutable_mock_env_path, install_mockery, mock_fetch, capsys
): ):
scratch = tmp_path / "working_dir" scratch = tmp_path / "working_dir"
mirror_dir = scratch / "mirror" mirror_dir = scratch / "mirror"
@ -1069,8 +1066,9 @@ def test_ci_rebuild_index(
buildcache_cmd("push", "-u", "-f", mirror_url, "callpath") buildcache_cmd("push", "-u", "-f", mirror_url, "callpath")
ci_cmd("rebuild-index") ci_cmd("rebuild-index")
with open(mirror_dir / "build_cache" / INDEX_JSON_FILE, encoding="utf-8") as f: with capsys.disabled():
jsonschema.validate(json.load(f), db_idx_schema) output = buildcache_cmd("list", "--allarch")
assert "callpath" in output
def test_ci_get_stack_changed(mock_git_repo, monkeypatch): def test_ci_get_stack_changed(mock_git_repo, monkeypatch):

View File

@ -8,6 +8,7 @@
import llnl.util.filesystem as fs import llnl.util.filesystem as fs
import spack.binary_distribution as bindist
import spack.util.executable import spack.util.executable
import spack.util.gpg import spack.util.gpg
from spack.main import SpackCommand from spack.main import SpackCommand
@ -172,23 +173,25 @@ def test_gpg(tmpdir, mutable_config, mock_gnupghome):
# Verification should now succeed again. # Verification should now succeed again.
gpg("verify", str(test_path)) gpg("verify", str(test_path))
relative_keys_path = bindist.buildcache_relative_keys_path()
# Publish the keys using a directory path # Publish the keys using a directory path
test_path = tmpdir.join("dir_cache") test_path = tmpdir.join("dir_cache")
os.makedirs("%s" % test_path) os.makedirs(f"{test_path}")
gpg("publish", "--rebuild-index", "-d", str(test_path)) gpg("publish", "--rebuild-index", "-d", str(test_path))
assert os.path.exists("%s/build_cache/_pgp/index.json" % test_path) assert os.path.exists(f"{test_path}/{relative_keys_path}/keys.manifest.json")
# Publish the keys using a mirror url # Publish the keys using a mirror url
test_path = tmpdir.join("url_cache") test_path = tmpdir.join("url_cache")
os.makedirs("%s" % test_path) os.makedirs(f"{test_path}")
test_url = "file://%s" % test_path test_url = f"file://{test_path}"
gpg("publish", "--rebuild-index", "--mirror-url", test_url) gpg("publish", "--rebuild-index", "--mirror-url", test_url)
assert os.path.exists("%s/build_cache/_pgp/index.json" % test_path) assert os.path.exists(f"{test_path}/{relative_keys_path}/keys.manifest.json")
# Publish the keys using a mirror name # Publish the keys using a mirror name
test_path = tmpdir.join("named_cache") test_path = tmpdir.join("named_cache")
os.makedirs("%s" % test_path) os.makedirs(f"{test_path}")
mirror_url = "file://%s" % test_path mirror_url = f"file://{test_path}"
mirror("add", "gpg", mirror_url) mirror("add", "gpg", mirror_url)
gpg("publish", "--rebuild-index", "-m", "gpg") gpg("publish", "--rebuild-index", "-m", "gpg")
assert os.path.exists("%s/build_cache/_pgp/index.json" % test_path) assert os.path.exists(f"{test_path}/{relative_keys_path}/keys.manifest.json")

View File

@ -6,6 +6,7 @@
import pytest import pytest
import spack.binary_distribution as bindist
import spack.cmd.mirror import spack.cmd.mirror
import spack.concretize import spack.concretize
import spack.config import spack.config
@ -365,8 +366,10 @@ def test_mirror_destroy(
install("--fake", "--no-cache", spec_name) install("--fake", "--no-cache", spec_name)
buildcache("push", "-u", "-f", mirror_dir.strpath, spec_name) buildcache("push", "-u", "-f", mirror_dir.strpath, spec_name)
blobs_path = bindist.buildcache_relative_blobs_path()
contents = os.listdir(mirror_dir.strpath) contents = os.listdir(mirror_dir.strpath)
assert "build_cache" in contents assert blobs_path in contents
# Destroy mirror by name # Destroy mirror by name
mirror("destroy", "-m", "atest") mirror("destroy", "-m", "atest")
@ -376,7 +379,7 @@ def test_mirror_destroy(
buildcache("push", "-u", "-f", mirror_dir.strpath, spec_name) buildcache("push", "-u", "-f", mirror_dir.strpath, spec_name)
contents = os.listdir(mirror_dir.strpath) contents = os.listdir(mirror_dir.strpath)
assert "build_cache" in contents assert blobs_path in contents
# Destroy mirror by url # Destroy mirror by url
mirror("destroy", "--mirror-url", mirror_url) mirror("destroy", "--mirror-url", mirror_url)

View File

@ -1068,9 +1068,7 @@ def install_mockery(temporary_store: spack.store.Store, mutable_config, mock_pac
@pytest.fixture(scope="module") @pytest.fixture(scope="module")
def temporary_mirror_dir(tmpdir_factory): def temporary_mirror_dir(tmpdir_factory):
dir = tmpdir_factory.mktemp("mirror") dir = tmpdir_factory.mktemp("mirror")
dir.ensure("build_cache", dir=True)
yield str(dir) yield str(dir)
dir.join("build_cache").remove()
@pytest.fixture(scope="function") @pytest.fixture(scope="function")
@ -1084,9 +1082,7 @@ def temporary_mirror(temporary_mirror_dir):
@pytest.fixture(scope="function") @pytest.fixture(scope="function")
def mutable_temporary_mirror_dir(tmpdir_factory): def mutable_temporary_mirror_dir(tmpdir_factory):
dir = tmpdir_factory.mktemp("mirror") dir = tmpdir_factory.mktemp("mirror")
dir.ensure("build_cache", dir=True)
yield str(dir) yield str(dir)
dir.join("build_cache").remove()
@pytest.fixture(scope="function") @pytest.fixture(scope="function")

View File

@ -1,54 +0,0 @@
{
"spec": {
"_meta": {
"version": 3
},
"nodes": [
{
"name": "archive-files",
"version": "2.0",
"arch": {
"platform": "test",
"platform_os": "debian6",
"target": {
"name": "core2",
"vendor": "GenuineIntel",
"features": [
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation": 0,
"parents": [
"nocona"
]
}
},
"compiler": {
"name": "gcc",
"version": "4.5.0"
},
"namespace": "builtin.mock",
"parameters": {
"cflags": [],
"cppflags": [],
"cxxflags": [],
"fflags": [],
"ldflags": [],
"ldlibs": []
},
"package_hash": "ncv2pr4o2yemepsa4h7u4p4dsgieul5fxvh6s5am5fsb65ebugaa====",
"hash": "l3vdiqvbobmspwyb4q2b62fz6nitd4hk"
}
]
},
"binary_cache_checksum": {
"hash_algorithm": "sha256",
"hash": "c226b51d88876746efd6f9737cc6dfdd349870b6c0b9c045d9bad0f2764a40b9"
},
"buildinfo": {
"relative_prefix": "test-debian6-core2/gcc-4.5.0/archive-files-2.0-l3vdiqvbobmspwyb4q2b62fz6nitd4hk",
"relative_rpaths": false
}
}

View File

@ -0,0 +1,29 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGf23+EBEAC6UqaiE43cF9jFuVjA8xJ5j31BMhufpnk0cwoE5Iks/GgR/Hki
LMYbzy36V7TZGObel+5DtFKipX+WCwWj2XsjbeqHeuCkxZhzHFwfi1UJl9FO2T28
iNn6OsBiGeU6ULNmehSia2hx0uhj1re/FUwJExOAvuYv8nc7M+nozqi7Pp/WjP8v
UTiqP2onzZJbidlSBvmZ2nheWk7G78e617gcV/ye+UyXZvciiF2UQBg9YV6D8JuD
YhBbNAVOzJOiyOdTBmZmOkmYsGx58sEbFVqGeOMB0xoxZrqKjMm9NhvjqjJF/sWs
hN/PD5ylW1UR05/fGxlG2GLKKfBInbdqnC101OFWXP5HenYHmKaBJoCKCAUfsoJ0
r/t/GVh3z3w/99p0TRDONnTecKm5S9z3/5QjjE5RsWcd4ll7mRikUiVpe1WhKRwT
4T76pQLq3XwNJqiOmuMQuSHoBE9OMufvRFiTYC0QHyLoCV2H5PCWtS2xSsIDN4PB
0RNd0hnHKanVV7d2TkIrGOagoAo0wXqyW/Op6KUG1NdaFYYziDFEHeZxfGoPKytO
iS5PEwZG2FqambAZhJU5OXwzgnCRIoE5DCZad4YS6U5YD/2zg+RrQ/5GUxl5Cc+W
Zwesn9FV5jywx/oFePYbTSNQVPQ6jbUDvhmHvZ8c/OfGOVXQr0VpvfIwdwARAQAB
tD1UZXN0IFNpZ25pbmcgS2V5IChHUEcgY3JlYXRlZCBmb3IgU3BhY2spIDxub2Jv
ZHlAbm93aGVyZS5jb20+iQJRBBMBCAA7FiEEqYoEuILhnYX9Nu4GlWXYCwVckv8F
Amf23+ECGwMFCwkIBwICIgIGFQoJCAsCBBYCAwECHgcCF4AACgkQlWXYCwVckv9i
pg//eGjBR9ph9hUYRsekzKWM1xB5zFOFfNoqlpCut/W7LAfy0XXkFy/y6EvPdcgn
lLWRWPsOFfsKGwZd7LgSovhEMQ2MRsAUUB/KNZx7s6vO/P773PmJspF3odQ/lcrM
1fum2lShChWqimdBdNLrXxG+8duO9uWaMBIp28diBCyB25M/MqpHtKYu00FB/QJ6
ZwQH4OsgXVQHRjyrtIGx/2FQoWt0ah3eJMJCEw46GgkgiojtoTfXQQc4fIJP324b
O1sxz5lx3xVBG/EZYzyV3xnSoG9aZNJ1cJq8EKO7ZoNKc/8jwkVu5gewGaXYI0LK
/WkOeiXcSHPMSdu7TpnitvLYFCjc9YAEKQnjooXdt7+BElwC3+5hZJNXEnoGPMzn
3UL60sQE/ViCsGcW+l9rtzXPNTmLMjEg4rGRqOhX+UmwyhvGD2QYbZtXlayu5xn+
5m/PfmdqgL1xsdvNsLo/BOo+6kizMdBk48Xfp0YM8AC4BzUEENypGzC4T0WYF0k1
Jfc6/eSwiytIcIkJ42GlaVfEFE8UxfYc1/2zqTBN9EdzWJqy0Bh+mVOgOaeb0Dzi
xWpUpChi1fBB3PXWJ5iAS/w0HSVn4G5/JAIEFAs7r6ju2YtKBfuk+u/K5Q28mo7W
6LrZQywN44nBMTvSQUhhXpSNYG+juyotXJUJ3F2u9Cf/jVU=
=TkbL
-----END PGP PUBLIC KEY BLOCK-----

View File

@ -0,0 +1,29 @@
-----BEGIN PGP PUBLIC KEY BLOCK-----
mQINBGfHlp4BEAC5wkZSHqF9z6GcymuHpk1m9aNXCJdt4ZWvE8ck8GcuVu1nbzlZ
h959jqtwk7nFMki5YaNMz6jcQf0eeS75viL4CoPAqFiVyhyCCh5am75h9F7vTBq6
190017lhu9IgkAkiklnjfDbyXH+BwqJ78nXp6e6R4ShFMHNGGvYLem1wmPKzqPlZ
zN0yjc0+d5pw4hu+IEFrM63yqGp2BVX1X132IKUEcROCQt1QOma5oORhYEtSCieX
PuhuHJOA7q6nJuFccPCs5OcDS4IbQgGAbWL4L1+LAGVLVGpK4IVtqEZ831Srclh8
0ruyFFeV/hqOONThwwile0Jwh5Jz/2sYxT5c+nlumXWK+CXTm4OCfGt1UuGy6c6u
Rz84PHfanbKnATp6RUjz4DMREkmA6qBnUFqGLLGaBKBsm42b7kbo7m5aeItuOwLE
U7AcnBEqqHLfI7O1zrHKjQCxhEWP/iok0kgEdiJ4tlPhfDjQRG6thlmZnVdt/08V
+bvVkbYZyWPzjbG3QHyFew1+uzPHb2UopgpByVKYEWhCgNfcFtE56lEI9c40Ba5o
LaZl0VlgfSLP4c+LoFB6gZp1gcVQuPo1JKd1v5WP60f1iHhazL5LEeMYcW6kvujK
58Q683gSH5DsVAnxaj1uU4nvtKDh8IF1CNKKXk8RVsltdpv9bGhV8b4qVQARAQAB
tD1UZXN0IFNpZ25pbmcgS2V5IChHUEcgY3JlYXRlZCBmb3IgU3BhY2spIDxub2Jv
ZHlAbm93aGVyZS5jb20+iQJOBBMBCgA4FiEE6J1JcfAJex56PrVzcbSEgC54180F
AmfHlp4CGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQcbSEgC54180aDg//
f7GqIW5LzYqIqkey+IjdkSSfeD47tlWc2ukKYStHu0gTlHhrUp4rHNJ/s8XQ1o6o
jwzWfNMYh68wt9sjuM2BEkkh3RUFEjVqqW+k562gS5ibfKTDtJb2Yj0n/CQKWvoi
vUUzO88xW0AnZFieP+vD5iI5Zw4H2dY8cH4X1XlWAJufFdH4WBaZjujNwNOcCsnd
w2nE050wKTR2wroWq0HKn1Ni3QNtKWPpLoHGAlhW6ACLa+EFqxHU6D3KhW6IV4Jc
sdt36nHNiRiy6nT99asqtN6Z0Yw+EnQSuIDosIbmSgZoieINh0gU6AKwgydxLUxL
Cu1w2fZHGuFR/ym0c/tTpM893DxHMc/EZ/SpU8fXkC9lYnQO3or/Y0mLHd0kSEv7
XoonvcOu1tOQzmvrvUQUtTn4+6OKpGViyZG5C8Lbk8/yKWFv5b+Gpss/EiGTHSsk
bPTHf5jMsWElv0GgFq2TpybtIcY52yJoZ1fBMEA9Nk76Y/MNFlN0d7HyS6tWGr6E
8FWJB7RYG5XHMEDIKSheq+Q5cORwz92JPFI+sovZukp+20G7f7/gwos441KamJPc
y1+M4uO21aKX2fA07bcgFtm25gNLoHyvjQLcmyDis6xogvciCV3iQ/mtunewgYp/
lUX1dv0R5o8TteaAIkbJicbdLtur/iuAWN404E/QShc=
=8P00
-----END PGP PUBLIC KEY BLOCK-----

View File

@ -0,0 +1 @@
{"keys":{"A98A04B882E19D85FD36EE069565D80B055C92FF":{},"E89D4971F0097B1E7A3EB57371B484802E78D7CD":{}}}

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1 @@
7f94d6038bb4e5e7fff817151da5b22d7dd6d1e2d9ad51bd55504676786c17bd

View File

@ -0,0 +1,124 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
{
"spec":{
"_meta":{
"version":4
},
"nodes":[
{
"name":"libdwarf",
"version":"20130729",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"n7axrpelzl5kjuctt4yoaaf33gvgnik6cx7fjudwhc6hvywdrr4q====",
"dependencies":[
{
"name":"libelf",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez",
"parameters":{
"deptypes":[
"build",
"link"
],
"virtuals":[]
}
}
],
"hash":"sk2gqqz4n5njmvktycnd25wq25jxiqkr"
},
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"811f500a89ae7d2f61e2c0ef6f56e352dfbac245ae88275809088a1481489d5b"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEE6J1JcfAJex56PrVzcbSEgC54180FAmfHlp8ACgkQcbSEgC54
180hlxAAisLofFhr/PQvLcQ79T3t3V0tqGgz9x6QnPKfbPCgvb66tTNlny+ML0fY
y1H9xXQO53QOxfN9cdXcf2EVbRQ2eT6ltmwekI3ZZuCaTguflNu/i11UV6UnDy3x
dXOYQhky5QjtPbhJ0NxG5XDKoRFoUPR/rgXsiNG5O0sk3M5H9ldpsj8af5W/6LCL
gCTNM8fF0TVbd4MF9TiIECFBng2CrxhHwpl2gPHHxab1zxLRCF6t1lZvL6To0hmC
e/Tqre+42PhRSCtXuwhK22r0rvreVUaiglYn8udjOJHwNVKdzLnTZ1OBAFeIq00U
9uuroyaF841pq9+8PitwUORurv0lsnHUbfbi/+ou0HzMiaXzz+MPdOXt8nUuyScs
oKOi8ExvpWJ7vn6klkvQtMK/Gakzd4YOxO/nk9K8BJgVN3qrODwHYSORk8RrdITS
tkjiEJiIoklddiwCf3NUzlxiIYWbiqKqNbY+Pxh4B+OpVDnvRmpkJHgoSuVoCS8b
coaOTIgqDpnIClHIj7ogxO+ureRjIIkGNNh6wVhlHDlgm1GzxNUOklMrzDkYMD01
eTYxrbicw7ZVwqhFtR8olODKT9QAqXUJOkGHS9IA6FJctatkUkIOG1DSI52AZV1r
PYzgdKtTxS60EkN8Igl6VMTkaC05anLygCTyOvGaV7sqVKmzHY8=
=8OR5
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,72 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
{
"spec":{
"_meta":{
"version":4
},
"nodes":[
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"48c8aa769a62535f9d9f613722e3d3f5a48b91fde3c99a644b22f277a4502d75"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCgAdFiEE6J1JcfAJex56PrVzcbSEgC54180FAmfHlp8ACgkQcbSEgC54
182ezg/7Bkil1mY6d4recJMkFhpBzzDs8aMD+WQOBPoy/bWHIGsPb1DyOOW7lTLa
QC9jh9Rq02oMeX0LWvNg7k6iMTayWcrPzJwk1rgh3pg/ySgCTZ576/aP/UOZwA8h
HT/3RzsDFlq7Wkh4yYaDgSEDVc5PgUevb1p2f126Z9HMFjG8siEWmuZQOcy4I9JG
osQFtwWTLmx96sBMzweZTu2i3iGTPNz4Ae1hu+v5clmSFg43eW7EWChEVoob+3hb
hLRxajZEPsIho4yR5yynoxduXeXrLLP7GH6XGnYt7Z2GJR0UamIrPfxYuWBK76V1
03Ie2rRXwOKfsjDWw9Z8ziTVu25G0aZ274DX6eQyaWKfvzz69cBXO0fgw1lU8B9S
K0j9k/xtnDCrIkPSh4QGQpFRlbzxkj20E+EnwgDCGIlK1rBzo2V5na4YNj+SbC91
0BmWrj6dRkQZUMJHeb95kBMfFpKG5B6u7HQxZtIwHFAfF0nypbiB7xmdy/gAmUao
ej3Cu34DvWtLVeSh7lRimeEc44WyBDk2YSPqYleAwYMZBn4WSozUS/KVLU2T/AhZ
VlLaEBaFrVngmsw5PCdck0XRSNSAN9HUgPItpOzYig20NeT1/69wIlUZVNpLEYGT
yvZsmqHFnkunAs6av3XmGl0i8rSA6DujunpNXML6hUciFEK5wg4=
=Aq8h
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,429 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"libdwarf",
"version":"20130729",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"n7axrpelzl5kjuctt4yoaaf33gvgnik6cx7fjudwhc6hvywdrr4q====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c",
"cxx"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
},
{
"name":"libelf",
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa",
"parameters":{
"deptypes":[
"build",
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"u5uz3dcch5if4eve4sef67o2rf2lbfgh"
},
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
},
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
},
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"0898457b4cc4b18d71059ea254667fb6690f5933c82e1627f9fed3606488dbca"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEqYoEuILhnYX9Nu4GlWXYCwVckv8FAmf23+QACgkQlWXYCwVc
kv9Xlg//d7uWhVbHjujSXRpoN3hzH5sUvvTSZ9xzvXGAXCoAu2oEGg4hxZPIFQJ3
pZzKysZMfeFg+UKwDzex5TlKZ3JtKgCTKYl64zZfUl2EQgo/d/Fjz5mSFHW/6sa1
1uTe3+sVt+HlijN72t2412Qbp+/uGvU+KBvXPA7kgkp88Kd/PL9xe3jlT9ytH5Nw
3LIghe++JiepjFAKXTfIA04EjLb8c50AAxsK5Xx37HOOVHHQ8L9anFnOVYM+DxAz
gn4dBYUQ9Uu5k5uEu5CwtxsED2/Yar7YWIepEnyp6z4zQVbwjO4/w0vZ3wSJ9c4P
UhZs8V2akuqIWyzlQuBOjywnEQc/nw9v0py+Dr/Qr3U4XWh/LARWABMxa4IqXMOK
aVmd6weVjV4U929gaOT/FCtZPfaFNRbk97YP8yAxuLhSdiGS0Mp16Ygz21fVWB7C
UjkGGsKK1cdiJQ0m1CffmydU/nbDjSuw4WZIoIgDzvN7SFm7YBtE+xY+RUPsHU22
QMAXojF5abwn48HJeP47MYdfR7+nUJq6XJiJ7/80a7Ciy8SAVxinQWqvigf/hmTf
kAiQaqOVSlRBJ2yry5fYBKHSIRvghCqS4t4es8o13R7n2wz68VqKu0JkNlT3Ijjc
QjJYtI+844PCDNetPVV8iNWF6upnTJnPHcFmKAEO1663hOc3Dh8=
=3fA5
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,317 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa"
},
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
},
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"c068bcd1a27a3081c07ba775d83e90228e340bb6a7f0d55deb18a462760c4bcf"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEqYoEuILhnYX9Nu4GlWXYCwVckv8FAmf23+QACgkQlWXYCwVc
kv/zSg/+NrS4JjT9TFSFR/q2vaN9aL7fSTunxp+M8eAzTmg0sgHc/D6ov2PMpUF7
1E2mnZ2gL5a5dHtsSCf30ILFzQoD+m+I9yOwcJopcbEjr8pcnXBFe6TT8lkxlXtI
EHNsYGMUHFbFvc+hFdWatQJicdDaIbdyEMGAC7Kobs/4KpdBF5VWV+sIrzD5+XzO
ACiKRjBmcaJpa950nuEaFzBITgq1aDtZ0EEZdXYvjRnzj9Bm6gbqmWzlllW1wf4r
5hSMTpAsRED4TxL433nuf0nKIvTD5Mywzs88kiLCtEABfDy1qccyBAnjyNypFF6B
fPqSDnr33s+JQ35t7RcHKfrgowk69UablE25YOUrQP6LtH4QzLBLj4/Z0zuz33hO
v+YYe51DgixsMQ2WCKWEO6sNcrcrLBJMFVwUP2FyTTdW3jCYRlFiTYLSfoDhTRJ/
4o7f2eEp3sVoOe12jKI6dw/P+c70dl8K4+1ICcnZkwsb0pd0vt2z4J2kPs2+1/0g
vpywJO1HL5Zy7/ZRlmeeSMHYEDX2eKhm7QRFbxw1IEbg3stQCA7a425JWztyJ05K
sfhFQgPt7F/xanJVFYk/hdza+3+5pFr1K/ARcLFBdLBKGxAXTMMR+NkMp3J5NiOo
SMZJ3jG6xA2ntvSkyx/GFawD0FpnlgEByU3E+R/WiQA4VojLpvo=
=kfWI
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,99 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"2c1c5576e30b7063aa02a22111eb24b3f2a93c35ac0f64b4e491c7078706c0ea"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEqYoEuILhnYX9Nu4GlWXYCwVckv8FAmf23+QACgkQlWXYCwVc
kv/T8BAAhK/v7CP6lMIKILj35nEi+Gftjs7B7f6qvb4QNtqcGHum6z9t3JxkOOrd
+q+Wd329kLYAFs/y9eaGe5X7wY1U7/f863i3XrxHbtmrnMci61D8qMjA1xnBGC+5
yd746aVeV/VRbJxTeB9kGcKPMcIQYcearlDMgj5fKfpCKM8a+VyJfw7qHNUyrTnu
d6LSGsEey6tGkJecgnJZTNSwryO3BZbg/4EviivMXm38AKGZrSib06qjkoHrPRvB
8ftGSGlK4YmFs5/YjKFL7QzuNJeqPNJt4mD64tsk21urOfbQJe5AmdMLPGY0PbW/
w++06c8lsd/6FmzUwlnTBUa39lKJjhkhoK7KFGVqZROcXZfhwAyqPZt7ReA5FDMV
l5X7sytjQuSFaQPGi5g1xXQGEI394T2I55p5T5/RuQ2PXcFxxSOmIcEcD8o6Z7+x
XWLq44KUWQyQP/StjaVhIz9YPogeBBJllA9hN+GzVrr2i+Esu1QO5uDgVuJP7pTA
9wwCLV/t0hf2TZcpU2fwEu+DMniaHm6haVwqiu6QGkbkMBx49zkV9b5i9L441GoC
Q86R2Gs9O0+QzHuN6egbQ0xKm/lfU8dmJSzV0snXawAeQ/vgCpdinx40EMc7Nz03
rgZ3j88c/ADvCb1DVKmu1Phf6U7WqG6/AvB9tYl4Zl30VX7ETaw=
=ifvQ
-----END PGP SIGNATURE-----

View File

@ -0,0 +1,151 @@
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"f33e7a6798a5fb2db6e538d3a530cc79b298e36d56a1df385d93889a9ba431d0"
}
}
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEqYoEuILhnYX9Nu4GlWXYCwVckv8FAmf23+QACgkQlWXYCwVc
kv+MsRAAsaQjZbB9iW/Lq9b87H/E5Zmv6RrClvpjSnwvhLR4nhPL3p0G70k6tI/b
NEdXctDyvBOJOEoLaEBrCODl/3GjV8B9Gj7OhT/BIKQjlOfJqVdwIrnHgav5ri+Q
UUXLtejhJiUNoxeILI/xZx2CoKT9q/3EpQ5ysqdybJmYJCf/hv+lXEhnwUIv8vV/
xdRYY//rfeMowCNIZtFPjSejMywXJfFKjl7h5dN5kwM63D6z/sh4zW7tqHq4kk+A
2m0WcorVg93wAm+YoJaQJVx8bYeMGfV/TjmY/cSouCt8PM4Vi93vwieZCkzEpXbM
BkVN4X3PTMZSOf0WTkEbnQD5v090/DoQPZyBrcDoJ/HmWDiz5Is2wUI0mLVkbg2L
+rKNC3ZajJhsWElMGNNtZRLmGeTIe8hT+LNAejo221vrOJbnUmpIjKxVjStDbXmW
nulgyEPSTfsJaXgbXmeJ8LOk0tWpBAGC16VzgXrPxoGD2XKxoiPCGLNrF/l1wyl+
n+nw3TchNFrofpPrqJzT/vS71B6KDb0PVSTQZfM9+FahrQ+YbsIkzDAuxVZb5t3q
HUME95RgoIBbccUGxAPwkaNme2OLaLzsJZ/Xhl5I8T1fraLYapsKNjQ5+CSKO8+t
MlJYgSHuazWSetRbZ2H7g7QJWqeHUAWi9i1szpNDYxTFSs8wgDY=
=edPy
-----END PGP SIGNATURE-----

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1 @@
57cad2589fae55cda3c35cadf4286d2e7702f90a708da80d70a76213fc45a688

View File

@ -0,0 +1,105 @@
{
"spec":{
"_meta":{
"version":4
},
"nodes":[
{
"name":"libdwarf",
"version":"20130729",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"n7axrpelzl5kjuctt4yoaaf33gvgnik6cx7fjudwhc6hvywdrr4q====",
"dependencies":[
{
"name":"libelf",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez",
"parameters":{
"deptypes":[
"build",
"link"
],
"virtuals":[]
}
}
],
"hash":"sk2gqqz4n5njmvktycnd25wq25jxiqkr"
},
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"f31bce1bfdcaa9b11cd02b869dd07a843db9819737399b99ac614ab3552bd4b6"
}
}

View File

@ -0,0 +1,53 @@
{
"spec":{
"_meta":{
"version":4
},
"nodes":[
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"core2",
"vendor":"GenuineIntel",
"features":[
"mmx",
"sse",
"sse2",
"ssse3"
],
"generation":0,
"parents":[
"nocona"
],
"cpupart":""
}
},
"compiler":{
"name":"gcc",
"version":"10.2.1"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"hash":"rqh2vuf6fqwkmipzgi2wjx352mq7y7ez"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"523ebc3691b0687cf93fcd477002972ea7d6da5d3e8d46636b30d4f1052fcbf2"
}
}

View File

@ -0,0 +1,410 @@
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"libdwarf",
"version":"20130729",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"n7axrpelzl5kjuctt4yoaaf33gvgnik6cx7fjudwhc6hvywdrr4q====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c",
"cxx"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
},
{
"name":"libelf",
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa",
"parameters":{
"deptypes":[
"build",
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"u5uz3dcch5if4eve4sef67o2rf2lbfgh"
},
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
},
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
},
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"2a6f045996e4998ec37679e1aa8a245795fbbffaf9844692ba2de6eeffcbc722"
}
}

View File

@ -0,0 +1,298 @@
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"libelf",
"version":"0.8.13",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ejr32l7tkp6uhdrlunqv4adkuxqwyac7vbqcjvg6dh72mll4cpiq====",
"dependencies":[
{
"name":"compiler-wrapper",
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
},
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[
"c"
]
}
},
{
"name":"gcc-runtime",
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n",
"parameters":{
"deptypes":[
"link"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"jr3yipyxyjulcdvckwwwjrrumis7glpa"
},
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
},
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"59141e49bd05abe40639360cd9422020513781270a3461083fee0eba2af62ca0"
}
}

View File

@ -0,0 +1,80 @@
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"compiler-wrapper",
"version":"1.0",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"ss7ybgvqf2fa2lvkf67eavllfxpxthiml2dobtkdq6wn7zkczteq====",
"annotations":{
"original_specfile_version":5
},
"hash":"qeehcxyvluwnihsc2qxstmpomtxo3lrc"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"f27ddff0ef4268acbe816c51f6f1fc907dc1010d31f2d6556b699c80f026c47d"
}
}

View File

@ -0,0 +1,132 @@
{
"spec":{
"_meta":{
"version":5
},
"nodes":[
{
"name":"gcc-runtime",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":{
"name":"m1",
"vendor":"Apple",
"features":[
"aes",
"asimd",
"asimddp",
"asimdfhm",
"asimdhp",
"asimdrdm",
"atomics",
"cpuid",
"crc32",
"dcpodp",
"dcpop",
"dit",
"evtstrm",
"fcma",
"flagm",
"flagm2",
"fp",
"fphp",
"frint",
"ilrcpc",
"jscvt",
"lrcpc",
"paca",
"pacg",
"pmull",
"sb",
"sha1",
"sha2",
"sha3",
"sha512",
"ssbs",
"uscat"
],
"generation":0,
"parents":[
"armv8.4a"
],
"cpupart":"0x022"
}
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"package_hash":"up2pdsw5tfvmn5gwgb3opl46la3uxoptkr3udmradd54s7qo72ha====",
"dependencies":[
{
"name":"gcc",
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq",
"parameters":{
"deptypes":[
"build"
],
"virtuals":[]
}
}
],
"annotations":{
"original_specfile_version":5
},
"hash":"izgzpzeljwairalfjm3k6fntbb64nt6n"
},
{
"name":"gcc",
"version":"10.2.1",
"arch":{
"platform":"test",
"platform_os":"debian6",
"target":"aarch64"
},
"namespace":"builtin.mock",
"parameters":{
"build_system":"generic",
"languages":[
"c",
"c++",
"fortran"
],
"cflags":[],
"cppflags":[],
"cxxflags":[],
"fflags":[],
"ldflags":[],
"ldlibs":[]
},
"external":{
"path":"/path",
"module":null,
"extra_attributes":{
"compilers":{
"c":"/path/bin/gcc-10",
"cxx":"/path/bin/g++-10",
"fortran":"/path/bin/gfortran-10"
}
}
},
"package_hash":"a7d6wvl2mh4od3uue3yxqonc7r7ihw3n3ldedu4kevqa32oy2ysa====",
"annotations":{
"original_specfile_version":5
},
"hash":"vd7v4ssgnoqdplgxyig3orum67n4vmhq"
}
]
},
"buildcache_layout_version":2,
"binary_cache_checksum":{
"hash_algorithm":"sha256",
"hash":"348f23717c5641fa6bbb90862e62bf632367511c53e9c6450584f0d000841320"
}
}

File diff suppressed because it is too large Load Diff

View File

@ -42,13 +42,22 @@ ci:
aud: "${OIDC_TOKEN_AUDIENCE}" aud: "${OIDC_TOKEN_AUDIENCE}"
- signing-job: - signing-job:
image: { "name": "ghcr.io/spack/notary:0.0.1", "entrypoint": [""] } image:
name: ghcr.io/spack/notary@sha256:d5a183b090602dea5dc89d5023fe777d1e64d9a7ddcb6cc9ec58a79bb410c168
entrypoint: [""]
tags: ["aws"] tags: ["aws"]
script: script:
- - aws s3 sync --exclude "*" --include "*spec.json*" ${SPACK_BUILDCACHE_DESTINATION}/build_cache /tmp - - export BUILD_CACHE="${SPACK_BUILDCACHE_DESTINATION}/${SPACK_BUILDCACHE_RELATIVE_SPECS_URL}"
- mkdir -p /tmp/input /tmp/output
- aws s3 sync --exclude "*" --include "*spec.manifest.json" ${BUILD_CACHE} /tmp/input
- /sign.sh - /sign.sh
- aws s3 sync --exclude "*" --include "*spec.json.sig*" /tmp ${SPACK_BUILDCACHE_DESTINATION}/build_cache - aws s3 sync --exclude "*" --include "*spec.manifest.json" /tmp/output ${BUILD_CACHE}
- aws s3 cp /tmp/public_keys ${SPACK_BUILDCACHE_DESTINATION}/build_cache/_pgp --recursive --exclude "*" --include "*.pub" - |+
for keyfile in $( find /tmp/public_keys -type f );
do
spack gpg trust $keyfile
done
- spack gpg publish --update-index --mirror-url ${SPACK_BUILDCACHE_DESTINATION}
id_tokens: id_tokens:
GITLAB_OIDC_TOKEN: GITLAB_OIDC_TOKEN:
aud: "${OIDC_TOKEN_AUDIENCE}" aud: "${OIDC_TOKEN_AUDIENCE}"
@ -62,10 +71,14 @@ ci:
- export SPACK_COPY_ONLY_SOURCE=${SPACK_BUILDCACHE_SOURCE//SPACK_REPLACE_VERSION/${SPACK_REPLACE_VERSION}} - export SPACK_COPY_ONLY_SOURCE=${SPACK_BUILDCACHE_SOURCE//SPACK_REPLACE_VERSION/${SPACK_REPLACE_VERSION}}
script: script:
- - spack env activate --without-view ${SPACK_CONCRETE_ENV_DIR} - - spack env activate --without-view ${SPACK_CONCRETE_ENV_DIR}
# TODO: remove this when we stop getting windows config includes added to the environment
- spack config add 'config:build_stage:["$tempdir/$user/spack-stage", "$user_cache_path/stage"]'
- spack config blame config
- echo Copying environment specs from ${SPACK_COPY_ONLY_SOURCE} to ${SPACK_COPY_ONLY_DESTINATION} - echo Copying environment specs from ${SPACK_COPY_ONLY_SOURCE} to ${SPACK_COPY_ONLY_DESTINATION}
- spack buildcache sync "${SPACK_COPY_ONLY_SOURCE}" "${SPACK_COPY_ONLY_DESTINATION}" - spack buildcache sync "${SPACK_COPY_ONLY_SOURCE}" "${SPACK_COPY_ONLY_DESTINATION}"
- curl -fLsS https://spack.github.io/keys/spack-public-binary-key.pub -o /tmp/spack-public-binary-key.pub - curl -fLsS https://spack.github.io/keys/spack-public-binary-key.pub -o /tmp/spack-public-binary-key.pub
- aws s3 cp /tmp/spack-public-binary-key.pub "${SPACK_COPY_ONLY_DESTINATION}/build_cache/_pgp/spack-public-binary-key.pub" - spack gpg trust /tmp/spack-public-binary-key.pub
- spack gpg publish --mirror-url "${SPACK_COPY_ONLY_DESTINATION}"
- spack buildcache update-index --keys "${SPACK_COPY_ONLY_DESTINATION}" - spack buildcache update-index --keys "${SPACK_COPY_ONLY_DESTINATION}"
when: "always" when: "always"
retry: retry:

View File

@ -9,8 +9,3 @@ mirrors:
push: ${PR_MIRROR_PUSH_DOMAIN}/${CI_COMMIT_REF_NAME}/${SPACK_CI_STACK_NAME} push: ${PR_MIRROR_PUSH_DOMAIN}/${CI_COMMIT_REF_NAME}/${SPACK_CI_STACK_NAME}
source: False source: False
binary: True binary: True
buildcache-shared:
fetch: ${PR_MIRROR_FETCH_DOMAIN}/shared_pr_mirror/${SPACK_CI_STACK_NAME}
push: ${PR_MIRROR_PUSH_DOMAIN}/shared_pr_mirror/${SPACK_CI_STACK_NAME}
source: False
binary: True

View File

@ -29,8 +29,9 @@ spack:
- signing-job: - signing-job:
before_script: before_script:
# Do not distribute Intel & ARM binaries # Do not distribute Intel & ARM binaries
- - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep intel-oneapi | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done - - export SPECS_PATH=${SPACK_BUILDCACHE_RELATIVE_SPECS_PATH}
- for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep armpl | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/ | grep intel-oneapi | awk '{print $4}' | sed -e "s?^.*${SPECS_PATH}/??g"); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/$i; done
- for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/ | grep armpl | awk '{print $4}' | sed -e "s?^.*${SPECS_PATH}/??g"); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/$i; done
cdash: cdash:
build-group: AWS Packages build-group: AWS Packages

View File

@ -33,8 +33,9 @@ spack:
- signing-job: - signing-job:
before_script: before_script:
# Do not distribute Intel & ARM binaries # Do not distribute Intel & ARM binaries
- - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep intel-oneapi | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done - - export SPECS_PATH=${SPACK_BUILDCACHE_RELATIVE_SPECS_PATH}
- for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/ | grep armpl | awk '{print $4}' | sed -e 's?^.*build_cache/??g'); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/$i; done - for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/ | grep intel-oneapi | awk '{print $4}' | sed -e "s?^.*${SPECS_PATH}/??g"); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/$i; done
- for i in $(aws s3 ls --recursive ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/ | grep armpl | awk '{print $4}' | sed -e "s?^.*${SPECS_PATH}/??g"); do aws s3 rm ${SPACK_REMOTE_MIRROR_OVERRIDE}/${SPECS_PATH}/$i; done
cdash: cdash:
build-group: AWS Packages build-group: AWS Packages

View File

@ -563,7 +563,7 @@ _spack_buildcache() {
then then
SPACK_COMPREPLY="-h --help" SPACK_COMPREPLY="-h --help"
else else
SPACK_COMPREPLY="push create install list keys check download save-specfile sync update-index rebuild-index" SPACK_COMPREPLY="push create install list keys check download save-specfile sync update-index rebuild-index migrate"
fi fi
} }
@ -651,6 +651,15 @@ _spack_buildcache_rebuild_index() {
fi fi
} }
_spack_buildcache_migrate() {
if $list_options
then
SPACK_COMPREPLY="-h --help -u --unsigned -d --delete-existing -y --yes-to-all"
else
_mirrors
fi
}
_spack_cd() { _spack_cd() {
if $list_options if $list_options
then then

View File

@ -697,6 +697,7 @@ complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a save-sp
complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a sync -d 'sync binaries (and associated metadata) from one mirror to another' complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a sync -d 'sync binaries (and associated metadata) from one mirror to another'
complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a update-index -d 'update a buildcache index' complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a update-index -d 'update a buildcache index'
complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a rebuild-index -d 'update a buildcache index' complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a rebuild-index -d 'update a buildcache index'
complete -c spack -n '__fish_spack_using_command_pos 0 buildcache' -f -a migrate -d 'perform in-place binary mirror migration (2 to 3)'
complete -c spack -n '__fish_spack_using_command buildcache' -s h -l help -f -a help complete -c spack -n '__fish_spack_using_command buildcache' -s h -l help -f -a help
complete -c spack -n '__fish_spack_using_command buildcache' -s h -l help -d 'show this help message and exit' complete -c spack -n '__fish_spack_using_command buildcache' -s h -l help -d 'show this help message and exit'
@ -861,6 +862,18 @@ complete -c spack -n '__fish_spack_using_command buildcache rebuild-index' -s h
complete -c spack -n '__fish_spack_using_command buildcache rebuild-index' -s k -l keys -f -a keys complete -c spack -n '__fish_spack_using_command buildcache rebuild-index' -s k -l keys -f -a keys
complete -c spack -n '__fish_spack_using_command buildcache rebuild-index' -s k -l keys -d 'if provided, key index will be updated as well as package index' complete -c spack -n '__fish_spack_using_command buildcache rebuild-index' -s k -l keys -d 'if provided, key index will be updated as well as package index'
# spack buildcache migrate
set -g __fish_spack_optspecs_spack_buildcache_migrate h/help u/unsigned d/delete-existing y/yes-to-all
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s h -l help -f -a help
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s h -l help -d 'show this help message and exit'
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s u -l unsigned -f -a unsigned
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s u -l unsigned -d 'Ignore signatures and do not resign, default is False'
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s d -l delete-existing -f -a delete_existing
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s d -l delete-existing -d 'Delete the previous layout, the default is to keep it.'
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s y -l yes-to-all -f -a yes_to_all
complete -c spack -n '__fish_spack_using_command buildcache migrate' -s y -l yes-to-all -d 'assume "yes" is the answer to every confirmation request'
# spack cd # spack cd
set -g __fish_spack_optspecs_spack_cd h/help m/module-dir r/spack-root i/install-dir p/package-dir P/packages s/stage-dir S/stages c/source-dir b/build-dir e/env= first set -g __fish_spack_optspecs_spack_cd h/help m/module-dir r/spack-root i/install-dir p/package-dir P/packages s/stage-dir S/stages c/source-dir b/build-dir e/env= first
complete -c spack -n '__fish_spack_using_command_pos_remainder 0 cd' -f -k -a '(__fish_spack_specs)' complete -c spack -n '__fish_spack_using_command_pos_remainder 0 cd' -f -k -a '(__fish_spack_specs)'