eef041abee
We've included a package hash in Spack since #7193 for CI, and we started using it on the spec in #28504. However, what goes into the package hash is a bit opaque. Here's what `spec.json` looks like now: ```json { "spec": { "_meta": { "version": 3 }, "nodes": [ { "name": "zlib", "version": "1.2.12", ... "patches": [ "0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76" ], "package_hash": "pthf7iophdyonixxeed7gyqiksopxeklzzjbxtjrw7nzlkcqleba====", "hash": "ke4alug7ypoxp37jb6namwlxssmws4kp" } ] } } ``` The `package_hash` there is a hash of the concatenation of: * A canonical hash of the `package.py` recipe, as implemented in #28156; * `sha256`'s of patches applied to the spec; and * Archive `sha256` sums of archives or commits/revisions of repos used to build the spec. There are some issues with this: patches are counted twice in this spec (in `patches` and in the `package_hash`), the hashes of sources used to build are conflated with the `package.py` hash, and we don't actually include resources anywhere. With this PR, I've expanded the package hash out in the `spec.json` body. Here is the "same" spec with the new fields: ```json { "spec": { "_meta": { "version": 3 }, "nodes": [ { "name": "zlib", "version": "1.2.12", ... "package_hash": "6kkliqdv67ucuvfpfdwaacy5bz6s6en4", "sources": [ { "type": "archive", "sha256": "91844808532e5ce316b3c010929493c0244f3d37593afd6de04f71821d5136d9" } ], "patches": [ "0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76" ], "hash": "ts3gkpltbgzr5y6nrfy6rzwbjmkscein" } ] } } ``` Now: * Patches and archive hashes are no longer included in the `package_hash`; * Artifacts used in the build go in `sources`, and we tell you their checksum in the `spec.json`; * `sources` will include resources for packages that have it; * Patches are the same as before -- but only represented once; and * The `package_hash` is a base32-encoded `sha1`, like other hashes in Spack, and it only tells you that the `package.py` changed. The behavior of the DAG hash (which includes the `package_hash`) is basically the same as before, except now resources are included, and we can see differences in archives and resources directly in the `spec.json` Note that we do not need to bump the spec meta version on this, as past versions of Spack can still read the new specs; they just will not notice the new fields (which is fine, since we currently do not do anything with them). Among other things, this will more easily allow us to convert Spack specs to SBOM and track relevant security information (like `sha256`'s of archives). For example, we could do continuous scanning of a Spack installation based on these hashes, and if the `sha256`'s become associated with CVE's, we'll know we're affected. - [x] Add a method, `spec_attrs()` to `FetchStrategy` that can be used to describe a fetcher for a `spec.json`. - [x] Simplify the way package_hash() is handled in Spack. Previously, it was handled as a special-case spec hash in `hash_types.py`, but it really doesn't belong there. Now, it's handled as part of `Spec._finalize_concretization()` and `hash_types.py` is much simpler. - [x] Change `PackageBase.content_hash()` to `PackageBase.artifact_hashes()`, and include more information about artifacts in it. - [x] Update package hash tests and make them check for artifact and resource hashes. Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
59 lines
2.1 KiB
Python
59 lines
2.1 KiB
Python
# Copyright 2013-2024 Lawrence Livermore National Security, LLC and other
|
|
# Spack Project Developers. See the top-level COPYRIGHT file for details.
|
|
#
|
|
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
|
|
|
|
import os
|
|
|
|
from spack.package import *
|
|
|
|
|
|
class HashTest1(Package):
|
|
"""Used to test package hashing"""
|
|
|
|
homepage = "http://www.hashtest1.org"
|
|
url = "http://www.hashtest1.org/downloads/hashtest1-1.1.tar.bz2"
|
|
|
|
version("1.1", sha256="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
|
|
version("1.2", sha256="bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
|
|
version("1.3", sha256="cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc")
|
|
version("1.4", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
|
|
version("1.5", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
|
|
version("1.6", sha256="eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee")
|
|
version("1.7", sha256="ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff")
|
|
version("1.8", sha256="1111111111111111111111111111111111111111111111111111111111111111")
|
|
|
|
patch("patch1.patch", when="@1.1")
|
|
patch("patch2.patch", when="@1.4")
|
|
|
|
variant("variantx", default=False, description="Test variant X")
|
|
variant("varianty", default=False, description="Test variant Y")
|
|
|
|
resource(
|
|
url="http://www.example.com/example-1.0-resource.tar.gz",
|
|
sha256="abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234",
|
|
when="@1.8",
|
|
)
|
|
|
|
def setup_dependent_build_environment(self, env, dependent_spec):
|
|
pass
|
|
|
|
@when("@:1.4")
|
|
def install(self, spec, prefix):
|
|
print("install 1")
|
|
os.listdir(os.getcwd())
|
|
|
|
# sanity_check_prefix requires something in the install directory
|
|
mkdirp(prefix.bin)
|
|
|
|
@when("@1.5:")
|
|
def install(self, spec, prefix):
|
|
os.listdir(os.getcwd())
|
|
|
|
# sanity_check_prefix requires something in the install directory
|
|
mkdirp(prefix.bin)
|
|
|
|
@when("@1.5,1.6")
|
|
def extra_phase(self, spec, prefix):
|
|
pass
|