specs: include source provenance in spec.json and package hash

We've included a package hash in Spack since #7193 for CI, and we started using it on
the spec in #28504. However, what goes into the package hash is a bit opaque. Here's
what `spec.json` looks like now:

```json
{
  "spec": {
    "_meta": {
      "version": 3
    },
    "nodes": [
      {
        "name": "zlib",
        "version": "1.2.12",
        ...
        "patches": [
          "0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
        ],
        "package_hash": "pthf7iophdyonixxeed7gyqiksopxeklzzjbxtjrw7nzlkcqleba====",
        "hash": "ke4alug7ypoxp37jb6namwlxssmws4kp"
      }
    ]
  }
}
```

The `package_hash` there is a hash of the concatenation of:

* A canonical hash of the `package.py` recipe, as implemented in #28156;
* `sha256`'s of patches applied to the spec; and
* Archive `sha256` sums of archives or commits/revisions of repos used to build the spec.

There are some issues with this: patches are counted twice in this spec (in `patches`
and in the `package_hash`), the hashes of sources used to build are conflated with the
`package.py` hash, and we don't actually include resources anywhere.

With this PR, I've expanded the package hash out in the `spec.json` body. Here is the
"same" spec with the new fields:

```json
{
  "spec": {
    "_meta": {
      "version": 3
    },
    "nodes": [
      {
        "name": "zlib",
        "version": "1.2.12",
        ...
        "package_hash": "6kkliqdv67ucuvfpfdwaacy5bz6s6en4",
        "sources": [
          {
            "type": "archive",
            "sha256": "91844808532e5ce316b3c010929493c0244f3d37593afd6de04f71821d5136d9"
          }
        ],
        "patches": [
          "0d38234384870bfd34dfcb738a9083952656f0c766a0f5990b1893076b084b76"
        ],
        "hash": "ts3gkpltbgzr5y6nrfy6rzwbjmkscein"
      }
    ]
  }
}
```

Now:

* Patches and archive hashes are no longer included in the `package_hash`;
* Artifacts used in the build go in `sources`, and we tell you their checksum in the `spec.json`;
* `sources` will include resources for packages that have it;
* Patches are the same as before -- but only represented once; and
* The `package_hash` is a base32-encoded `sha1`, like other hashes in Spack, and it only
  tells you that the `package.py` changed.

The behavior of the DAG hash (which includes the `package_hash`) is basically the same
as before, except now resources are included, and we can see differences in archives and
resources directly in the `spec.json`

Note that we do not need to bump the spec meta version on this, as past versions of
Spack can still read the new specs; they just will not notice the new fields (which is
fine, since we currently do not do anything with them).

Among other things, this will more easily allow us to convert Spack specs to SBOM and
track relevant security information (like `sha256`'s of archives). For example, we could
do continuous scanning of a Spack installation based on these hashes, and if the
`sha256`'s become associated with CVE's, we'll know we're affected.

- [x] Add a method, `spec_attrs()` to `FetchStrategy` that can be used to describe a
      fetcher for a `spec.json`.

- [x] Simplify the way package_hash() is handled in Spack. Previously, it was handled as
      a special-case spec hash in `hash_types.py`, but it really doesn't belong there.
      Now, it's handled as part of `Spec._finalize_concretization()` and `hash_types.py`
      is much simpler.

- [x] Change `PackageBase.content_hash()` to `PackageBase.artifact_hashes()`, and
      include more information about artifacts in it.

- [x] Update package hash tests and make them check for artifact and resource hashes.

Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>

lib/spack/spack/caches.py

@@ -74,6 +74,6 @@ def store(self, fetcher, relative_dest):
 
 
 #: Spack's local cache for downloaded source archives
-FETCH_CACHE: Union[spack.fetch_strategy.FsCache, llnl.util.lang.Singleton] = (
+FETCH_CACHE: Union["spack.fetch_strategy.FsCache", llnl.util.lang.Singleton] = (
     llnl.util.lang.Singleton(_fetch_cache)
 )

lib/spack/spack/fetch_strategy.py

@@ -49,6 +49,7 @@
 import spack.util.archive
 import spack.util.crypto as crypto
 import spack.util.git
+import spack.util.spack_yaml as syaml
 import spack.util.url as url_util
 import spack.util.web as web_util
 import spack.version
@@ -110,6 +111,20 @@ def __init__(self, **kwargs):
 
         self.package = None
 
+    def spec_attrs(self):
+        """Create a dictionary of attributes that describe this fetch strategy for a Spec.
+
+        This is included in the serialized Spec format to store provenance (like hashes).
+        """
+        attrs = syaml.syaml_dict()
+        if self.url_attr:
+            attrs["type"] = "archive" if self.url_attr == "url" else self.url_attr
+        for attr in self.optional_attrs:
+            value = getattr(self, attr, None)
+            if value:
+                attrs[attr] = value
+        return attrs
+
     def set_package(self, package):
         self.package = package
 
@@ -254,6 +269,16 @@ def __init__(self, *, url: str, checksum: Optional[str] = None, **kwargs) -> Non
         self.extension: Optional[str] = kwargs.get("extension", None)
         self._effective_url: Optional[str] = None
 
+    def spec_attrs(self):
+        attrs = super(URLFetchStrategy, self).spec_attrs()
+        if self.digest:
+            try:
+                hash_type = spack.util.crypto.hash_algo_for_digest(self.digest)
+            except ValueError:
+                hash_type = "digest"
+            attrs[hash_type] = self.digest
+        return attrs
+
     @property
     def curl(self) -> Executable:
         if not self._curl:

lib/spack/spack/hash_types.py

@@ -5,7 +5,6 @@
 """Definitions that control how Spack creates Spec hashes."""
 
 import spack.deptypes as dt
-import spack.repo
 
 hashes = []
 
@@ -13,20 +12,17 @@
 class SpecHashDescriptor:
     """This class defines how hashes are generated on Spec objects.
 
-    Spec hashes in Spack are generated from a serialized (e.g., with
+    Spec hashes in Spack are generated from a serialized JSON representation of the DAG.
-    YAML) representation of the Spec graph.  The representation may only
+    The representation may only include certain dependency types, and it may optionally
-    include certain dependency types, and it may optionally include a
+    include a canonicalized hash of the ``package.py`` for each node in the graph.
-    canonicalized hash of the package.py for each node in the graph.
 
-    We currently use different hashes for different use cases."""
+    """
 
-    def __init__(self, depflag: dt.DepFlag, package_hash, name, override=None):
+    def __init__(self, depflag: dt.DepFlag, package_hash, name):
         self.depflag = depflag
         self.package_hash = package_hash
         self.name = name
         hashes.append(self)
-        # Allow spec hashes to have an alternate computation method
-        self.override = override
 
     @property
     def attr(self):
@@ -54,18 +50,6 @@ def __repr__(self):
 )
 
 
-def _content_hash_override(spec):
-    pkg_cls = spack.repo.PATH.get_pkg_class(spec.name)
-    pkg = pkg_cls(spec)
-    return pkg.content_hash()
-
-
-#: Package hash used as part of dag hash
-package_hash = SpecHashDescriptor(
-    depflag=0, package_hash=True, name="package_hash", override=_content_hash_override
-)
-
-
 # Deprecated hash types, no longer used, but needed to understand old serialized
 # spec formats
 

lib/spack/spack/package_base.py

@@ -9,12 +9,10 @@
 packages.
 """
 
-import base64
 import collections
 import copy
 import functools
 import glob
-import hashlib
 import importlib
 import io
 import os
@@ -50,6 +48,7 @@
 import spack.url
 import spack.util.environment
 import spack.util.path
+import spack.util.spack_yaml as syaml
 import spack.util.web
 from spack.error import InstallError, NoURLError, PackageError
 from spack.filesystem_view import YamlFilesystemView
@@ -1754,7 +1753,7 @@ def all_patches(cls):
 
         return patches
 
-    def content_hash(self, content=None):
+    def artifact_hashes(self, content=None):
         """Create a hash based on the artifacts and patches used to build this package.
 
         This includes:
@@ -1767,52 +1766,47 @@ def content_hash(self, content=None):
         determinable) portions of the hash will be included.
 
         """
-        # list of components to make up the hash
+        hashes = syaml.syaml_dict()
-        hash_content = []
 
         # source artifacts/repositories
         # TODO: resources
         if self.spec.versions.concrete:
+            sources = []
             try:
-                source_id = fs.for_package_version(self).source_id()
+                fetcher = fs.for_package_version(self)
+                sources.append(fetcher.spec_attrs())
+
             except (fs.ExtrapolationError, fs.InvalidArgsError):
                 # ExtrapolationError happens if the package has no fetchers defined.
                 # InvalidArgsError happens when there are version directives with args,
                 #     but none of them identifies an actual fetcher.
-                source_id = None
 
-            if not source_id:
+                # if this is a develop spec, say so
-                # TODO? in cases where a digest or source_id isn't available,
-                # should this attempt to download the source and set one? This
-                # probably only happens for source repositories which are
-                # referenced by branch name rather than tag or commit ID.
                 from_local_sources = "dev_path" in self.spec.variants
+
+                # don't bother setting a source id if none is available, but warn if
+                # it seems like there should be one.
                 if self.has_code and not self.spec.external and not from_local_sources:
                     message = "Missing a source id for {s.name}@{s.version}"
                     tty.debug(message.format(s=self))
-                hash_content.append("".encode("utf-8"))
+
-            else:
+            for resource in self._get_needed_resources():
-                hash_content.append(source_id.encode("utf-8"))
+                sources.append(resource.fetcher.spec_attrs())
+
+            if sources:
+                hashes["sources"] = sources
 
         # patch sha256's
         # Only include these if they've been assigned by the concretizer.
         # We check spec._patches_assigned instead of spec.concrete because
         # we have to call package_hash *before* marking specs concrete
         if self.spec._patches_assigned():
-            hash_content.extend(
+            hashes["patches"] = [p.sha256 for p in self.spec.patches]
-                ":".join((p.sha256, str(p.level))).encode("utf-8") for p in self.spec.patches
-            )
 
         # package.py contents
-        hash_content.append(package_hash(self.spec, source=content).encode("utf-8"))
+        hashes["package_hash"] = package_hash(self.spec, source=content)
 
-        # put it all together and encode as base32
+        return hashes
-        b32_hash = base64.b32encode(
-            hashlib.sha256(bytes().join(sorted(hash_content))).digest()
-        ).lower()
-        b32_hash = b32_hash.decode("utf-8")
-
-        return b32_hash
 
     @property
     def cmake_prefix_paths(self):

lib/spack/spack/spec.py

@@ -1478,6 +1478,12 @@ def __init__(
         for h in ht.hashes:
             setattr(self, h.attr, None)
 
+        # dictionary of source artifact hashes, set at concretization time
+        self._package_hash = None
+
+        # dictionary of source artifact hashes, set at concretization time
+        self._artifact_hashes = None
+
         # Python __hash__ is handled separately from the cached spec hashes
         self._dunder_hash = None
 
@@ -1968,10 +1974,6 @@ def spec_hash(self, hash):
         Arguments:
             hash (spack.hash_types.SpecHashDescriptor): type of hash to generate.
         """
-        # TODO: currently we strip build dependencies by default.  Rethink
-        # this when we move to using package hashing on all specs.
-        if hash.override is not None:
-            return hash.override(self)
         node_dict = self.to_node_dict(hash=hash)
         json_text = sjson.dump(node_dict)
         # This implements "frankenhashes", preserving the last 7 characters of the
@@ -1981,7 +1983,7 @@ def spec_hash(self, hash):
             return out[:-7] + self.build_spec.spec_hash(hash)[-7:]
         return out
 
-    def _cached_hash(self, hash, length=None, force=False):
+    def _cached_hash(self, hash, length=None):
         """Helper function for storing a cached hash on the spec.
 
         This will run spec_hash() with the deptype and package_hash
@@ -1991,7 +1993,6 @@ def _cached_hash(self, hash, length=None, force=False):
         Arguments:
             hash (spack.hash_types.SpecHashDescriptor): type of hash to generate.
             length (int): length of hash prefix to return (default is full hash string)
-            force (bool): cache the hash even if spec is not concrete (default False)
         """
         if not hash.attr:
             return self.spec_hash(hash)[:length]
@@ -2001,7 +2002,7 @@ def _cached_hash(self, hash, length=None, force=False):
             return hash_string[:length]
         else:
             hash_string = self.spec_hash(hash)
-            if force or self.concrete:
+            if self.concrete:
                 setattr(self, hash.attr, hash_string)
 
             return hash_string[:length]
@@ -2172,7 +2173,10 @@ def to_node_dict(self, hash=ht.dag_hash):
         if self.namespace:
             d["namespace"] = self.namespace
 
-        params = syaml.syaml_dict(sorted(v.yaml_entry() for _, v in self.variants.items()))
+        # Get all variants *except* for patches.  Patches are included in "artifacts" below.
+        params = syaml.syaml_dict(
+            sorted(v.yaml_entry() for _, v in self.variants.items() if v.name != "patches")
+        )
 
         # Only need the string compiler flag for yaml file
         params.update(
@@ -2197,28 +2201,23 @@ def to_node_dict(self, hash=ht.dag_hash):
         if not self._concrete:
             d["concrete"] = False
 
-        if "patches" in self.variants:
+        if self._concrete and hash.package_hash:
-            variant = self.variants["patches"]
+            # We use the attribute here instead of `self.package_hash()` because this should
-            if hasattr(variant, "_patches_in_order_of_appearance"):
+            # *always* be assigned at concretization time. We don't want to try to compute a
-                d["patches"] = variant._patches_in_order_of_appearance
+            # package hash for concrete spec where a) the package might not exist, or b) the
+            # `dag_hash` didn't include the package hash when the spec was concretized.
+            if hasattr(self, "_package_hash") and self._package_hash:
+                d["package_hash"] = self._package_hash
 
-        if (
+            if self._artifact_hashes:
-            self._concrete
+                for key, source_list in sorted(self._artifact_hashes.items()):
-            and hash.package_hash
+                    # sources may be dictionaries (for archives/resources)
-            and hasattr(self, "_package_hash")
+                    def order(source):
-            and self._package_hash
+                        if isinstance(source, dict):
-        ):
+                            return syaml.syaml_dict(sorted(source.items()))
-            # We use the attribute here instead of `self.package_hash()` because this
+                        return source
-            # should *always* be assignhed at concretization time. We don't want to try
-            # to compute a package hash for concrete spec where a) the package might not
-            # exist, or b) the `dag_hash` didn't include the package hash when the spec
-            # was concretized.
-            package_hash = self._package_hash
 
-            # Full hashes are in bytes
+                    d[key] = [order(source) for source in source_list]
-            if not isinstance(package_hash, str) and isinstance(package_hash, bytes):
-                package_hash = package_hash.decode("utf-8")
-            d["package_hash"] = package_hash
 
         # Note: Relies on sorting dict by keys later in algorithm.
         deps = self._dependencies_dict(depflag=hash.depflag)
@@ -2917,12 +2916,16 @@ def _finalize_concretization(self):
             # We only assign package hash to not-yet-concrete specs, for which we know
             # we can compute the hash.
             if not spec.concrete:
-                # we need force=True here because package hash assignment has to happen
+                # package hash assignment has to happen before we mark concrete, so that
-                # before we mark concrete, so that we know what was *already* concrete.
+                # we know what was *already* concrete.
-                spec._cached_hash(ht.package_hash, force=True)
+                # can't use self.package here b/c not concrete yet
+                pkg_cls = spack.repo.PATH.get_pkg_class(spec.name)
+                pkg = pkg_cls(spec)
 
-                # keep this check here to ensure package hash is saved
+                # TODO: make artifact hashes a static method
-                assert getattr(spec, ht.package_hash.attr)
+                artifact_hashes = pkg.artifact_hashes()
+                spec._package_hash = artifact_hashes.pop("package_hash")
+                spec._artifact_hashes = artifact_hashes
 
         # Mark everything in the spec as concrete
         self._mark_concrete()
@@ -3558,6 +3561,8 @@ def _dup(self, other, deps: Union[bool, dt.DepTypes, dt.DepFlag] = True, clearde
             self._normal = other._normal
             for h in ht.hashes:
                 setattr(self, h.attr, getattr(other, h.attr, None))
+            self._package_hash = getattr(other, "_package_hash", None)
+            self._artifact_hashes = getattr(other, "_artifact_hashes", None)
         else:
             self._dunder_hash = None
             # Note, we could use other._normal if we are copying all deps, but
@@ -3565,6 +3570,8 @@ def _dup(self, other, deps: Union[bool, dt.DepTypes, dt.DepFlag] = True, clearde
             self._normal = False
             for h in ht.hashes:
                 setattr(self, h.attr, None)
+            self._package_hash = None
+            self._artifact_hashes = None
 
         return changed
 
@@ -4427,6 +4434,8 @@ def clear_cached_hashes(self, ignore=()):
             if h.attr not in ignore:
                 if hasattr(self, h.attr):
                     setattr(self, h.attr, None)
+        self._package_hash = None
+        self._artifact_hashes = None
         self._dunder_hash = None
 
     def __hash__(self):
@@ -4702,6 +4711,14 @@ def from_node_dict(cls, node):
         for h in ht.hashes:
             setattr(spec, h.attr, node.get(h.name, None))
 
+        # old and new-style package hash
+        if "package_hash" in node:
+            spec._package_hash = node["package_hash"]
+
+        # all source artifact hashes
+        if "sources" in node:
+            spec._artifact_hashes = syaml.syaml_dict([("sources", node["sources"])])
+
         spec.name = name
         spec.namespace = node.get("namespace", None)
 

lib/spack/spack/test/util/package_hash.py

@@ -19,29 +19,27 @@
 datadir = os.path.join(spack.paths.test_path, "data", "unparse")
 
 
-def compare_sans_name(eq, spec1, spec2):
+def canonical_source_equal_sans_name(spec1, spec2):
     content1 = ph.canonical_source(spec1)
     content1 = content1.replace(spack.repo.PATH.get_pkg_class(spec1.name).__name__, "TestPackage")
     content2 = ph.canonical_source(spec2)
     content2 = content2.replace(spack.repo.PATH.get_pkg_class(spec2.name).__name__, "TestPackage")
-    if eq:
+
-        assert content1 == content2
+    return content1 == content2
-    else:
-        assert content1 != content2
 
 
-def compare_hash_sans_name(eq, spec1, spec2):
+def package_hash_equal_sans_name(spec1, spec2):
     content1 = ph.canonical_source(spec1)
     pkg_cls1 = spack.repo.PATH.get_pkg_class(spec1.name)
     content1 = content1.replace(pkg_cls1.__name__, "TestPackage")
-    hash1 = pkg_cls1(spec1).content_hash(content=content1)
+    hash1 = ph.package_hash(spec1, source=content1)
 
     content2 = ph.canonical_source(spec2)
     pkg_cls2 = spack.repo.PATH.get_pkg_class(spec2.name)
     content2 = content2.replace(pkg_cls2.__name__, "TestPackage")
-    hash2 = pkg_cls2(spec2).content_hash(content=content2)
+    hash2 = ph.package_hash(spec2, source=content2)
 
-    assert (hash1 == hash2) == eq
+    return hash1 == hash2
 
 
 def test_hash(mock_packages, config):
@@ -57,11 +55,11 @@ def test_different_variants(mock_packages, config):
 def test_all_same_but_name(mock_packages, config):
     spec1 = Spec("hash-test1@=1.2")
     spec2 = Spec("hash-test2@=1.2")
-    compare_sans_name(True, spec1, spec2)
+    assert canonical_source_equal_sans_name(spec1, spec2)
 
     spec1 = Spec("hash-test1@=1.2 +varianty")
     spec2 = Spec("hash-test2@=1.2 +varianty")
-    compare_sans_name(True, spec1, spec2)
+    assert canonical_source_equal_sans_name(spec1, spec2)
 
 
 def test_all_same_but_archive_hash(mock_packages, config):
@@ -70,60 +68,63 @@ def test_all_same_but_archive_hash(mock_packages, config):
     """
     spec1 = Spec("hash-test1@=1.3")
     spec2 = Spec("hash-test2@=1.3")
-    compare_sans_name(True, spec1, spec2)
+    assert canonical_source_equal_sans_name(spec1, spec2)
 
 
 def test_all_same_but_patch_contents(mock_packages, config):
     spec1 = Spec("hash-test1@=1.1")
     spec2 = Spec("hash-test2@=1.1")
-    compare_sans_name(True, spec1, spec2)
+    assert canonical_source_equal_sans_name(spec1, spec2)
 
 
 def test_all_same_but_patches_to_apply(mock_packages, config):
     spec1 = Spec("hash-test1@=1.4")
     spec2 = Spec("hash-test2@=1.4")
-    compare_sans_name(True, spec1, spec2)
+    assert canonical_source_equal_sans_name(spec1, spec2)
 
 
 def test_all_same_but_install(mock_packages, config):
     spec1 = Spec("hash-test1@=1.5")
     spec2 = Spec("hash-test2@=1.5")
-    compare_sans_name(False, spec1, spec2)
+    assert not canonical_source_equal_sans_name(spec1, spec2)
 
 
-def test_content_hash_all_same_but_patch_contents(mock_packages, config):
+def test_package_hash_all_same_but_patch_contents_different(mock_packages, config):
     spec1 = Spec("hash-test1@1.1").concretized()
     spec2 = Spec("hash-test2@1.1").concretized()
-    compare_hash_sans_name(False, spec1, spec2)
+
+    assert package_hash_equal_sans_name(spec1, spec2)
+    assert spec1.dag_hash() != spec2.dag_hash()
+    assert spec1.to_node_dict()["patches"] != spec2.to_node_dict()["patches"]
 
 
-def test_content_hash_not_concretized(mock_packages, config):
+def test_package_hash_not_concretized(mock_packages, config):
-    """Check that Package.content_hash() works on abstract specs."""
+    """Check that ``package_hash()`` works on abstract specs."""
-    # these are different due to the package hash
+    # these are different due to patches but not package hash
     spec1 = Spec("hash-test1@=1.1")
     spec2 = Spec("hash-test2@=1.3")
-    compare_hash_sans_name(False, spec1, spec2)
+    assert package_hash_equal_sans_name(spec1, spec2)
 
     # at v1.1 these are actually the same package when @when's are removed
     # and the name isn't considered
     spec1 = Spec("hash-test1@=1.1")
     spec2 = Spec("hash-test2@=1.1")
-    compare_hash_sans_name(True, spec1, spec2)
+    assert package_hash_equal_sans_name(spec1, spec2)
 
-    # these end up being different b/c we can't eliminate much of the package.py
+    # these end up being different b/c without a version, we can't eliminate much of the
-    # without a version.
+    # package.py when canonicalizing source.
     spec1 = Spec("hash-test1")
     spec2 = Spec("hash-test2")
-    compare_hash_sans_name(False, spec1, spec2)
+    assert not package_hash_equal_sans_name(spec1, spec2)
 
 
-def test_content_hash_different_variants(mock_packages, config):
+def test_package_hash_different_variants(mock_packages, config):
     spec1 = Spec("hash-test1@1.2 +variantx").concretized()
     spec2 = Spec("hash-test2@1.2 ~variantx").concretized()
-    compare_hash_sans_name(True, spec1, spec2)
+    assert package_hash_equal_sans_name(spec1, spec2)
 
 
-def test_content_hash_cannot_get_details_from_ast(mock_packages, config):
+def test_package_hash_cannot_get_details_from_ast(mock_packages, config):
     """Packages hash-test1 and hash-test3 would be considered the same
     except that hash-test3 conditionally executes a phase based on
     a "when" directive that Spack cannot evaluate by examining the
@@ -135,18 +136,38 @@ def test_content_hash_cannot_get_details_from_ast(mock_packages, config):
     """
     spec3 = Spec("hash-test1@1.7").concretized()
     spec4 = Spec("hash-test3@1.7").concretized()
-    compare_hash_sans_name(False, spec3, spec4)
+    assert not package_hash_equal_sans_name(spec3, spec4)
 
 
-def test_content_hash_all_same_but_archive_hash(mock_packages, config):
+def test_package_hash_all_same_but_archive_hash(mock_packages, config):
     spec1 = Spec("hash-test1@1.3").concretized()
     spec2 = Spec("hash-test2@1.3").concretized()
-    compare_hash_sans_name(False, spec1, spec2)
+
+    assert package_hash_equal_sans_name(spec1, spec2)
+
+    # the sources for these two packages will not be the same b/c their archive hashes differ
+    assert spec1.to_node_dict()["sources"] != spec2.to_node_dict()["sources"]
+
+    assert spec1.dag_hash() != spec2.dag_hash()
 
 
-def test_content_hash_parse_dynamic_function_call(mock_packages, config):
+def test_package_hash_all_same_but_resources(mock_packages, config):
+    spec1 = Spec("hash-test1@1.7").concretized()
+    spec2 = Spec("hash-test1@1.8").concretized()
+
+    # these should be the same
+    assert canonical_source_equal_sans_name(spec1, spec2)
+    assert package_hash_equal_sans_name(spec1, spec2)
+
+    # but 1.7 has a resource that affects the hash
+    assert spec1.to_node_dict()["sources"] != spec2.to_node_dict()["sources"]
+
+    assert spec1.dag_hash() != spec2.dag_hash()
+
+
+def test_package_hash_parse_dynamic_function_call(mock_packages, config):
     spec = Spec("hash-test4").concretized()
-    spec.package.content_hash()
+    ph.package_hash(spec)
 
 
 many_strings = '''\

var/spack/repos/builtin.mock/packages/hash-test1/package.py

@@ -14,13 +14,14 @@ class HashTest1(Package):
     homepage = "http://www.hashtest1.org"
     url = "http://www.hashtest1.org/downloads/hashtest1-1.1.tar.bz2"
 
-    version("1.1", md5="a" * 32)
+    version("1.1", sha256="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
-    version("1.2", md5="b" * 32)
+    version("1.2", sha256="bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
-    version("1.3", md5="c" * 32)
+    version("1.3", sha256="cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc")
-    version("1.4", md5="d" * 32)
+    version("1.4", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
-    version("1.5", md5="d" * 32)
+    version("1.5", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
-    version("1.6", md5="e" * 32)
+    version("1.6", sha256="eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee")
-    version("1.7", md5="f" * 32)
+    version("1.7", sha256="ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff")
+    version("1.8", sha256="1111111111111111111111111111111111111111111111111111111111111111")
 
     patch("patch1.patch", when="@1.1")
     patch("patch2.patch", when="@1.4")
@@ -28,6 +29,12 @@ class HashTest1(Package):
     variant("variantx", default=False, description="Test variant X")
     variant("varianty", default=False, description="Test variant Y")
 
+    resource(
+        url="http://www.example.com/example-1.0-resource.tar.gz",
+        sha256="abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234",
+        when="@1.8",
+    )
+
     def setup_dependent_build_environment(self, env, dependent_spec):
         pass
 

var/spack/repos/builtin.mock/packages/hash-test2/package.py

@@ -14,10 +14,13 @@ class HashTest2(Package):
     homepage = "http://www.hashtest2.org"
     url = "http://www.hashtest1.org/downloads/hashtest2-1.1.tar.bz2"
 
-    version("1.1", md5="a" * 32)
+    version("1.1", sha256="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
-    version("1.2", md5="b" * 32)
+    version("1.2", sha256="bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
-    version("1.3", md5="c" * 31 + "x")  # Source hash differs from hash-test1@1.3
+
-    version("1.4", md5="d" * 32)
+    # Source hash differs from hash-test1@1.3
+    version("1.3", sha256="cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccf")
+
+    version("1.4", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
 
     patch("patch1.patch", when="@1.1")
 

var/spack/repos/builtin.mock/packages/hash-test3/package.py

@@ -14,11 +14,11 @@ class HashTest3(Package):
     homepage = "http://www.hashtest3.org"
     url = "http://www.hashtest1.org/downloads/hashtest3-1.1.tar.bz2"
 
-    version("1.2", md5="b" * 32)
+    version("1.2", sha256="bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
-    version("1.3", md5="c" * 32)
+    version("1.3", sha256="cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc")
-    version("1.5", md5="d" * 32)
+    version("1.5", sha256="dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd")
-    version("1.6", md5="e" * 32)
+    version("1.6", sha256="eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee")
-    version("1.7", md5="f" * 32)
+    version("1.7", sha256="ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff")
 
     variant("variantx", default=False, description="Test variant X")
     variant("varianty", default=False, description="Test variant Y")