This changes the hash algorithm so that it does much less object
allocation and copying, and so that it is correct.
The old version of `_cmp_key()` would call `sorted_deps`, which would
call `flat_dependencies` to get a list of dependencies so that it
could sort them in alphabetical order. This isn't necessary in the
`_cmp_key()`, and in fact we want more DAG structure than that to be
included in the `_cmp_key()`.
The new version constructs a tuple without copying the Spec DAG, and
the tuple contains hashes of sub-DAGs that are computed recursively
in-place. This is way faster than the previous algorithm and reduces
the numebr of copies significantly. It is also a correct DAG hash.
Example timing and copy counts for the different hashing algorithms
we've tried:
Original (wrong) Spec hash:
```
106,170 copies
real 0m5.024s
user 0m4.949s
sys 0m0.104s
```
Spec hash using YAML `dag_hash()`:
```
3,794 copies
real 0m5.024s
user 0m4.949s
sys 0m0.104s
New no-copy, no-YAML hash:
```
3,594 copies
real 0m2.543s
user 0m2.435s
sys 0m0.104s
```
So now we have a hash that is correct AND faster.
The remaining ~3k copies happen mostly during concretization, and as
all packages are initially loaded. I believe this is because Spack
currently has to load all packages to figure out virtual dependency
information; it could also be becasue there ar a lot of lookups of
partial specs in concretize. I can investigate this further.
- _cross_provider_maps() had suffered some bit rot (map returned was
ill-formed but still worked for cases with one vdep)
- ProviderIndex.satisfies() was only checking whether the result map
was non-empty. It should check whether all common vdeps are *in*
the result map, as that indicates there is *some* way to satisfy
*all* of them. We were checking whether there was some way to
satisfy *any one* of them, which is wrong.
- Above would cause a problem when there is more than one vdep provider.
- Added test that covers this case.
- Added `constrained()` method to Spec. Analogous to `normalized()`:
`constrain():constrained() :: normalize():normalized()`
Added test for, e.g.:
import spack.pkg.builtin.mock.mpich
import spack.pkg.builtin.mock.mpich as mpich
from spack.pkg.builtin.mock.mpich import Mpich
Among others. These ensure that direct package imports work so that
packages can be extended.
Package repositories now look like this:
top-level-dir/
repo.yaml
packages/
libelf/
package.py
mpich/
package.py
...
This leaves room at the top level for additional metadata, source,
per-repo configs, indexes, etc., and it makes it easy to see that
something is a spack repo (just look for repo.yaml and packages).
unit tests, so tracking tests with sets wouldn't work unless I extracted the
details relevant to the particular test. For now a simple count will work so
using a set was unnecessary anyways.
1. Adding a plugin to keep track of the total number of tests run as well as the
number of tests with failures/errors.
2. Some nose plugins (including xunit which will be added in a future commit)
assign stdout to a stream object that does not have a .fileno attribute.
spack.util.executable.Executable now avoids passing stdout to subprocess (and
always uses subprocess.PIPE)
TODO:
1. Still need to figure out how to activate the plugin (as of now it is
being ignored by nose). Newer versions of nose appear to make this simpler
(e.g. the "addplugins" argument to nose.run)
2. Need to include new version of nose in order to use xunit
